In Data Exchange errors can be categorized as
Recoverable errors: These errors occur when there is a network connection failure or service interruption. You can configure the recovery settings in Data Exchange Administration Site > Settings> Recovery. Even though these errors are configured to retry, they always cause the transformation to stop. Data Exchange then internally tries to restart the transformation (if the retry option is selected) after the retry interval.
Data errors during transformation: These errors occur during the process of transforming the data from the source to the target. These are related to the actual data record and cannot be recovered. You can configure to handle such errors in two places:
In DDW, using the “Data Transform Error Handling Policy”. These errors occur during the data transformation phase in Data Exchange. You can set the policy to stop the transformation when the error occurs or to continue the transformation by truncating or rejecting the data. For more information about setting this policy, see Data Transform Error Handling Policy.
The error messages are logged in the Runtime Service log (DataExchangeRuntimeServiceLog.txt) and the truncated or rejected data is logged in DataTransformErrorLog.txt.
An example of message logged in DataTransformErrorLog.txt is as follows:
CRITICAL DataLossLogger - Message:Data truncation happened for transformation TransformDJOFFICE with message header HostMessageHeader { SourceDBType = DatabaseType.DMS OpFlag = Operation.Delete TableName = COUNTRY TimeStamp = SchemaName = DJOFFICE !inTransState !IsLongTrans SourceDB = acus1.rsvl.unisys.com||APP016 }.
Truncation Detail: Target value 8012345678 is truncated from source calculated value '80123456789'
Source Classifier: DJOFFICE.INDIVIDUAL {IND_ID(Key) = '8',IND_NAME = 'John Doe',IND_ADDR = 'Sydney',PHONE_NO = '100008'}
Target Classifier: DJOFFICE.INDIVIDUAL
Source features(s): IND_ID AlphaNumeric(10)
Target feature: IND_ID string
Expression: <<IND_ID>>+"0123456789"In Administration Site, using the “Target Commit Error Handling Policy”. These errors occur when Data Exchange tries to commit the transformed data into the target data store. You can set the policy to stop the transformation or ignore the error and continue the transformation. If the transformation stops, a message is logged in the Administrative Service log (DataExchangeRuntimeAdministrativeServiceLog.txt). The command that caused the target commit error (stopped or ignored) are logged in TargetCommitErrorLog.txt.
An example of message logged in TargetCommitErrorLog.txt is as follows:
WARN TargetCommitLogger - Message:Data rejection detected for transformation TrasnsformationMockName when committing to target Target Database: TargetHostName\TargetDBName Command Executed: INSERT INTO TB1(C1) VALUE(1); Reason: Primary key violation
Note: Target Commit Error Handling Policy is applicable only for SQL Server or Oracle as the target. It is not applicable for Enterprise Database Server or Kafka as the target.
Data Exchange handles the following types of target commit errors:
Target Data Violation Error: This includes Primary Key Violation, Foreign Key Violation, Not Null Constraint Violation, Unique Constraint Violation, and Check Constraint Violation. This type of error is non-recoverable; the error handling options are to either stop the transformation or to ignore the data record that caused the error and continue the transformation.
Target Deadlock Error: When a deadlock error occurs, Data Exchange can retry without stopping the transformation. If the retry is successful, then the transformation will go on without the user noticing any issues on the surface. Otherwise, you can choose to stop the transformation or to ignore the data record that caused the deadlock and continue the transformation.
Other Error: This is the type of target commit error for which Data Exchange cannot determine the cause. The options are also to stop the transformation or to ignore the data record that caused the error and continue the transformation.
Note: While all errors are logged in the respective log files, in the event of a transformation being started, restarted, or stopped, an event entry is added to the Windows event log.