The messages that Data Exchange sends to Kafka are wrapped in an XML wrapper. This wrapper contains meta data around the transformation and the transaction being processed. It also contains the details of the Primary keys associated with the database, and the data related to the Before and After images, depending on what action is being performed on the record in the database. The Before and After images are in JSON format.
The following is a sample transformation and the XSD generated for it:
Note: This example includes only the keys and a selection of data fields of the type string, decimal, and null.
<xs:element name="EVENT"> <xs:complexType> <xs:sequence> <xs:element name="GLB_DTIME"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:maxLength value="24" /> </xs:restriction> </xs:simpleType> </xs:element> <xs:element name="ACCBAL"> <xs:simpleType> <xs:restriction base="xs:decimal"> <xs:fractionDigits value="2" /> <xs:totalDigits value="10" /> <xs:minInclusive value="-99999999.99" /> <xs:maxInclusive value="99999999.99" /> </xs:restriction> </xs:simpleType> </xs:element> <xs:element name="ACTMTH"> <xs:simpleType> <xs:restriction base="xs:decimal"> <xs:fractionDigits value="0" /> <xs:totalDigits value="4" /> <xs:minInclusive value="-9999" /> <xs:maxInclusive value="9999" /> </xs:restriction> </xs:simpleType> </xs:element> <xs:element name="CUSTOMER"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:maxLength value="6" /> </xs:restriction> </xs:simpleType> </xs:element> ….. <xs:key name="EVENT_FPRIMEKEYEVENT"> <xs:selector xpath="EVENT" /> <xs:field xpath="GLB_DTIME" /> </xs:key> </xs:element>
Note: If a field can contain Null then the XSD will have
<xs:element name="ANULLFIELD" nillable="true"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:minLength value="0" /> <xs:maxLength value="5" /> </xs:restriction> </xs:simpleType> </xs:element>
In an XSD,
The field ANULLFIELD can contain NULL values (hence the field is nillable). If the value is null, the JSON sent will be {“ANULLFIELD”:null}.
The primary key is in the key part of the XSD schema. These are the fields that uniquely identify the record and are the fields that go into the KeysOfTarget element. On the OS 2200, the Primary key fields are in the JSON messages. The actions that locate a record (Update, Merge, Delete) are in the BeforeImage.
While the XSD has all the fields of the table, it does not mean that all the fields will be in the generated JSON message. The XSD is there for building the transformation.