Data transformation versus data translation
When talking about integration, whether it is application to application (A2A), business to business (B2B) or data and cloud integration, the goal is to exchange data, documents, information, and/or messages between a source and a target. To accomplish this goal, we identify business patterns related to the sources and targets of these “integrations” and then apply the identified patterns to similar scenarios that meet specified criteria. Most, if not all of these patterns will require data mappings, also known as “maps” or “rulesets”, at some point during the integration process. These maps are ultimately used to perform one of two operations: to transform or translate data. I often use the terms, transform and translate, interchangeably. But, is there more to them? Are these terms synonymous, or do their meanings differ?
Let’s look at this a little closer to see if there is a difference. Data translation can be defined as the process of converting data from one syntax to another and performing value lookups or substitutions from the data during the process. Translation can include data validation as well. One example of data translation is to convert EDI purchase order document data into purchase order database files or even flat files while performing data validation on the source data.
In defining data transformation, the attributes are a super-set of data translation, and include:
- Data enrichment – populating target document data based on formulas or complex expressions to improve the source data.
- Data routing – Directing data to its target based on where it came from or its content.
- Data-driven process initiation – Activating business processes based on data content. This can occur with data routing, but can occur within the “map” as well. This also includes initiating web services to assist with the transformation.
- Multiple outputs – Producing multiple output documents, such as writing to an xml file while also updating a record in a database table.
An example of data transformations is routing a purchase order to a specific process that will perform a data translation to convert the shipping and billing address information to an invoice document. We can also use the translation example from above, transforming an EDI purchase order to a database and then during this transformation, also perform additional processing using formulas such as determining a total number of items or total dollar amount by looping through specific data constructs. Other actions that can be performed during the transformation process include invoking web services and calling a process. Another transformation example may be to convert character data from one character-encoding scheme to another.
It looks like we were able to answer the question. Data transformation is not only data translation, but a lot more. Data translation is limited to data operations, whereas data transformation combines data operations and process control in a single model.