Data Transformation Mapping – Is it a “Solved Problem”?

Mapping.  We all know how important it is to all kinds of business integration – it defines the data result of an integration process.  Despite the apparent simplicity of drag-and-drop mapping tools, we also know what a pain it can be to get right.  And how the time required to specify and test data transformation maps contributes to the cost of business integration.

Based on years of implementation engagements, EXTOL has determined that mapping can account for up to two-thirds of the time in a medium-to-large project.  The percentage is highest in large conversion projects conducted by companies with many customer relationships.  And the percentage is lower in smaller projects (like onboarding a single partner) and in companies that own value chains and have the power to set the terms of integration with partners.  But even in smaller projects, mapping is usually the single most time-consuming – and costly – design-time activity.

Despite the dominant influence that mapping exerts on life cycle costs, the state-of-the-art for mapping technology has stalled at the classic “line mesh” mapping model, introduced during the early 1990s.  At that time, the simplicity with which data associations could be specified by simply drawing lines between source and target documents was hailed as a marked improvement over coding (which it was), but the line mesh model did not deliver dramatic productivity improvements for complex mapping cases, in which hundreds or thousands of associations and transformations must be specified.

While industry experts concede that the cost of integration remains an important issue, the most time-consuming and costly integration activity – mapping – is treated by most technology vendors as a solved problem.  With the exception of minor advances in user interface design, the problem of mapping cost and productivity has been largely ignored.

The irony here is that integration technology has played such an important role in boosting business productivity.   Automating B2B, application, and data integration processes has enabled businesses of every kind to reduce processing costs, increase business throughput, and reduce errors.  Yet the technologies that have enabled those improvements suffer from productivity deficits of their own.

In my next post in this series, I’ll examine ways in which automation can be applied to data transformation mapping, and how different automation approaches produce very different outcomes.

Leave a Reply

Your email address will not be published. Required fields are marked *