As data exchange grows at an explosive rate, moving data between formats and systems occurs in many ways. Common data transformations involve data format conversions between EDI, XML, flat files, databases and more recently spreadsheets. In an application-to-application (A2A) environment, data typically is converted between interface files and back-end databases such as ERP systems. In the EXTOL Business Integrator, we call the objects that facilitate these movements Rulesets.
Simply moving data from a field in the source to a field in the target is not enough in this complex world of ours. Typically, we need to do something more with the source field than simply moving the data. Parsing the data to use a value as a lookup code against a database table or as a parameter in a web service call is very commonplace. Performing calculations based on source and other externally derived values is also typical these days.
Consider other things that can take place in a Ruleset. Most mapping tools operate on data in the sequence that it arrives, such as the segment ordering of an EDI document. Requiring a map to operate on the data in arrival sequence poses challenges that the map designer must accommodate, such as storing values for specific pieces of data to be used later. It would be quite beneficial for a mapping tool to expose the entire document for direct access. That would eliminate the need for sequence-oriented processing and allow direct access to data at any hierarchical level.
Extending the transformation capabilities further would require the ability to pass source data items as parameters to functional components and to map the results into the target data structure. An example of this capability includes string manipulation such as substring processing, mathematical operations or specialized business logic such as pass/fail conditions. Going one step further, the rules that can be created must support conditionalization. For conditionalization to work effectively, data from the source document must be available to be passed as parameters into any conditional functions.
These requirements allow anyone to create Rulesets that can transform data statically or conditionally and perform data enrichment by augmenting source data with external information.
What if maps could be used for more than moving data from sources to targets? There is so much more that we can do with a Ruleset than simply moving data. Consider a Ruleset that inspects the source data and determines how the data should be processed. This is a powerful capability that can be employed to handle a variety of situations. A common example is inspection of an outbound invoice for identifying information such as a customer code and determining what type of transformation should be performed. Most customers may receive their invoices via EDI, but what about lower-volume customers? They may need to receive invoices as spreadsheets. Data Analysis allows the organization to be flexible and accommodate its trading partner’s needs. Another example is the notion of carbon copies. Depending on the trading partner, a copy of the data may also need to be sent to a third-party. This is very common in shipment processing involving a third-party logistics carrier. To complicate matters even further, let us consider the scenario where the 3PL needs to receive the data in a different format? Data inspection and process routing can minimize the impact of these challenges and allow the organization to “be more like their customers.”
The final capability that I would like to address, now that we can inspect data as the map is traversing our data, is the ability to initiate execution. We have discussed performing additional transformations to produce data in varying formats depending on our trading partner’s needs, but what about the challenge of using data to determine what processes to execute? This is a powerful capability that “transforms” our mapping tool from a standard “map data from source to target” tool into a content-based router. This is significant because the tool now has a new dimension to the value that it provides. Execution capabilities are valuable because maps can inspect the data and based on the content make decisions about programs to run, business processes to initiate or even web services to call. Consider how this capability can improve inbound purchase order processing. As orders are received, they are inspected and certain orders (we can use the example of orders for a new trading partner) are routed to an inspection program to make sure that the data received passes the appropriate business logic conditions for pricing or bill-of-material component modularization. Not all of the orders received may need this extensive inspection: we can leverage the advanced capabilities of our mapping tool to route only the orders that need to be reviewed.
With the ability to execute activity, business alerts can also be raised. A simple example is that the CFO would like an email whenever a purchase order over $100,000.00 is received. More complex examples include a web service call within a map to determine product availability at a 3PL warehouse. If the quantity on hand is below an expected level or the order cannot be fulfilled, an alert can be raised that signals to manufacturing that more production is required.
The possibilities are endless but the ability to deliver is very real and available today. Mapping tools are not simple data movers anymore. They provide conditional processing, incorporate additional functionality and can orchestrate execution from within a single tool framework.