Tag Archives: schema

EXTOL’s Migration Assistant

EXTOL’s Migration Assistant provides a path for the users to upgrade the schemas they’re using in a Transformation in place.  The Migration Assistant is a first class archivable object that can be found in its own node in the Workbench.  This feature works for all syntax categories.  Use cases would be an upgrade of an EDI version (i.e. 4010 850 to 5010 850), upgrade of an .xsd (i.e. elements/attributes added or removed from an xml schema or other structural changes), changes made to a database (i.e. tables or columns added or removed), upgrade of a delimited flat file (i.e. fields added or removed), or the upgrade of a Spreadsheet (i.e. cells or tabs added or removed).

The Migration Assistant is a two step process.  First the user is required to create a map called a Schema Association Map between the “Old Schema” and the “New Schema”.  This map is created by using a Ruleset like editor called the Schema Association Map Editor.  This can be done manually or it can use the Auto Association Algorithm.  The Auto Association Algorithm is used by toggling the “Auto-Associate Children” button in the toolbar and dragging a desired source node to a desired target node.  All child nodes of the desired source will be auto associated with all child nodes of the desired target.  The recommended Associations are displayed in an Approval Editor where the user can review and change Associations.  The suggested Associations are ranked by a confidence level.  It is recommended for schemas that are drastically different that if it is desired to use the Auto Association Algorithm that the user start at lower level source elements that they are confident belong with lower level target elements and work their way up to the root.  For schemas where the changes between the “Old” and “New” schemas are reasonable (no huge structural changes) it is permissible to associate the two roots and let all of their children be auto associated, but one should carefully review the results.

The second step to the process is to apply the Schema Association Map to the desired Ruleset.  This can be applied in a batch mode (used when a schema is being used in multiple Rulesets) or it can be applied to a single Ruleset.  The Schema Association Map can be applied to a single Ruleset by right clicking on a Ruleset in the Rulesets tab and clicking Convert Ruleset->Source/Target.  A dialog will be displayed asking the user to select which Schema Association Map they want to apply.  Once the proper Schema Association Map is selected the Ruleset Conversion Editor will be displayed.  This is a diff like editor that will allow the user to review and apply their desired changes as dictated by the Schema Association Map.  To initiate the batch conversion mode the user can right click on the desired Schema Association Map and select Convert Rulesets.  A dialog will be displayed listing all of the Rulesets that the selected Schema Association Map can be applied to.  For each desired conversion the user must select whether they want to review their changes with the Ruleset Conversion Editor or if they want to automatically apply the changes dictated by the Schema Association Map to the Ruleset.

The hope is that the solution is flexible enough so that for simple cases there is very little for the user to do but is able to handle the really difficult cases as well.  Hopefully this has been a helpful and informative guide to upgrading your schemas in EBI.

Dealing with the Pain of Version Upgrades

If you’ve ever dealt with changes to a working version of a schema, whether it is database, EDI, XML, or whatever format your data may be in, then you know how painful it is.  In most shops in the typical data processing scenario, either a tool or a custom program is used to process the data in one format and convert it to another format to be piped off for further processing somewhere else.  The most difficult to deal with example can be changes to an XML schema.  The reason is that XML is so extensible and just about anything can be done with it.  The contrasting example would be EDI data where the changes are usually miniscule and the structure itself does not vastly change.  The typical example that most IT shops face is a change in a database which could be the addition of a table or column, the deletion of a table or column, the moving of a table or column, or a change in table/column properties.

If we look at this from the perspective of a model, a schema is really a tree or graph (depending on whether it’s recursive) with entities representing the schema structure. Continue reading