Author Archives: Andrew Knott

EXTOL’s Migration Assistant

EXTOL’s Migration Assistant provides a path for the users to upgrade the schemas they’re using in a Transformation in place.  The Migration Assistant is a first class archivable object that can be found in its own node in the Workbench.  This feature works for all syntax categories.  Use cases would be an upgrade of an EDI version (i.e. 4010 850 to 5010 850), upgrade of an .xsd (i.e. elements/attributes added or removed from an xml schema or other structural changes), changes made to a database (i.e. tables or columns added or removed), upgrade of a delimited flat file (i.e. fields added or removed), or the upgrade of a Spreadsheet (i.e. cells or tabs added or removed).

The Migration Assistant is a two step process.  First the user is required to create a map called a Schema Association Map between the “Old Schema” and the “New Schema”.  This map is created by using a Ruleset like editor called the Schema Association Map Editor.  This can be done manually or it can use the Auto Association Algorithm.  The Auto Association Algorithm is used by toggling the “Auto-Associate Children” button in the toolbar and dragging a desired source node to a desired target node.  All child nodes of the desired source will be auto associated with all child nodes of the desired target.  The recommended Associations are displayed in an Approval Editor where the user can review and change Associations.  The suggested Associations are ranked by a confidence level.  It is recommended for schemas that are drastically different that if it is desired to use the Auto Association Algorithm that the user start at lower level source elements that they are confident belong with lower level target elements and work their way up to the root.  For schemas where the changes between the “Old” and “New” schemas are reasonable (no huge structural changes) it is permissible to associate the two roots and let all of their children be auto associated, but one should carefully review the results.

The second step to the process is to apply the Schema Association Map to the desired Ruleset.  This can be applied in a batch mode (used when a schema is being used in multiple Rulesets) or it can be applied to a single Ruleset.  The Schema Association Map can be applied to a single Ruleset by right clicking on a Ruleset in the Rulesets tab and clicking Convert Ruleset->Source/Target.  A dialog will be displayed asking the user to select which Schema Association Map they want to apply.  Once the proper Schema Association Map is selected the Ruleset Conversion Editor will be displayed.  This is a diff like editor that will allow the user to review and apply their desired changes as dictated by the Schema Association Map.  To initiate the batch conversion mode the user can right click on the desired Schema Association Map and select Convert Rulesets.  A dialog will be displayed listing all of the Rulesets that the selected Schema Association Map can be applied to.  For each desired conversion the user must select whether they want to review their changes with the Ruleset Conversion Editor or if they want to automatically apply the changes dictated by the Schema Association Map to the Ruleset.

The hope is that the solution is flexible enough so that for simple cases there is very little for the user to do but is able to handle the really difficult cases as well.  Hopefully this has been a helpful and informative guide to upgrading your schemas in EBI.

An Overview: Class and Object Design

Class and object design can be a tricky thing.  It’s not really something that can be taught or quantified.  This skill is developed through years of experience.  This may be more of an art than it is a science.  No expert can come up with an exact formula on how this should be done but all experts can pretty much agree on what is a bad design.

A possible starting point of a process could be to develop a written problem statement outlining the issue at hand.  This can be done by interviewing a customer or whoever the requirements are coming from.  Once the problem statement is developed you will find that the nouns in the statement will turn into classes and the verbs in the statement will turn into operations.  From there relationships between the classes can be derived.  Statements like “is a” can denote inheritance, “has a” can denote composition, “uses” can denote delegation, etc.

A UML diagram should be drawn modeling what was discovered by the problem statement.  It is also good to keep a catalogue of design patterns at hand so that a given pattern may be applied to a module of the software system.  Remember that Object Oriented Software is about loose coupling, encapsulation, and reuse, which design patterns help enforce.  Once the initial diagram is developed it is really more of a guideline than concrete fact at this point.  When the implementation process begins new facts will come to light that haven’t been considered before which will ultimately force the redesign some components of the software.  Once this happens the diagram should be updated to reflect the new functionality.  This will create a software cycle that will bounce back and forth from implementation to redesign approaching a final version of the software.

This is really only a generalized overview of an absolutely monstrous topic in which there are many different paradigms and many different ways to go about tackling the problem.  As a closing thought, keep in mind that a design is never really complete, that it can always be improved upon and made more robust, clearer, and more efficient.

Dealing with the Pain of Version Upgrades

If you’ve ever dealt with changes to a working version of a schema, whether it is database, EDI, XML, or whatever format your data may be in, then you know how painful it is.  In most shops in the typical data processing scenario, either a tool or a custom program is used to process the data in one format and convert it to another format to be piped off for further processing somewhere else.  The most difficult to deal with example can be changes to an XML schema.  The reason is that XML is so extensible and just about anything can be done with it.  The contrasting example would be EDI data where the changes are usually miniscule and the structure itself does not vastly change.  The typical example that most IT shops face is a change in a database which could be the addition of a table or column, the deletion of a table or column, the moving of a table or column, or a change in table/column properties.

If we look at this from the perspective of a model, a schema is really a tree or graph (depending on whether it’s recursive) with entities representing the schema structure. Continue reading