Tag Archives: flat file

Assessing the Intermediate Database

The types of databases employed can often determine whether a secondary database is necessary.  This secondary database is typically referred to as a staging or intermediate database, because it resides outside the base application.

Should the company “Enterprise Resource Planning” (“ERP”) application support all required business transactions – both inbound and outbound with customers – then the need to have an intermediate database is lessened.  However, the company ERP might provide support for “core” business transactions but might be limited for “extended” business transactions.  This creates a business problem – where to store the extended business transaction data. Continue reading

EXTOL Business Integrator: Dealing with Proprietary Flat File Data (Part 1)

In my next two blogs, I will be discussing a common challenge facing EXTOL users — the handling of proprietary flat file data received from trading partners. The flat file trend is becoming more popular and, more importantly, being forced on users by their trading partners. We’ve seen a trend where you are either forced to handle the data in the format it is presented or lose the business. Another side of the increase in processing flat file data is to accommodate smaller “Mom & Pop” shops without the means to present the data in a better format.

Let’s explore the different flavors of flat file data. Flat file data can fall into one of two format types: single format or multiple format. Single format files have a common record layout throughout the entire payload data. This means, for example, that every record in the payload has the same exact layout, all of the fields are identified in the same manner in every record and every record of data is treated the same way in the pending data transformation. Multiple format files contain more than a single record layout throughout the payload. This means you will see multiple record layouts needing to be identified and treated as different records in the pending transformation. Continue reading

Selecting a Database: Flat File vs. Relational

Determining the type of database to be deployed for a project is a combination of access requirements and preference. In an effort to make an informed decision on which to deploy, the application engineer should be familiar with the types of databases as well as the pros/cons of each. In this entry we will consider two general types of databases and explore some of their applications and key points in the decision cycle when faced with a project. Continue reading

Our Friend, the Flat File

A common staple in the developer’s diet has been the flat file. These interesting creatures come in as many varieties as there are animals in nature. They have been instrumental in allowing us to send data back and forth between systems and application, often being the lowest common denominator that can be used as a data transport. Let’s explore what these files are and how they came to be adopted as a common basis for how we exchange information today.

First, we should discuss the various flavors of flat files. At the basic level, a flat file is a character string of data.  This long string, ranging from a few bytes to several gigabytes, contains all of the information that the document is trying to convey.

Flat files are used for more than simply containing a single instance of a document. Continue reading

B2B to Be

For my inaugural post on this blog, I want to revisit one of those “solved problems” that still dogs many of the companies we talk with, namely, how to handle B2B integration requirements that don’t involve standard EDI. Companies still find it difficult to cope with the full range of B2B connections and content types needed to integrate with large and small trading partners, including:

  • Standard EDI (and in some cases, EDI that does not fully conform to standards)
  • “Standard” XML, which ranges from well-developed, horizontal standards like RosettaNet to hundreds of loosely-defined vertical transaction sets
  • EDI-like flat file standards (most of these are older, vertically-focused cases)
  • EDI-based web forms
  • Proprietary, partner-defined flat files
  • Proprietary, partner-defined spreadsheets
  • Proprietary, partner-defined web portals
  • Proprietary, partner-defined documents sent by email or fax

Did I miss any? Probably. But the point is that standard EDI is just one of numerous conventions used for B2B integration.  Of course, standard X12 and EDIFACT EDI are still the mainstay of B2B integration. And there is little evidence to suggest that companies are ready to invest in replacing all of their EDI connections with something “better”.  In fact, EDI adoption is increasing.

Continue reading