Author Archives: Josh Baran

The End of Mapping is Here… Almost

A recent blog post by Steve Keifer, titled “The End of Mapping (in B2B Integration)”, asks why B2B translators can’t automatically identify and map source-to-target fields. Well, the answer is they can – at least EXTOL’s translators can. Our customers have been saving countless hours on mapping activities using EXTOL’s design-time automation technology for more than a decade!

With the Advanced Automapper feature in our EXTOL EDI Integrator for IBM i (EEI), our customers simply select a reference map, and the Advanced AutoMapper will compare it with other maps using the same files, generating a new map with the appropriate data fields. It is a very effective tool for the majority of customers, who have multiple trading partners, but trade the same or similar documents. Each new trading partner can be on-boarded much more quickly, eliminating most of the manual and repetitive mapping tasks. Continue reading

Increasing Productivity With a Local Eclipse Mirror

We use Eclipse pretty heavily here at EXTOL. In addition to using it as our development environment, we also use a lot of the Eclipse platform and tools in our own product. It really helps our productivity both by making it easier for us to write our code, and by allowing us to reuse 3rd party code instead of writing it ourselves.

However, when new releases are made available, there is a whole lot of downloading going on. Each of our developers has to download the new Eclipse install, install it, and then download whatever other features they need from the update sites. Our Internet connection is pretty fast, but it still takes a lot of time. Time that would be better spent doing our actual work.

First, we started keeping the installs in a shared folder. That worked for just the installs. One person would download it first, and then the rest of us copy it from the shared folder. But that involves having  someone do the initial download and save first. No specific person is responsible for that, so it isn’t done consistently. Or, sometimes someone does it only to find that someone else already did. It also doesn’t solve the problem of the update sites. After installing, each developer still has to download all their features from them directly.

The next thing we tried was to set up a local copy of the update sites. Eclipse has a mirroring tool that can replicate an update site to another location. That was progressing pretty well, but then we thought about how disjointed it was doing manual downloads for the installs, this mirror for the update sites, and having to manage both of them.

I started looking into becoming an actual Eclipse mirror site. Hard drives are cheap, so storage space wouldn’t be a problem. But, we did not want to actually be a public mirror, since that would defeat the purpose of trying to save bandwidth. Then, after looking around a bit, I found this page. I saw that they accepted requests for internal only mirrors. That was perfect. I signed up, and a short time later we had access.

Then came the setup. They make it easy for you by using rsync to do the task, and they also provide a configuration script to get you started. Since the total size was over 100GB, I had to break it into chunks to download over a few nights. In that script, I would enable a few more pieces each night until it was fully synced. From then on, it would just download the changed content each night.

One problem we ran into, though, was that once the whole site was synced, we noticed it was missing some pieces. After further investigation, and an email to the Eclipse webmaster, we found out that what they were calling the full sync was not actually a true full sync. They cut out some of the subprojects because some of the mirror sites were worried about the space that was needed. The webmaster offered to put us on the true full sync, which was much larger. But, by today’s standards, it is still not all that much data. It is currently about 360GB. My laptop that is a year old has more than enough space to contain that. I don’t know what people were worried about.

In any case, our true full mirror is now up and running, and it saves a huge amount of time for us. Now when a new release is available, the rsync automatically gets it. Our developers then download it from our local mirror in seconds, instead of the minutes it used to take. The update sites are also pointed to our local mirror, so those downloads also complete in a fraction of the time it used to take. Finally, our automated build process can also use the local mirror. That lets us set up those builds to automatically get updates, but still have a fast turnaround time.

SOAP Worst Practices Part 3: Not Completely Specifying the Message Format

(This is the third in a series of articles that attempts to expose some of the more common mistakes and misunderstandings people make when implementing a SOAP Web Service.)

This is somewhat related to the previous post about trying to be too flexible. However, usually this problem is accidental, where the previous was intentional. This problem also is usually the result of misunderstanding or misusing a Web Services development tool.

XML has an element type called “any.” It basically means, “Anything that is valid XML is OK here.” It’s a way to make your document more flexible without the burden of defining type extensions. It should be used very sparingly though,  because the point of a structured data format like XML is to define a structure for a piece of data. When you use the “any” element, it completely ignores that point and allows any structure.

One cause of this problem is misuse of a Web service tool. There are many tools available to ease your creation of Web services. There are tools for most programming languages where you can hand it a business object, and it will generate a WSDL that defines an XML version of that object, and even the code to handle transferring it and converting it into the actual object. You just then need to write the code that does the real work. The tool does all the mundane communication work. Continue reading

SOAP Worst Practices Part 2: Trying to Be Too Flexible

(This is the second in a series of articles that attempt to expose some of the more common mistakes and misunderstandings people make when implementing a SOAP Web Service.)

XML is a quite flexible data format. It allows the user many options, such as Choices, Sequences, Complex Types, Type Extension. But the flexibility can very easily be overdone. While it may seem like a good idea to have your service be flexible, it also makes using the service that much harder to use because of all the different formats that need to be handled.

A good example of taking flexibility too far is the SalesForce.com Web Services interface. It takes the idea of Type Extension to the absolute extreme. For those not familiar with Type Extension, see this tutorial. Type Extension can be very useful when you have types that are mostly similar, with just a few extra attributes added onto the extended types. For example, a good usage could be a personnel data model. Continue reading

SOAP Worst Practices Part 1: Non-XML Content

There are a lot of useful SOAP Web Services. Whether it is a simple zip code lookup service, or a more complex order entry system, a well-implemented SOAP Web Service can make machine to machine communication very reliable, efficient, and best of all, easy.

However, there are also a lot of poorly-implemented SOAP Web Services, and they can cause more problems than they intend to solve. This is the first in a series of articles that attempt to expose some of the more common mistakes and misunderstandings people make when implementing a SOAP Web Service.

SOAP Worst Practice #1: Non-XML Content

One of the main reasons to create a SOAP Web Service over a REST one is that SOAP provides a standardized method to publish the message format for your service. The message is in XML, and its structure is defined in the WSDL, or Web Service Definition Language. If you have an XML document that you want to transfer, embed its format in the WSDL and any consumer will be able to know the format to send to it.

But, if you have non-XML data to transfer, SOAP is probably not your best choice. Continue reading