Make your own free website on Tripod.com

How-To Articles JavaScript Apostle Human Interface Online Feature Articles Case Studies Editor's Corner Third-Party News

ELECTRONIC DATA INTERCHANGE ON THE INTERNET

By Benoît Marchal

Send comments and questions about this article to View Source. 

A quick look at the out box of any accounting department shows that many documents -- including invoices, orders, tax and customs declarations, delivery instructions, and checks -- are printed by computers. These documents are produced by the accounting package, printed, put in envelopes, stamped, and mailed to the recipients. The weird thing is that upon reception, most of these documents will be reentered into another accounting package for further processing.

All this printing, stamping, and encoding is a waste of time and money, as well as a source of errors. Multiple entry of the same information increases the risk of typing errors, such as adding or deleting a 0, inverting two characters, or entering an incorrect date.

Wouldn't it be easier and cheaper to electronically exchange documents among the various accounting packages of the world? Wouldn't it be faster as well? Imagine that instead of printing an invoice, a supplier sends it via e-mail. Upon reception the e-mail would automatically be processed by the accounting package. On the due date, another electronic message would be sent to the bank with transfer instructions. The only human intervention would be to create the original invoice and to accept it. The customer wouldn't risk paying an incorrect amount, and the bank couldn't make a mistake when processing the check.

ELECTRONIC DATA INTERCHANGE

Does this sound like science fiction? It's not! Electronic Data Interchange, or EDI, has been around for more than 20 years. EDI has been widely deployed and used worldwide. It has spawned a large and lively industry and enabled significant cost savings.

Banks exchange more than three million EDI messages per day internationally through the SWIFT banking network. If you sent money to a foreign country recently, chances are it resulted in a SWIFT message. The automotive industry is another champion of EDI. By reducing the time to process orders, EDI has enabled cost savings through just-in-time management of automotive parts. The transportation industry, which continually exchanges routing and customs documents, has also embraced EDI. Major computer manufacturers such as Compaq depend on EDI for a growing number of their transactions.

Many U.S. government organizations accept administrative documents in electronic format. Likewise, the European Commission has sponsored EDI efforts in a variety of sectors, such as TESS in the social security sector.

Even though it may not be as well-known as the web, and it definitely lacks the web's glamor, EDI has been and still is one of the major business applications of intercompany computer networks.

EDI AND THE INTERNET

There is a natural connection between EDI, the electronic exchange of business documents, and the Internet, the largest computer network. This article will review various efforts to merge these technologies. We'll identify four major stages in the development of EDI over the Internet:

  1. Internet mail
  2. Standard ICs
  3. Web-EDI
  4. XML/EDI
Web-EDI is the most popular option; it's the first approach that comes to mind when a company thinks about EDI over the Internet. Still, the other options should not be ignored. As we'll see, Web-EDI poses some challenges and suffers from limitations that a new approach, XML/EDI, attempts to address.

PRE-INTERNET EDI

Before moving to the specifics of EDI on the Internet, let's review the traditional EDI approaches.

EDI was initially developed in a very specific context. When EDI started, computers were expensive and 300 bps was the norm for modems. Few companies could afford computers, and those that could were large. Software packages were few and most applications were custom-made. All these factors had a major influence on EDI standards.

EDI standards are essentially directories of standard messages for most business needs -- for example, orders, invoices, customs declarations, statistics, insurance documents, and bills of transport. You can think of these standard messages as the electronic equivalent of the preprinted forms that can be bought in stationery stores.

Industry groups, national and international standards bodies, and ad hoc committees have developed numerous standard messages. The effort has culminated in the international UN/EDIFACT standard, endorsed by the United Nations.

Unfortunately, in the business world exception is the rule. When filling in an order form, a sales representative is likely to add comments; since these comments reflect accepted business practices, standard EDI messages must support them in order to be complete and useful. Realize, too, that as far as international standards are concerned, each country has its own set of exceptions. No wonder the resulting messages include many options that ultimately make them very complex.

In practice, to implement an EDI solution, the various business partners must not only pick the appropriate standard messages but also must tailor them to their needs, removing the fat they don't use. The resulting implementation convention (IC) is their tailored version of a standard message. Developing ICs is costly because it typically requires the work of highly trained analysts.

The existence of multiple ICs, essentially one for each set of business partners, also means that no two implementations of the same message are interoperable. On the Internet, we take it for granted that any web browser works against any web server. The equivalent is not true for EDI: an EDI-enabled package will work only with those packages that implement the same ICs.

The Problem of Mapping

One of the most costly steps in implementing EDI is known as mapping. Information is never stored as EDI messages; the messages are used solely for intercompany exchange of documents. Internally information may be stored in relational databases. To create standard messages, companies must map from their internal representation to standard messages. Mapping is not trivial because the internal representation often has little in common with the record-oriented structure of standard EDI messages. EDI vendors have produced tools, like Mercator from TSI Software, that simplify mapping.

Example 1 is a UN/EDIFACT order message. The funny syntax is a legacy from the past, but you shouldn't be concerned with that. The message is organized into many small records, called segments. Each segment is prefixed by a three-letter identifier. For example, the date segments are prefixed with DTM. As you can see, the mapping from a relational database would not be straightforward.


Example 1

UNB+UNOB:1+003897733:01:MFGB-PO+PARTNER ID:ZZ+970101:1050+00000000000916++ORDERS'
UNH+1+ORDERS:S:93A:UN'
BGM+221+P1M24987E+9'
DTM+4:980101:101'
RFF+CT:123-456'
RFF+CR:1'
NAD+SE+8049P::92++ACME INC'
NAD+BT+B2::92++TOON TOWN'
CUX+2:USD:9'
PAT+1++1:1:D:45'
PAT+22++1:1:D:30'
PCD+12:2'
TOD+2+CC+:::ORIGIN COLLECT'
LIN+000001++107315-001:BP'
IMD+F+8+:::TOON EXPLOSIVE'
QTY+21:10000000:PCE'
UNS+S'
UNT+29+1'
UNZ+1+000000000000916'

Mapping is further complicated by code lists. Codes are short language-independent symbols that represent language-dependent strings. For example, the ISO has published a standard list of country codes: US is the code for the United States, CA is for Canada, BE is for Belgium, and so on. Companies may use code lists internally that have no direct mapping to the standard. Going back and forth may result in loss of data. For example, if a company uses five codes to identify dangerous goods, how can they be mapped to a standard that may only recognize three codes?

Security Concerns

EDI is typically done through value-added networks (VANs). Besides performing typical network services, VANs play the role of clearinghouses by guaranteeing the security and delivery of messages -- but they charge a premium rate for their effort.

Security is critical for business documents, including EDI messages. Imagine a customs declaration being lost; the goods will be rejected by the customs administration. Likewise, EDI documents may contain sensitive data. For example, it's possible to predict the short-term strategy of a company from its ordering pattern.

The Cost of Doing EDI

We've looked at only some aspects of EDI here, but it should be clear that costs are high. To break even, the savings must be sizable as well. Obviously the amount of savings depends on the number of documents exchanged; large companies that process numerous business documents can offset costs in a relatively short period of time.

On the other hand, small- and medium-sized enterprises (SMEs) find it harder to justify EDI. Indeed, in practice, EDI has developed under the momentum of larger companies. Most SMEs who are using EDI have done so under the pressure of a larger business partner.

The world has changed in 20 years. On the technical side, more SMEs are equipped with PCs and have access to networks, thanks to the Internet. On the business side, the role of SMEs is growing. SMEs have more interaction with large companies to buy or sell goods and services; this results in a greater flow of business documents between them.

The challenge of EDI on the Internet is to lower the cost of doing EDI enough to empower SMEs with EDI tools, enabling savings for the whole business community.

INTERNET MAIL

One of the first approaches to bringing EDI to the Internet has been, logically enough, to replace VANs with Internet service providers (ISPs), which are cheaper. Unlimited connection to the Internet can cost as little as $20 a month, whereas some VANs still charge per volume as well as per connection.

The main issue in doing EDI on the Internet is the same issue that limits electronic commerce in general: security. Specifically, SMTP, the Internet mail protocol, lacks:

The EDIINT working group from the Internet Engineering Task Force (IETF) has published several recommendations for the use of e-mail and HTTP for secure EDI over the Internet. Several vendors support this solution, including Netscape.

Management of certificates and the legal responsibilities associated with them remains a challenge for many implementations. The development of certificate authorities, such as Verisign, is a step in the right direction.

For multinational EDI communities, non-American partners pose an additional challenge. For legal reasons, U.S. software vendors do not export strong-encryption software. The workaround is to use software developed outside the U.S., but such software is less popular and less widely deployed.

Cheap, secure bandwidth is a prerequisite to larger adoption of EDI, but Internet mail as a way of doing EDI doesn't resolve the other cost-related issues of ICs and mapping.

STANDARD ICS

Having standard ICs tackles the first of the two remaining issues. Recognizing that EDI standards are too complex for many applications and that having a large number of options makes competitive implementations noninteroperable, various standards bodies have worked to produced simpler, leaner, and completely defined EDI messages targeted for very specific applications. Unlike other standard messages, these specifications leave no room for options, alleviating the need for ICs. Ultimately, application vendors will be able to package these standards with their products.

One successful example is Open Buying on the Internet (OBI). OBI has very precisely standardized a set of messages for high-volume, low-dollar transactions, such as transactions for office and laboratory supplies and maintenance materials. These transactions, which have little strategic significance but account for more than 80% of all transactions, are ideal targets for cost savings through EDI.

OBI has gone one step further: along with messages, it standardizes scenarios, such as when to send a particular message and what to reply. In practice, OBI integrates electronic catalogs with EDI messages to propose a complete solution for buying goods in a business-to-business environment over the Internet. Netscape CommerceXpert is an example of an application that implements OBI.

WEB-EDI

Web-EDI is probably the best-known approach for doing EDI over the Internet. Most major EDI vendors, such as FORESIGHT Corporation, offer or are working on a Web-EDI product.

Web-EDI recognizes that even standard ICs supported by various competing products will not lower the cost of doing EDI enough for those SMEs who process a low volume of business documents. Therefore, Web-EDI moves the cost of doing EDI where the savings are -- that is, to larger companies.

The goal of Web-EDI is to enable small businesses to participate in the EDI exchange with only a browser and an Internet connection. Unlike all the other approaches so far, Web-EDI is asymmetric: one party supports the cost of implementing EDI and reaps full benefits from it; the other party can participate in the EDI exchange but will not benefit from EDI integration. In particular, there's no need for either ICs or mapping. (Unfortunately, integration with the local business system is also nonexistent; we'll see a solution to this problem in the next section.)

Web-EDI, illustrated in Figure 1, is a web interface for EDI messages. One of the partners, typically the larger company, develops or buys web forms for every EDI message it accepts, and customizes these forms according to its own IC. When installed on a web site, the forms become an interface to the EDI system. Other business partners, usually smaller partners, log into the web site and select and fill out the forms of interest to them. The result is submitted to the web server, where a server-side process (CGI, WAI, or Java servlet) validates it and packages it as an EDI message. >From that point on, the message is routed as a normal EDI message. To support the return path (when messages go from the web site owner to the other partners), the messages are either converted to clear-text e-mail or to a web form.


Figure 1. Web-EDI

In other words, all the cost associated with an EDI transaction occurs only once -- on the web site -- for all the business partners.

Let's look at an example. Acme Ltd. wants to send an invoice to Toon Town Inc. The Acme accountant connects to the Toon Town web site and then follows a link to a closed extranet section. After logging onto the extranet and being presented with a list of options, the accountant selects the option to create an invoice. The next screen is an HTML form, where he fills in the details of the invoice. When submitted, the form is transparently packaged as an EDI message by the web server and sent to Toon Town's accounting application for processing. Upon accepting the invoice, the Toon Town application generates a receipt EDI message. This message is routed back to the web server, which converts it into readable e-mail that is then sent to the Acme mail account.

Clearly, this solution is affordable even for SMEs. A ubiquitous web browser and cheap Internet access are the only prerequisites. The costly EDI software and mapping development are taken care of on the server side. In addition, larger corporations enjoy all the benefits of EDI, including a lower error rate and lower cost per transaction.

You may legitimately question the use of EDI messages on the server. Indeed it's possible to bypass EDI messages entirely and connect directly to the enterprise databases from the web site. There are many valid reasons to take the indirection through EDI, though. EDI messages are a well-known, secure, robust, and well-tested interface to enterprise databases. Web-EDI has minimal impact on existing enterprise applications while enabling fast and easy deployment of extranet solutions. In many cases, it's both easier and faster to take an existing byroad than to explore a more direct road. Any other solution may require specific development on the server for the enterprise databases, something that many companies are reluctant to do.

For the existing EDI system, the web site is just a new EDI partner -- or rather, plenty of EDI partners!

XML/EDI

XML/EDI differs from the other approaches outlined in this article in two respects: it's still under development, and it aims to be as simple as Web-EDI while allowing the various partners to fully benefit from an EDI integration.

To achieve its ambitious goal, XML/EDI focuses on the Achilles' heel of EDI: mapping. XML/EDI introduces the notion of templates, a formal definition of the structure of messages. Templates describe not the data but the structure of the message and how to interpret it. They enable programming-free mapping of EDI messages.

On the user computer, a software agent interprets the templates to process the message in the best way, given the environment. If the local application of the user has been XML/EDI-enabled, the agent will automatically build the proper mapping and import the message. Otherwise, the agent may present a web form to the user. Unlike Web-EDI, XML/EDI processes the message on the client side. Thanks to the templates, mapping is automatic and cheap. Figure 2 illustrates XML/EDI.


Figure 2. XML/EDI

Let's take an example. Another accountant for Acme Ltd. connects to the web site of Toon Town Inc. Toon Town has just upgraded its Web-EDI system to a full XML/EDI solution. After logging into the system, the accountant downloads an XML/EDI template that describes how to create the message. Because Acme Ltd. currently has no support for XML/EDI, the accountant also downloads a simple agent that builds a form she can fill in. For Acme this is no different from the Web-EDI solution.

Now suppose Acme upgrades its accounting package to a newer version that supports XML/EDI. The next time the accountant logs into the Toon Town web site, she downloads the XML/EDI templates for invoices. Her accounting package agent takes over (no need to download an agent) and interprets the template. The agent helps the accountant create her invoice. Obviously this is more comfortable and less error-prone than Web-EDI. In particular, product references and descriptions are loaded from the local database.

Thanks to the templates supported by software agents, the user always has the best integration for his or her environment. The templates enable on-the-fly, dynamic integration with local applications.

XML/EDI is a work in progress. The technology basis exists, but most aspects of the implementation are still being refined. Pilots are envisioned for this year. You're invited to join the XML/EDI Group to help develop the technology.

CONCLUSION

EDI enables electronic exchange of business documents. It has been very successful as a cost-saving technology with large organizations, but the price of implementation has been a barrier for smaller organizations.

Although the core technology for EDI has been around for more than 20 years, the Internet has given it new momentum. In particular, the Internet promises to lower the cost of doing EDI, thereby bringing more users to it. Web-EDI is currently the most popular approach for doing EDI on the Internet, but OBI and EDIINT are other options -- and experimental approaches like XML/EDI aim at better adapting EDI to the reality of the Internet.


View Source wants your feedback!
Write to us and let us know
what you think of this article.

Many thanks to Martin Bryan, from the SGML Centre, and Peter Vander Auwera, from CMASS, for their assistance in preparing this article.

Benoît Marchal runs his own consulting company, Pineapplesoft s.p.r.l. His interests include distributed applications, object-oriented programming and design, system programming, handhelds, and computer languages (notably including Java). He also enjoys teaching and writing. He is a founding member of the XML/EDI Group.

(2.98)


DevEdge Online FAQ
Developer Response Center
Join DevEdge Program
Copyright © 1998 Netscape Communications Corporation.
This site powered by: Netscape Enterprise Server and Netscape Compass Server.