Maps are the one of the most painful and costly aspects of a B2B integration program. But maps are also the most important part of your B2B program. Maps are the key conduit through which information flows from your enterprise applications (SAP, Oracle, Infor) into the systems of your business partners (customers, suppliers, banks). That means that every time you make changes to the data structures in your enterprise applications you also need to make modifications to the associated maps.
Every time you win a new account, you need to create a new map that converts the information the customer sends into the format of your enterprise application systems. And maps cannot easily be ported from one vendor’s translation tool to another vendor’s. If you want to change translators then you often need to rewrite your maps completely.
Wouldn’t it be great if we could find a way to stop having to use maps? Shouldn’t we be able to develop an algorithm that can automatically map and convert the data from one file format into the format of another? This type of artificial intelligence technology is certainly within reach (if not already present). Consider that:
- IBM’s Watson can interpret natural language queries and respond with answers faster than even the world’s smartest Jeopardy players.
- Siri can interpret a wide variety of human speech patterns and respond within seconds with answers to most common questions.
- Shazam can listen to a 30 seconds of music playing on a radio and match it the artist and title of the song.
- Google Goggles can scan an item via a cell phone camera and identify it with a high degree of accuracy.
So why can’t B2B translators consume structured files and automatically identify what the data fields in the source file map to the respective fields in a target? Fields like bank account numbers, street addresses, part numbers and unit prices follow a relatively consistent pattern that should be easily recognizable. Optical Character Recogition (OCR) software is able to do this today. OCR applications can scan and match fields on document such as invoices and paper checks with 90% or greater accuracy.
It is time to replace the expensive and time consuming process of mapping with an algorithm. But how would you build such an algorithm? The algorithm should require minimal inputs. For example, the algorithm might be a RESTful API that requires only the name of the receiver, type of transaction and the input file format. For example, you should only need to tell the algorithm that you want to send an invoice to Walmart from the attached EDI file. Or that you want to send a wire transfer instruction to HSBC via the attached SAP Idoc.
The algorithm would maintain a library of known blueprints for translating documents that it builds up over time. For example, the algorithm might have a set of blueprints for converting wire transfer instructions into the respective HSBC format. And it might have a set of blueprint for converting invoices into Walmart’s preferred format. Of course, there will be new translation scenarios for which no blueprints exist. In these scenarios the algorithm will need to auto-magically identify the input fields and correlate them to the associated output fields. This could be done by reading the tags next to each data field. For example, “Invoice #” next to a field is a dead giveaway.
The key to success is building such an algorithm will be having enough sample data to test and optimize it until it is 99.999% accurate. We can take some lessons from companies such as Facebook and Google here. How did Google develop its online “translation” capabilities? They identified hundreds of books which had been reprinted in different languages. Google used these known associations between words and phrases from the books in different languages to build their language translation algorithm. How did Google develop its speech recognition capabilities? Google offered free “411” information lookups to gather a wide sample set of human speech patterns that it could use to build its voice recognition software.
Cloud-based B2B integration providers could take a similar approach. They already have massive amounts of transactions that are being processed. These same transactions could be replayed in a test environment to refine and optimize an algorithm. It could take several years to optimize an algorithm, but whoever can crack the code on this first will be able to save their customers thousands of hours of mapping activities every year.