Processing EDI Files in Java

Excellent Video on EDI Processing:

In Java there many library available to process large EDI file especially EDI Health Care transaction related files. One particular Library really stands out is smooks.

Lets Say :

We have to transform huge EDI files into single data sets (XML). Huge means up to 700K ‘data sets’ which is a file size of 250MB. Each file has a global header and n data groups. Each data group has global (on their) level information as well and n data sets. The file structure looks like this:

file |-- header [1] |-- data group [0 - *] | |-- header [1] | |-- data sets [0 - *]

These are the issues:
- transform the EDI into XML
- get the global and parent information into each data set
- split the file into n data sets
- export the data sets to file system or JMS (in this sample we will use file export)

The problem is the size of the file. We can’t transform it inside the RAM but we have to use something like data stream in and data stream out. Here comes smooks into game. Smooks is a library for transforming files. One possible combination is EDI in and XML out (check out the smooks page for more options and samples). To face our issues smooks gives us a rich toolset:
- transform the EDI into XML with a simple mapping language
- get the global and parent information into each data set is easy with mixing dom and sax
- split the file into n data sets can be done by freemarker in combination with dom/sax support
- export the data sets to file system or JMS (in this sample we will use file export) via one of the export cartridges

Camel Code Snippet:

from(“file://target/in?noop=true”)
.process(
new Processor() {
public void process(Exchange exchange) throws XMLStreamException, TransformerException {
Message in = exchange.getIn();
StreamSource source = exchange.getContext().getTypeConverter().tryConvertTo(StreamSource.class, exchange, in.getBody());
if (source != null) {
in.setBody(source);
}
}
})
.log(“starting splitting…”)
.to(“smooks://src/main/resources/file-config.xml”);

from(“file://target/out?delete=true”)
.to(“log:smooks?level=INFO&groupSize=1000″)
.setBody(constant(“”))
.to(“mock:out”);

Please read the complete blog here