Using JAXB without a schema

A Yahoo HotJobs API project has been my first exposure to REST APIs. Usually, I'm working with SOAP web-services. Bullhorn is actually thinking about trying to support REST with its own APIs, so this was a good opportunity to learn about the strengths and weaknesses of REST.

With a typical SOAP API, my strategy would look something like:

Find the link to the WSDL in their documentation.
Use the wsimport command to create JAXB stubs for the web-service.
Copy the stubs over to my project.
Start coding some unit tests to make sure they work, and you got your prototype.

First problem: there is no WSDL in REST. Instead, they use a similar WADL standard. There is even a wadl2java tool to create stubs.

Second problem: this actual web-service doesn't support WADL. In fact, many REST web-services don't support it. The main problem is that there is no codified standard for REST; it evolved to its current state. WADL was an after-thought, and is still gaining traction.

So, I needed to produce XML and post it to this API somehow. My initial implementation was to create the XML by hand, and post it to the API using Apache's HttpClient. HttpClient is great, because it supports all the standard HTTP methods (POST, GET), plus the less frequently used ones needed for REST (PUT, DELETE, etc).

For the XML part; I had been looking for a reason to play around with StringTemplate. I started with a template XML file that looked like:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<atom:feed xmlns:yheader="http://schemas.yahoo.com/ypost/jobsHeader/3.0"
   xmlns:atom="http://www.w3.org/2005/Atom"
   xmlns:yjob="http://schemas.yahoo.com/ypost/jobs/3.0"
   xmlns:ycontrol="http://schemas.yahoo.com/ypost/control/1.0">
   <yheader:Credential>
       <yheader:Login>$login$</yheader:Login>
       <yheader:Password>$password$</yheader:Password>
       <yheader:Version>3.0</yheader:Version>
       <yheader:LicenseKey>$license$</yheader:LicenseKey>
   </yheader:Credential>
   <atom:id>$account$</atom:id>
</atom:feed>

... and a code segment that looked like:

    private static final String AUTH_XML_FILE = "HotJobs.template.auth.xml";
    private static Map<String, String> replacements = new HashMap<String, String>() {{
            put("login", "user@user.com");
            put("password", "password");
            put("account", "12345");
            put("license", "abc123");
        }};

    public static String getXml() {

        StringTemplate xml
                = new StringTemplate(getFileContents(AuthRequest.class.getResourceAsStream(AUTH_XML_FILE)));

        for (String key: replacements.keySet())
            xml.setAttribute(key, replacements.get(key));

        return xml.toString();
    }

    public static String getFileContents(InputStream resourceAsStream) {
        // left as an exercise for the reader ;)
    }

Pretty soon, I ran into into a situation where I was posting invalid XML. Of course, I should have realized that you can't just cram any string into an XML element; it may contain an invalid character, such as the ampersand. Or it may just be the wrong encoding. The hack fix is to escape the values:

   // using org.apache.commons.lang.StringEscapeUtils
   xml.setAttribute(key, StringEscapeUtils.escapeXml(replacements.get(key)));

However, this is really a symptom of a poor design. In general, it's not a good idea to generate XML by hand like this.

I set out to use JAXB, instead. With no schema provided by the vendor, I would need to make one. I loaded up my trusted copy of XMLSpy, and copy and pasted an example XML document. Then I selected DTD/Schema -> Generate DTD/Schema:

There were a few tweaks necessary. String fields came across as enumerations, which is somewhat understandable because XMLSpy has to guess at what types the fields are. I couldn't find a configuration option to change it, so I edited it by hand:

<xs:element name="id">
 <xs:simpleType>
  <xs:restriction base="xs:string">
   <xs:enumeration value="1-JYNURD"/>
  </xs:restriction>
 </xs:simpleType>
</xs:element>

became...

<xs:element name="id">
 <xs:simpleType>
  <xs:restriction base="xs:string">
  </xs:restriction>
 </xs:simpleType>
</xs:element>

Then, all I needed to do was to generate the JAXB stubs. Xjc handled the multiple chained XSD files just fine.

xjc -d generated -p com.bullhorn.athens.jobboards.hotjobs.generated.login login.xsd

Once I plugged this into my existing implementation, it almost worked. By default, JAXB was marshalling the objects into XML without the custom namespaces. For example, "yheader" was becoming "ns1", while "atom" was not namespaced at all. The namespace declarations were properly changed to match, so it was valid XML, just not what the API was expecting.

This would work for many REST APIs, but this particular one doesn't seem to be using an XML parser on the other end. I assume they are parsing the XML by hand using regular expressions or something. This is another good argument for using XML parsers versus doing it yourself!

Fixing the namespace issue was easy. All you have to do is provide an implementation of NamespacePrefixMapper. In my case, it looked like:

import com.sun.xml.bind.marshaller.NamespacePrefixMapper;

public class YahooNamespacePrefixMapper extends NamespacePrefixMapper {

    public String getPreferredPrefix(String namespaceUri, String suggestion, boolean requirePrefix) {

        if (namespaceUri.equalsIgnoreCase("http://www.w3.org/2005/Atom")) return "atom";
        return "yheader";
    }
}

Then, in my marshalling code, you have to pass in the custom prefix mapper:

    public static String marshall(Class className, Object value) {

        try {

            JAXBContext jaxbContext = JAXBContext.newInstance(className);
            Marshaller marshaller = jaxbContext.createMarshaller();
            marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
            marshaller.setProperty(NAMESPACE_PREFIX_MAPPER, new YahooNamespacePrefixMapper());
            ByteArrayOutputStream bytes = new ByteArrayOutputStream();
            marshaller.marshal(value, bytes);
            return bytes.toString();

        } catch (JAXBException e) {
            throw new RuntimeException(e);
        }
    }

Success! Of course, generating a schema by hand is not ideal, either. But if there is no first-party schema, at least it's an option.

Chase Seibert