Generating XML Schemas Painlessly and Documenting RESTful Web Services

I have been creating quite a few RESTful web services that respond with both XML and JSON via Spring MVC. Normally with REST, you document the services by example, but in this case someone also wanted an XML schema.

Here was my process to generate XML schemas.

1. Created Java classes annotated with XStream annotations
2. Created example objects from the classes and marshalled them to XML via Spring's XStreamMarshaller
3. Cut and pasted the XML into Freeformatter's XSD/XML Schema Generator
4. Saved the generated .xsd file locally and tweaked it to so as to provide missing information such as:
• Enumerations
• set appropriate minOccurs and maxOccurs attributes on elements
5. Validated my XML and my edited schema file by pasting them into Freeformatter's XML Validator - XSD (XML Schema)
6. Tweaked the Java classes as necessary
7. Rinse and Repeat until done

The process to generate JSON examples:

1. The Java classes were annotated with Fasterxml's Jackson annotations
2. Created example objects from the classes and serialized them using Jackson's ObjectMapper 

To generate the documentation, I leveraged the code that generated the XML and JSON examples and wrote them to individual files on the filesystem. I then created AsciiDoctor .adoc templates that described the services and included the XSD, XML, and JSON files. Running Asciidoctor against the .adoc templates generates nice HTML that includes everything.

iptables

iptables is a Linux utility for managing firewall rules. Some people will directly edit the iptables file (/etc/sysconfig/iptables in CentOS). This can be a bit dangerous, so it often suggested that the command line version of iptables be used. For example, to open port 8080:

iptables -I INPUT -p tcp --dport 8080 -j ACCEPT

This will open port 8080, but only until the next start of the iptables service or reboot of the server which restarts the service. To manually restart the service:

service iptables restart

Any changes to iptables entries that were not saved or configured by editing the iptables file will be lost at restart. To save the current configuration:

service iptables save

Another useful command is to turn off all of the iptables rules without losing those rules. This can be accomplished by flushing iptables:

iptables -F

This will effectively turn off the firewall until the next restart of iptables. This can be very useful for debugging connectivity issues.

Beware Util Classes

Any class that has 'Util' in its name should be treated with suspicion. Util classes are what I call 'trash magnets'. Developers who are too lazy to figure out where a method really belongs are tempted to put all types of unrelated code into Util classes. Often these are static methods which increase coupling, making code brittle, and make classes difficult to test.

There are, of course, exceptions. A StringUtil class or DateUtil class that really only provides utilities for Strings and Dates is acceptable. But even those should be eyed with a bit of distrust as often the same functionality can be found in open source libraries that are more thoroughly thought out and tested. Utility classes almost always break the Single Responsibility Principle by breaking encapsulation. If you absolutely have to create a utility class, at least ensure that all the methods are related to a single concept.

Unfortunately, I often see utility classes that have totally unrelated methods grouped together. For instance, one would expect that a class called StrutsUtil would have utilities that are strictly related to the Apache Struts framework.Finding a method such as this in StrutsUtil is totally unacceptable:

public static long getMaximumWidgets()

Whatever Widget is, it is not a Struts concept. More than likely it is a business concept that belongs either in the domain layer or service layer. Something like StrutsUtil belongs strictly in the UI tier.

Two years from now after the current developer has left the project, some executive may tell the current development team that they have a month to convert their Struts application to GWT or framework XYZ. The team should really have the expectation that they can easily jettison something called StrutsUtil, but to their surprise they find business logic there.

Or a business analyst comes along and tells the team that the downstream system can now handle ten times as many Widgets as before and that the system should be able to easily support that. By all rights the team should expect code related to this to be in something like Widget.java, WidgetService.java, etc.

I've found that worst offending utility classes are named after the application. You can bet that a class called the XyzSystemUtil that belongs to the XyzSystem is going to be full of all types of foulness.

So Little Code - So Many Issues

ResultSet rs = pstatement.executeQuery();

boolean flag = false;



if (rs.next()) {

   if ("Y".equalsIgnoreCase(rs.getString(1)))

       ;

   {

       flag = true;

   }

}



if (flag) {

   // do something

}

How many problems can you find in the above code? I can think of a few.

The egregious issue is the stray semicolon after the condition check. Add two more brackets and it should become obvious what the issue is.

ResultSet rs = pstatement.executeQuery();

boolean flag = false;



if (rs.next()) {

   if ("Y".equalsIgnoreCase(rs.getString(2))) {

       ;

   }

   {

       flag = true;

   }

}



if (flag) {

   // do something

}

The semicolon is treated as a no-op and the conditional check treats that statement as to what should be executed if the condition is true. Because of this flag will always be set to true. I'm assuming that the developer intended that the flag only be set to true if the condition is true, but that is not what is happening.

The second issue may not cause a problem right away, but could easily cause an issue if the SQL query is changed in the future. Relying on the order of columns brought back by a query makes the code less readable and therefore more prone to future errors. Lets assume that the original query is something like:

select EMP_ID, ACTIVE_FLAG  from employees;

If someone comes along in the future and decides that we also need to retrieve the department number like so:

select EMP_ID, DEPT_NO, ACTIVE_FLAG from employees;

and changes the query, but forgets to change the index on the rs.getString(2), then we are going to have a big issue.

Rather than using the column index, it is preferable to use the column name:

rs.getString("ACTIVE_FLAG");

Another big issue is that multiple rows could be returned by the ResultSet, but the code assumes that we are only interested in the first row.

The hard-coded "Y" and bad variable name of 'flag' are also bad practices, but not as bad as the above.

Exploratory Testing: Did a Parameter Object Get Changed?

Exploratory testing is a useful way to find out how legacy code works. If you can write tests that exercise the legacy code in all its paths, you can then feel much safer in refactoring the code. If the refactored code still passes the tests, the refactoring is correct and has not created any side effects.

Sometimes parameter objects are modified inside methods. One way to test for this is to create two instances, pass one in to the method, and then compare the one passed in to see if it is equal to the unmodified one. This assumes that you have a correct implementation of equals().

public class LegacyTest {
    private Legacy legacy;
    private PassedIn passedIn;
    private PassedIn untouched;
    
    
    @Before
    public void setUp()
    throws Exception {
        legacy  = new Legacy();
        passedIn = new PassedIn();
        untouched = new PassedIn();
    }
    
    @Test
    public void passedInDidNotChange() {
        assertEquals(untouched, passedIn);
        legacy.method(passedIn);
        assertEquals(untouched, passedIn);
    }
}

Refactoring Code with Static Dependencies for Testability

Methods that have dependencies on static methods can be difficult to test. In this example we have a dependency that we obtain via a static method that we do not have control over and we want to test the surrounding code.

public void myMethod(Long id) {
     // ...
     Dependency dependency = DependencyUtil.getDependency();
     // ...
 }

We can refactor this to:

public void myMethod(Long id) {
     Dependency dependency = DependencyUtil.getDependency();
     myMethod(dependency);
 }

 void myMethod(Long id, Dependency dependency) {
     // ...
     doSomething(dependency);
     // ...
 }

We overload the method signature and preserve the public API. We make the new method have default access to make it testable. We can now mock Dependency however we like.

Bad Comments

Over the years I've found some 'interesting' comments in legacy code. One recent one is a seventeen line screed / excuse for how bad a particular piece of code is that ends with 'This makes me feel really dirty, fortunately, the Client/Server conversions have lowered the users expectations to the point where this seems inconsequential'.

A version history on this file shows that the commenter is the same person who wrote another comment that 'explains' why something is wrong because '... we are RETARDED' (referring to the organization). Supposedly this guy now works for Google. Amazing.

Indigestion from Digester and BeanUtils

We had some very simple XML that needed to be parsed. Apache Commons Digester was already in the project, so I used that and quickly whipped up a solution. All the happy-path tests worked, so I started adding edge-case tests.

A test that threw total nonsense malformed XML at it passed without any problems.

The next test was to pass in XML where one the attributes should have been a number, but the test passed a String. The test expected an Exception to be thrown, but there wasn't one and it failed. Surprisingly, Digester silently ignored the fact that the attribute could not be parsed into a long and simply left the attribute on the target class as zero.

Googling revealed two issues.

The first was that the default ErrorHandler for Digester swallows Exceptions. Bad stuff, but can be remedied by creating your own ErrorHandler:

ErrorHandler errorHandler = new ErrorHandler() {
 public void warning (SAXParseException e) throws SAXParseException { throw e; }
        public void error (SAXParseException e) throws SAXParseException { throw e; }
        public void fatalError(SAXParseException e) throws SAXParseException { throw e; }
};

Digester digester = new Digester();
digester.setErrorHandler(errorHandler);

This did not fix the test!

Further digging found that Digester depends on Apache Commons BeanUtils. In particular, it uses ConvertUtils to perform type conversions. The default converter is the culprit which silently does nothing if a conversion cannot be performed.

OK, so just right our own converter, right? Maybe.

You can right your own converter and you register it with ConvertUtils. But, the method signature for register is:

public static void register(Converter converter, Class clazz)

static!

This means that your custom converter will be used by all the Digester instances in your application. What's wrong with that? Well, in my case, this is a legacy application with test coverage around 1%. There may be code that 'depends' on the brokenness of the default converter behavior in Digester. Also, Digester is used by some other open source frameworks that may be in the project.

In the end, I decided not to use Digester and instead used dom4j which also allowed for a quick solution, but did the right thing and allowed me to control behavior on an instance by instance basis rather than using statics.

Commons-logging Headaches with Axis

I've written a web service using Axis 1.4 which depends on Apache commons-logging. Deploying on Tomcat, everything works fine, but some JUnit tests run fine in Eclipse, but break when running under CruiseControl throwing an exception:

org.apache.commons.discovery.DiscoveryException: No implementation defined for org.apache.commons.logging.LogFactory

The root cause is that commons-logging depends on Commons Discovery which plays tricks with the class-loader to 'discover' the logging implementation to use at runtime. For an exhaustive list of reasons as to why this is wrong-headed, read this article by Ceki Gülcü.

The normal way to specify which implementation to use is to have a commons-logging.properties file in the classpath with entries something like this:

org.apache.commons.logging.Log = org.apache.commons.logging.impl.Log4JLogger
org.apache.commons.logging.LogFactory = org.apache.commons.logging.impl.LogFactoryImpl

or to specify these as system properties using the -D option when running your app. In my case, I could not (easily) modify the ant targets to change the classpath because Configuration Management maintains shared build scripts that have to go through a heavy process to make any changes. To get around this, I set the system properties in code inside the test:

 @BeforeClass
 public static void beforeClass() {
  System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.Log4JLogger");
  System.setProperty("org.apache.commons.logging.LogFactory", "org.apache.commons.logging.impl.LogFactoryImpl");
 }

This fixed the issue and the tests now run under CruiseControl.

Tapestry 5 with Run Jetty Run

I'm currently going through the Tapestry 5 tutorial and am quite impressed. However, the tutorial is a bit out of date and recommends using the Jetty Launcher Eclipse plug-in to launch Jetty and to only use Jetty 5 as Jetty Launcher does not support Jetty 6.

I could not get Jetty Launcher to work at all under Eclipse 3.4 (Ganymede) on Ubuntu Linux. The Jetty Launcher page notes that it is no longer being maintained and recommends trying Run Jetty Run.

After installing Run Jetty Run, everything seemed to be copacetic with class changes being loaded live without a server restart. That is, everything was OK until beaneditform component was added. At that point, the app bombed with:

Error obtaining injected value for field org.apache.tapestry5.corelib.components.Form.logger: java.lang.ClassNotFoundException: org.slf4j.Logger

Some googling led me to: Tapestry5HowToSetupEclipseRunJettyRun.

I could not really follow the directions about adding -Dorg.mortbay.jetty.webapp.parentLoaderPriority=true to the VM arguments for the launch configuration.

Finally I was able to get this to work by adding -Dorg.mortbay.jetty.webapp.parentLoaderPriority=true to the default JRE's VM arguments.

Go to Window -> Preferences -> Installed JREs. Select the default JRE and click the Edit button. Paste -Dorg.mortbay.jetty.webapp.parentLoaderPriority=true into the Default VM Arguments field and click the Finish button.

Code Syntax Highlighting

Here is a plug-in that formats code (Java, xml, etc) within html in a very nice way.
http://code.google.com/p/syntaxhighlighter/

This looks great. I'll be adding this functionality ASAP to handle code snippets.

Injecting a Spring Bean into a Servlet

I've done some searching on the web for how to cleanly inject a spring-managed bean into a servlet. There are a number of ways, but I believe this is the cleanest.

Assuming you have a bean, limaBean, that you want to inject into a servlet, here is the applicationContext.xml:

  <bean id="limaBean" class="com.example.LimaBeanImpl">

  <bean class="org.springframework.web.context.support.ServletContextAttributeExporter">
      <property name="attributes">
          <map>
              <entry key="limaBeanInServletContext">
                  <ref bean="limaBean">
              </ref>
          </entry></map>
      </property>
  </bean>

The ServletContextAttributeExporter takes an existing managed bean, 'limaBean', and injects it into the ServletContext under the name 'limaBeanInServletContext'.

The servlet accesses this bean like:

public class DocumentViewerServlet extends HttpServlet
{

  public void doGet(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException
  {
      LimaBean limaBean = getLimaBean();
      limaBean.doSomething();

     ...
  }

  private DocumentService getLimaBean()
  {
      ServletContext servletContext = this.getServletContext();

      return (LimaBean) servletContext
          .getAttribute("limaBeanInServletContext");
  }
}

At runtime, Spring will inject the bean and everything works.

The reason you wanted to inject the bean in the first place, though, was so that you could easily write a test that injects the bean.

Set the attribute on org.springframework.mock.web.MockServletConfig and pass that into the servlets init(ServletConfig) method.

public class VegetableServletTest extends TestCase
{
  private VegetableServlet servlet = null;
  private LimaBean fakeLimaBean = null;

  protected void setUp() throws Exception
  {
      super.setUp();

      servlet = new VegetableServlet();
      fakeLimaBean = new FakeLimaBean();

      MockServletConfig config = new MockServletConfig();
           config.getServletContext().setAttribute("limaBeanInServletContext", 
           fakeLimaBean);
   
      servlet.init(config);
   }

}