The Holy Java

Building the right thing, building it right, fast

Cool Tools: Fault Injection into Unit Tests with JBoss Byteman – Easier Testing of Error Handling

Posted by Jakub Holý on February 25, 2012

How do you test error handling in your application? Do you? Is it at all possible to simulate SocketTimeoutException, SQLException? Normally the answer would be that it isn’t possible or very difficult, requiring a complicated setup. Not anymore – with JBoss Byteman you can easily inject whatever code – e.g. throw new SocketTimeoutException() – in any place – e.g. Socket.connect. The code is injected before your test runs and unloaded when it finishes. Here is an example:

@RunWith(BMUnitRunner.class)
public class BytemanJUnitTests {
@Test(expected=MyServiceUnavailableException.class)
   @BMRule(name="throw timeout at 1st call",
   targetClass = "Socket",
   targetMethod = "connect",
   action = "throw new java.io.IOException()")
   public void testErrorInPipeline() throws Exception {
      // Invokes internally Socket.connect(..):
      new MyHttpClient("http://example.com/data").read();
   }
}


What’s even more cool, the Byteman rules also support various conditions such as count downs or rendezvous – a special object that can be used to synchonize and order independent thread, e.g. to ensure that thread A executes method a before thread B executes method b – which makes it possible to test different orders of execution and concurrency issues that they might cause.

Fault injection in JUnit tests and the use of conditions such as rendezvous is well explained in the official tutorial Fault Injection Testing With Byteman.

In general, JBoss Byteman 2.0.0 is a Bytecode Manipulation, Testing, Fault Injection, Logging and generally aspect-oriented programming tool. More exactly it is a Java agent which helps testing, tracing, and monitoring code. Additional code is injected (instrumentation) based on simple scripts (rules) in the event-condition-action form (the conditions may use counters, timers etc.). Contrary to standard AOP, there is no need to create classes or compile code. “Byteman is also simpler to use and easier to change, especially for testing and ad hoc logging purposes.” “Byteman was invented primarily to support automation of tests for multi-threaded and multi-JVM Java applications using a technique called fault injection.”

Yyou can download Byteman from the project site or get it from the JBoss Maven repository (let’s hope they’ll release into Maven Central too.)

Get a complete example project from my JavaZone 2012 talk at GitHub.

Update: Brett L. Schuchert argues very well in Modern Mocking Tools and Black Magic – An example of power corrupting why using such black magic (JMockIt in his case) should be avoided as only treating the symtomps in favor of actually fixing the code.

About these ads

10 Responses to “Cool Tools: Fault Injection into Unit Tests with JBoss Byteman – Easier Testing of Error Handling”

  1. Rogerio said

    Sounds similar to what the JMockit mocking API provides:

    public class JMockitEquivalentOfBytemanJUnitTests
    {
    @Test(expected=MyServiceUnavailableException.class)
    public void testErrorInPipeline() throws Exception
    {
    new NonStrictExpectations() {
    Socket s;
    { s.connect((SocketAddress) any); result = new IOException(); }
    };

    // Invokes internally Socket.connect(..):
    new MyHttpClient(“http://example.com/data”).read();
    }
    }

    • If JMockit works as any other mocking framework then you need to pass the mock – in this case a mock Socket – to the code using it. But it the code under test doesn’t accept it from its environemnt and instead creates it on its own (inside MyHttpClient.read: Socket s = new Socket(…);) then a mocking framework cannot help you.

      Byteman uses code instrumentation (injection) and thus can inject failures anywhere.

Sorry, the comment form is closed at this time.

 
%d bloggers like this: