Cool Tools: Fault Injection into Unit Tests with JBoss Byteman – Easier Testing of Error Handling

How do you test error handling in your application? Do you? Is it at all possible to simulate SocketTimeoutException, SQLException? Normally the answer would be that it isn’t possible or very difficult, requiring a complicated setup. Not anymore – with JBoss Byteman you can easily inject whatever code – e.g. throw new SocketTimeoutException() – in any place – e.g. Socket.connect. The code is injected before your test runs and unloaded when it finishes. Here is an example:

@RunWith(BMUnitRunner.class)
public class BytemanJUnitTests {
@Test(expected=MyServiceUnavailableException.class)
   @BMRule(name="throw timeout at 1st call",
   targetClass = "Socket",
   targetMethod = "connect",
   action = "throw new java.io.IOException()")
   public void testErrorInPipeline() throws Exception {
      // Invokes internally Socket.connect(..):
      new MyHttpClient("http://example.com/data").read();
   }
}


What’s even more cool, the Byteman rules also support various conditions such as count downs or rendezvous – a special object that can be used to synchonize and order independent thread, e.g. to ensure that thread A executes method a before thread B executes method b – which makes it possible to test different orders of execution and concurrency issues that they might cause.

Fault injection in JUnit tests and the use of conditions such as rendezvous is well explained in the official tutorial Fault Injection Testing With Byteman.

In general, JBoss Byteman 2.0.0 is a Bytecode Manipulation, Testing, Fault Injection, Logging and generally aspect-oriented programming tool. More exactly it is a Java agent which helps testing, tracing, and monitoring code. Additional code is injected (instrumentation) based on simple scripts (rules) in the event-condition-action form (the conditions may use counters, timers etc.). Contrary to standard AOP, there is no need to create classes or compile code. “Byteman is also simpler to use and easier to change, especially for testing and ad hoc logging purposes.” “Byteman was invented primarily to support automation of tests for multi-threaded and multi-JVM Java applications using a technique called fault injection.”

Yyou can download Byteman from the project site or get it from the JBoss Maven repository (let’s hope they’ll release into Maven Central too.)

Get a complete example project from my JavaZone 2012 talk at GitHub.

Update: Brett L. Schuchert argues very well in Modern Mocking Tools and Black Magic – An example of power corrupting why using such black magic (JMockIt in his case) should be avoided as only treating the symtomps in favor of actually fixing the code.

Published by Jakub Holý

I’m a JVM-based developer since 2005, consultant, and occasionally a project manager, working currently with Iterate AS in Norway.

10 thoughts on “Cool Tools: Fault Injection into Unit Tests with JBoss Byteman – Easier Testing of Error Handling

  1. Sounds similar to what the JMockit mocking API provides:

    public class JMockitEquivalentOfBytemanJUnitTests
    {
    @Test(expected=MyServiceUnavailableException.class)
    public void testErrorInPipeline() throws Exception
    {
    new NonStrictExpectations() {
    Socket s;
    { s.connect((SocketAddress) any); result = new IOException(); }
    };

    // Invokes internally Socket.connect(..):
    new MyHttpClient(“http://example.com/data”).read();
    }
    }

    1. If JMockit works as any other mocking framework then you need to pass the mock – in this case a mock Socket – to the code using it. But it the code under test doesn’t accept it from its environemnt and instead creates it on its own (inside MyHttpClient.read: Socket s = new Socket(…);) then a mocking framework cannot help you.

      Byteman uses code instrumentation (injection) and thus can inject failures anywhere.

      1. JMockit works just like Byteman, using java.lang.instrument. Actually, from reading the Byteman tutorial I would say that JMockit is more advanced at this point (it uses Attach API, class redefinition as well as transformation, and can make arbitrary modifications to “java.lang” classes on demand).

      2. I put the link to the project site on my name. 🙂
        You can find hundreds of working examples there. The closest to Byteman rules, I think, is the ability to load “mock classes” (Java classes annotated with @MockClass and @Mock) at startup by listing them in the “jmockit-mocks” system property or having a user-defined jmockit.properties file inside a jar.

      3. I can Google out the project myself :-), I was thinking about a link to an example that does the same thing as my code above. But thank you anyway.

      4. If you are referring to the “BytemanJUnitTests” test class in the article, then the “JMockitEquivalentOfBytemanJUnitTests” class I wrote before is what you want. Just put it in a JUnit test suite and add jmockit.jar to the classpath, before junit.jar (with JDK 1.6+).

      5. Right, sorry, JMockit looks like magic to me so I find it difficult to understand what it does (especially given that most mock libs just create mocks and can’t change what existing classes return). Thank you!

Comments are closed.

%d bloggers like this: