The Holy Java

Building the right thing, building it right, fast

Archive for February, 2012

Most interesting links of February ’12

Posted by Jakub Holý on February 29, 2012

Recommended Readings

  • List of open source projects at Twitter including e.g. their scala_school – Lessons in the Fundamentals of Scala and effectivescala – Twitter’s Effective Scala Guide
  • M. Fowler & P. Sadalage: Introduction into NoSQL and Polyglot Persistence (pdf, 11 slides) – what RDBMS offer and why it sometimes isn’t enough, what the different NoSQL incarnations offer, how and on which projects to mix and match them
  • Two phase release planning – the best way to plan something somehow reliably is to just start doing it, i.e. just start the project with the objective of answering “Can this team produce a respectable implementation of that system by that date?” in as short time as possible (i.e. few weeks). Then: “Phase 2: At this point, there’s a commitment: a respectable product will be released on a particular date. Now those paying for the product have to accept a brute fact: they will not know, until close to that date, just what that product will look like (its feature list). What they do know is that it will be the best product this development team can produce by that date.” Final words: “My success selling this approach has been mixed. People really like the feeling of certainty, even if it’s based on nothing more than a grand collective pretending.”
  • Tumblr Architecture – 15 Billion Page Views A Month And Harder To Scale Than Twitter – what SW (Scala, Finagle, heavily partitioned MySQL, …) and HW they use, the architecture (Firehose – event bus, cell design), lessons learned (incl. “MySQL (plus sharding) scales, apps don’t.”
  • Jay Fields’ Thoughts: Compatible Opinions on Software – about teams and opinion conflicts – there are some areas where no opinion is really right (e.g. powerful language vs. powerful IDE) yet people may have very strong feeling about them. Be aware of what your opinions are and how strong they are – and compose teams so that they include more less people with compatible (not same!) opinions – because if you team people with strong opposing opinions, they’ll loose lot of productivity. Quotes: “I also believe that you can have two technically excellent people who have vastly different opinions on the most effective way to deliver software.” “I suggest that you do your best to avoid working with someone who has both an opposing view and is as inflexible as you are on the subject. The more central the subject is to the project, the more likely it is that productivity will be lost.”
  • Jay Fields’ Thoughts: Lessons Learned while Introducing a New Programming Language (namely Clojure) – introducing a new language and winning the hearts of (sufficient subset of) the people is difficult and requires lot of extra effort. This is both an experience report and a pretty good guide for doing it.
  • Jay Fields’ Thoughts: Life After Pair Programming – a proponent of pair-programming comes to the conclusion that in some contexts pairing may not be beneficial, i.e. the benefits of pair-programming don’t overweight the costs (for a small team, small software, …)
  • The Why Monitoring Sucks (and what we’re doing about it) – the #monitoringsucks initiative- what tools there are, why they suck, what to do, new tools, what metrics to collect, blogs, …
  • JBoss Byteman 2.0.0: Bytecode Manipulation, Testing, Fault Injection, Logging – a Java agent which helps testing, tracing, and monitoring code, code is injected based on simple scripts (rules) in the event-condition-action form (the conditions may use counters, timers etc.). Contrary to AOP, there is no need to create classes or compile code. “Byteman is also simpler to use and easier to change, especially for testing and ad hoc logging purposes.” “Byteman was invented primarily to support automation of tests for multi-threaded and multi-JVM Java applications using a technique called fault injection.” It was used e.g. to orchestrate the timing of activities performed by independent threads, for monitoring and statistics gathering, for application testing via fault injection. Contains a JUnit4 Runner for easily instrumenting the code under test, it can automatically load a rule before a test and unload it afterwards:
    @Test
    @BMRule(name="throw IOException at 1st call",
    targetClass = "TextLineProcessor",
    targetMethod = "processPipeline",
    action = "throw new java.io.IOException()")
    public void testErrorInPipeline() throws Exception { ... }
  • How should code search work? – a thought-provoking article about how much better code completion could be if it profited more from patterns of usage in existing source codes – and how to achieve that. Intermediate results available in the Code Recommenders Eclipse plugin.

REST

  • What Makes Jersey Interesting: Parameter Classes (by Coda Hale, 5/2009) – brief yet rich and very practical introduction into Jersey (the reference implementation of JAX-RS. i.e. REST, for Java) including error handling, parameter classes (automatic wrapping of primitive values). The following article, What Makes Jersey Interesting: Injection Providers, might be of interest too.
  • How to GET a Cup of Coffee, 10/2008 – good introduction into creating applications based on REST, explained on an example of building REST workflow for the ordering process in Starbucks – a “self-describing state machine”. The advantage of this article is that it presents the whole REST workflow with GET, OPTIONS, POST, PUT and “advanced” features such as the use of If-Unmodified-Since/If-Match, Precondition Failed, Conflict. The workflow steps are connected via the Location header and a custom <next> link tag with rel and uri. Other keywords: etag, microformats, HATEOS (-> derive the next resource to access from the links in the previous one), Atom and AtomPub, caching (web trades latency for scaleability; if 1+s latency isn’t acceptable than web isn’t the right platform), URI templates (-> more coupling than links in responses), evolution (-> links from responses, new transitions), idempotency. “The Web is a robust framework for integrating systems at local, enterprise, and Internet scale.”

Links to Keep

Tools, Libraries etc.

  • ClusterSSH – whatever commands you execute in the master SSH session are also execute in the slave sessions – useful if you often need to execute the same thing on multiple machines (requires Perl); to install on Mac: “brew install csshx”
  • HTML5 Boilerplate (H5BP) – customizable initial HTML5 project template for a website; can be combined e.g. with Bootstrap, the HTML/JS/CSS toolkit (there is even a script to set them both up). Includes server configs for optimal performance, “delivers best practices, standard elements”.
  • High performance libraries in Java – disruptor, Java Chronicle (ultra-fast in-memory db), Colt Matrix library (scientific computations), Javolution (RT Java), Trove collections for primitives, MG4J (free full-text search engine for large document collections), some serialization & other banchmarks links.
  • Twitter Finagle – “library to implement asynchronous Remote Procedure Call (RPC) clients and servers. Finagle is flexible enough to support a variety of RPC styles, including request-response, streaming, and pipelining; for example, HTTP pipelining and Redis pipelining. It also makes it easy to work with stateful RPC styles; for example, RPCs that require authentication and those that support transactions.” Supports also failover/retry, service discovery, multiple protocol (e.g. http, thrift). Build on Netty, Java NIO. See the overview and architecture.
  • Eclipse Code Recommenders – interesting plugin in incubation that tries to bring more more intelligent completion based more on context and the wisdom of the crowds (i.e. patterns of usage in existing source codes) to Eclipse

Clojure Corner

  • Clojure/huh? – Clojure’s Governance and How It Got That Way – an interesting description how the development of Clojure and inclusion of new libraries is managed. “Rich is extremely conservative about adding features to the language, and he has impressed this view on Clojure/core for the purpose of screening tickets.” E.g. it took two years to get support for named arguments – but the result is a much better and cleaner way of doing it.
  • Clojure Monads Series – comprehensive explanations of monads starting with Monads In Clojure

Quotes

A language that doesn’t affect the way you think about programming, is not worth knowing-

 - Alan Perlis

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

Eric S. Raymond, “How to Become a Hacker”

Posted in General, j2ee, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Cool Tools: Fault Injection into Unit Tests with JBoss Byteman – Easier Testing of Error Handling

Posted by Jakub Holý on February 25, 2012

How do you test error handling in your application? Do you? Is it at all possible to simulate SocketTimeoutException, SQLException? Normally the answer would be that it isn’t possible or very difficult, requiring a complicated setup. Not anymore – with JBoss Byteman you can easily inject whatever code – e.g. throw new SocketTimeoutException() – in any place – e.g. Socket.connect. The code is injected before your test runs and unloaded when it finishes. Here is an example:

@RunWith(BMUnitRunner.class)
public class BytemanJUnitTests {
@Test(expected=MyServiceUnavailableException.class)
   @BMRule(name="throw timeout at 1st call",
   targetClass = "Socket",
   targetMethod = "connect",
   action = "throw new java.io.IOException()")
   public void testErrorInPipeline() throws Exception {
      // Invokes internally Socket.connect(..):
      new MyHttpClient("http://example.com/data").read();
   }
}

Read the rest of this entry »

Posted in Testing, Tools | Tagged: , | 10 Comments »

Profiling Tomcat Webapp with VisualVM and NetBeans – Pitfalls

Posted by Jakub Holý on February 25, 2012

Profiling a webapp running on Tomcat with VisualVM or NetBeans wasn’t as easy as expected, so this is a brief record of what to avoid to succeed.

Environment: Mac OS X, Java JDK 1.6.0_29, Netbeans 7.1, VisualVM 1.3.3 (installed separately), Tomcat 6.

The Pitfalls:

VisualVM

  • VisualVM Sampler and Profiler: To be able to drill down to the slow methods you need to take a snapshot before you stop the sampling/profiling (there is a [Snapshot] button above the Hot Spots list). This is not very intuitive and the interface doesn’t communicate it.
  • VisualVM Profiler: Excludes Thread.sleep() & Object.wait() time (as opposed to the NetBeans profiler where you can choose to include/exclude them) => if you method is spending lot of time waiting for a lock, you won’t discover it
  • You might need to allow unsafe, passwordless JMX connections in your Tomcat config, see Resources

NetBeans Profiler

  • While VisualVM was able to dynamically connect to my Tomcat, NetBeans wasn’t able to do it and hasn’t provided any notification about the failure. The only visible manifestation was that it wasn’t showing and collecting any data. Solution: Use the “Direct” attach invocation, i.e. starting the target java application with the NetBeans profiling agentlib.

Tools Overview

VisualVM

Extremely useful tool, included in JDK since 6.0 (command line: jvisualvm) or on the VisualVM page.

  • Automatically discovers local Java processes and can connect to them (if they run Java 6+)
  • Monitoring – threads, heap, permgen, CPU, classes
  • Sampler – low-overhead profiling tool (takes thread snapshot at regular intervals and compares their stack traces to find out where most time is spent)
  • Plugins, such as MBeans, Tracers
  • Profiler – a simpler version of NetBeans profiler (comparison here); it can dynamically attach to running (Java 6+) processes and instrument classes on-the-fly. The not very visible checkbox Settings in the right-top corner can be used to set the classes to start profiling from (syntax: my.package.** to include subpackages or my.package.* or my.pkg.MyClass) and the packages not to / only to profile (syntax differs here: my.package.* to include subpackages or my.package. or my.pkg.).

Resources

Posted in Languages, Tools | Tagged: , , | Comments Off

Release 0.9.9 of Static JSF EL Expression Validator with Annotated Beans Autodetection

Posted by Jakub Holý on February 13, 2012

I’ve released version 0.9.9 of Static JSF EL Expression Validator (tool to check that EL expressions in JSF pages use only existing beans and properties), available for download from Maven Central. The main addition since the last version is the ability to detect managed beans based on annotations instead of reading them from faces-confix.xml or Spring config, thanks to the cool Reflections lib, and support for ui:repeat.

Read the rest of this entry »

Posted in j2ee, Languages, Tools | Tagged: , , , | 2 Comments »

Using Java Compiler Tree API to Extract Generics Types

Posted by Jakub Holý on February 7, 2012

I was looking for some way to extract information about types of elements in Java collections/maps using generics (List<String>, Map<String, MyBean>) so that the users of the Static JSF Expression Validator wouldn’t need to declare the type of the elements manually. One possible way to get this information is to process the source codes with the Sun Compiler Tree API, available since JDK 6.

It might be best to go and check the resulting 263 lines of CollectionGenericsTypeExctractor.java now. The code is little ugly, largely due to the API being ugly.

Read the rest of this entry »

Posted in Languages | Tagged: , , , | Comments Off

Separating Integration and Unit Tests with Maven, Sonar, Failsafe, and JaCoCo

Posted by Jakub Holý on February 5, 2012

Goal: Execute the slow integration tests separately from unit tests and show as much information about them as possible in Sonar.

The first part – executing IT and UT separately – is achieved by using the maven-failsafe-plugin and by naming the integration tests *IT (so that the unit test running surefire-maven-plugin will ignore them while failsafe will execute them in the integration-test phase and collect results in the verify phase).

The second part – showing information about integration tests in Sonar – is little more tricky. Metrics of integration tests will not be included in the Test coverage + Unit tests success widget. You can add Integration test coverage (IT coverage) widget if you enable JaCoCo but there is no alternative for the test success metrics. But don’t despair, read on!

Important notice: The integration of Sonar, JaCoCo and Failsafe evolves quite quickly so this information may easily get outdated with the next releases of Sonar

Versions: Sonar 2.12, Maven 3.0.3

Read the rest of this entry »

Posted in Languages, Testing, Tools | Tagged: , , , | 5 Comments »