The Holy Java

Building the right thing, building it right, fast

Archive for July, 2011

Most interesting links of July

Posted by Jakub Holý on July 31, 2011

Recommanded Readings

  • Martin Fowler, M. Mason: Why not to use feature branches and prefer feature toggles instead, when branches can actually be used (video, 12min) – feature branches are pretty common yet they are a hindrance for a good and stable development pace due to “merging hells”. With trusted developers, feature toggles are a much better choice.
  • M. Fowler: The LMAX Architecture – Martin describes the innovative and paradigm shaking architecture of the high-performance, high-volume financial trading platform LMAX. The platform can handle 6 million orders per second – using only a single java thread and commodity hardware. I highly recommend the article for two reasons: First, it crashes the common view that to handle such volumes you need multithreading. Second, for the rigorous, scientific approach used to arrive to this architecture. The key enablers are: 1) The main processing component does no blocking operations (I/O), those are done outside (in other threads). 2) There is no database – the state of the processor can be recreated by replaying the (persistent) input events. 3) To get further from 10k to 100k TPS they “just” wrote good code - well-factored, small methods (=> Hotspot more efficient, CPU can cache better). 4) To gain another multitude they implemented more efficient, cache-friendlier collections. All that was done based on evidence, enabled by thorough performance testing. 5) The processor and input/output components communicate without locking, using a shared (cyclic) array, where each of them operates on sum range of indexes and no element can ever be written by more than one component. Their internal range indexes do ever only increase so it is safe to read them without synchronization (at worst you will get old, lower value). The developers also tried Agents but found them in conflict with modern CPUs for their require context switch leading to emptying of the fast CPU caches.
    Updated: Martin has published the post titled Memory Image which discusses the LMAX approach to persistence in a more general way.
  • S. Mancuso: Working with legacy code with the goal of continual quality improvement – this was quite interesting for me as our team is in the same situation and arrived to quite similar approach. According to the author, the basic rule is “always first write tests for the piece code to be changed,” even though it takes so much time – he justifies it saying “when people think we are spending too much time to write a feature because we were writing tests for the existing code first, they are rarely considering the time spend elsewhere .. more time is created [lost] when bugs are found and the QA phase needs to be extended”. But it is also important to remember when to stop with refactoring to get also to creating business value and the rule for that is that quality improvements are done only with focus on a particular task. I like one of the concluding sentences: “Constantly increasing the quality level in a legacy system can make a massive difference in the amount of time and money spend on that project.”
  • Uncle Bob: The Land that Scrum Forgot – Scrum makes it possible to be very productive at the beginning but to be able to keep the productivity and continue meeting the expectations that are thus created we need to concentrate on some essential technical practices and code quality. Because otherwise we create a mess of growing complexity – the ultimate killer of productivity. Uncle Bob advices us what practices and how to apply to attain both high, sustainable productivity and (as required for it) high code quality. It’s essential to realize that people do what they are incented to do and thus we must measure and reward both going fast and staying clean.
    How do we measure quality? There is no perfect measure but we can build on the available established metrics – coverage, # (new) tests, # defects, size of tests (~ size of production code, 5-20 lines per method), test speed, cyclomatic complexity, function (< 20) and class (< 500) sizes, Brathwaite Correlation (> 2), dependency metrics (no cycles, in the direction abstraction).
    The practices that enable us to stay clean include among others TDD, using tools like Chceckstyle, FindBugs to find problems and duplication, implementing Continuous Integration.
  • Getting Started: Testing Concurrent Java Code – very good and worthy overview of tools for checking and testing of concurrent code with links to valuable resources. The author mentions among others FindBugs, concurrent test coverage (critical sections examined by multiple threads) measurement with IBM’s ConTest, multithreaded testing with ConTest (randomly tries to create thread interleaving situations; trial version – contact the authors for the full one) and MultithreadedTC (which divides time into “ticks” and enables you to fine-configure the interactions)
  • The top 9+7 things every programmer or architect should know – quite good selection of nine, respectively 7 things from the famous (on-line available) books 97 Things every programmer/architect should know.

Some fun:

  • Beans and noses – J. Spool reveals the First Rule of Consultancy: “No matter how much you try, you can’t stop people from sticking beans up their nose.” In other words, sometimes your clients decide to do some very unwise thing and no amount of reasoning can discourage them from that (quite understandably, as already the way they got to this decision defies logic).  “The only thing I can do in a beans-and-noses situation is wait. Wait until the bean is in its final resting place.” Then you ask the person how it is working for him and “… if sticking a bean deep into their nostril doesn’t meet the very high expectations they’d had, I can now start talking alternative approaches to reaching those expectations.” Already before you can actually ask them about their expectations, in some cases (50:50) this discussion can lead them to realize they could achieve them with an alternative, less painful approach. Now, if you have read up to this point, you clearly have enough time, so go an read the article because it’s really orth it!

Free e-Books

Posted in Languages, Testing, Tools, Top links of month | Tagged: , , , , , , | Comments Off

Experiencing JSF 1.2: Good but Needs a Framework

Posted by Jakub Holý on July 30, 2011

I’ve got an opportunity to experiment with JSF 1.2 while preparing a lecture about DbUnit Express and test webapp for JSF EL Validator. Same as JSF 1.1, it’s much easier to use than the low level, too HTTP-ish servlets but still it is not flexible enough and usable per se, it is absolutely necessary to use a decent framework (such as Rich or ICE Faces) to be really productive. Automatic value and method binding is cool but not sufficient.

For example forcing dataTable to get filtered when you change selection in a select list was rather non-trivial and making the table sortable by every column required quite lot of work and if I wanted to have that for every table, I’d need to develop framework of my own. Of course with various JSF component libraries you get such things out of the box but still JSF 1.2 is little to rigid, for example methods that could take parameters (which came in JSF 2.0) would make things a lot simpler.

I’ve also considered using business objects – such as EJBs – directly as JSF backing beans but I feel that in this version it might be suitable only for small applications (where the overhead of concern separation doesn’t pay off) or  some special ones. The problem is that even though backing beans are POJOs, JSF still forces them to a rather specific shape, which should be confined to the web layer  – such as having action methods that take no parameters and return an indication of which view to dislay next.

I would very much like to experiment with and compare other interesting frameworks such as Play!, Grails, and GWT (and given that I’ve finally got to learning Clojure, perhaps also Compojure), though I likely won’t have time for that. If I do, you will come to know :-)

Related posts

Posted in j2ee, Languages | Tagged: , , , | 4 Comments »

Simple Logging HTTP Proxy with Grinder

Posted by Jakub Holý on July 28, 2011

Sometimes I need to observe HTTP communication between my and another machine. I usually use Eclipse’ integrated TCP/IP monitor for it’s simple and does its job well but today for a large response it just displayed “The HTTP content is too large to display.” so I started searching for alternatives and found the Grinder TCPProxy, written in Java and distributed under the BSD license.

Grinder is a Java load testing framework and the proxy is just a part of it. Here is how you would start the proxy to forward local port 6080 to the remote address example.webservices.com:80 and log the HTTP communication into a file:

java -cp lib/grinder.jar net.grinder.TCPProxy -console -localhost 127.0.0.1  -localport 6080 -remotehost example.webservices.com -remoteport 80 > http.log

The optional flag -console makes it  to display a window for shutting it down cleanly (likely unnecessary under Linux/Mac). When you want it to log just into the console, add -colour for nicely colored output.

Run it with -? to see all the available options.

Posted in Tools | Tagged: , , | Comments Off

Having Database Test Ready in 10 Minutes with DbUnit Express

Posted by Jakub Holý on July 27, 2011

Update: The project has been renamed to dbunit-express.

DbUnit Express is my wrapper around DbUnit that intends to make it extremely easy to set up a test of a code that interacts with a database. It is preconfigured to use an embedded Derby database (a.k.a. JavaDB, part of SDK) and uses convention over configuration to find the test data. It can also with one call create the test DB from a .ddl file. Aside of simplifying the setup it contains few utilities to make testing easier such as getDataSource() (essential for testing Spring JDBC) and RowComparator.

Originally I was using DbUnit directly but I found out that for every project I was copying lot of code and thus I decided to extract it into a reusable project. Since that it has further grown to be more flexible and make testing even easier.

Here are the seven easy steps to have a running database test:
Read the rest of this entry »

Posted in Databases, Languages, Testing | Tagged: , , , | 1 Comment »

Ivy: Retrieve Both .jar And -sources.jar Into A Folder – Note to Self

Posted by Jakub Holý on July 25, 2011

After some experimenting I’ve found out how to tell Ivy (2.2.0) to retrieve also .jar with source codes into a folder:

Ivy.xml:

<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd"
            xmlns:e="http://ant.apache.org/ivy/extra">
...
   <dependency org="net.jakubholy.testing" name="dbunit-embeddedderby-parenttest" rev="1.2.0" conf="test" >
	<artifact name="dbunit-embeddedderby-parenttest" type="jar" />
	<artifact name="dbunit-embeddedderby-parenttest" type="sources" e:classifier="sources"  ext="jar" />
	<exclude org="junit" />
   </dependency>

The important thing here is the second artifact line with type and classifier sources and extension jar.

Ant:

<target name="ivy" description="Retrieve dependencies managed by Ivy into lib/">
	<ivy:settings file="ivysettings.xml" />
	<!-- Notice that with sync=true Ivy will also remove the files which do not need to be there -->
	<ivy:retrieve pattern="lib/[conf]/[artifact].[ext]" sync="true" type="jar" />
	<ivy:retrieve pattern="lib/[conf]/[artifact]-sources.[ext]" sync="false" type="sources" />
</target>

Notice the second retrieve line – the type is set to sources and we’ve changed also the pattern not to overwrite the normal .jar and we’ve also disabled sync for otherwise the normal .jar would be removed (I’m sure there must be a better way, perhaps combining the two into one line with type=”jar,sources” and a pattern which takes the type into account).

Posted in Tools | Tagged: | Comments Off

Going to Present “Programmer’s Survival Kit: Code Injection for Troubleshooting” at JavaZone 2011

Posted by Jakub Holý on July 18, 2011

I’m going to hold a lightning talk called Programmer’s Survival Kit: Code Injection for Troubleshooting at JavaZone in Oslo in September. This presentation is aimed at giving the participants the knowledge of code injection that they may (or I should rather say “will”) need and at persuading them that learning basics of code injection is really worth the little of the time that it takes. I’ll present three different real-world cases where code injection came to my rescue, solving each one with a different tool, fitting best the constraints at hand. I’ll also provide them with resources to learn the basics and tools easily and quickly. This will be a practical presentation; I won’t introduce AOP and will explain code injection only briefly, plunging right into the tools and how they can help developers.

Outline

  • Why every developer should know the basics of code injection – its irreplaceable role in troubleshooting
  • The power of code injection – what are the unique benefits of code injection
  • 3 tools, 2 (3) examples – I’ll introduce three rather different tools with different areas of application on practical, real-world examples
  • 1. The omnipresent Java Proxy – pros & cons, example of troubleshooting one failed batch update among many
  • 2. The independent Javassist – pros & cons (example not presented but available online – performance monitoring)
  • 3. The almighty AspectJ – pros & cons, example of forcing a 3rd party method to log its arguments upon failure
  • End – link to a blog post with much broader and deeper explanation of the topic and expanded source codes of the examples including support for building and running them

Expected Audience

Any, especially enterprise, Java developers that either know nothing about code injection/AOP, or know the concepts but not how to apply them or know a code injection tool and want to learn about others and how to use them for troubleshooting.

Hope to see you there!

Posted in General, Languages | Tagged: | Comments Off