The Holy Java

Building the right thing, building it right, fast

Archive for April, 2012

Most interesting links of April ’12

Posted by Jakub Holý on April 30, 2012

Recommended Readings

  • V. Duarte: Story Points Considered Harmful – Or why the future of estimation is really in our past… (also as 1h video) – thoughtful and data-backed claim that there is a much cheaper way for estimating work throughput than estimating each story in story points (SP) and that is simply counting the stories. Even though their sizes differ, over (not that much) longer periods, where it really matters, these differences will even out. The author argues that estimating in number of stories provides the same reliability and benefits as SP and is much easier. (Keep in mind that estimation is just an attempt at predicting the future and humans are proved to be terrible at doing that; why to pretend that we can do it?) I’d recommand this to anybody doing Scrum and similar.
  • M. Fowler: Test Coverage – it’s obvious that increasing test coverage for the sake of test coverage it’s a nonsense but some people still need to be reminded of it :-). Fowler explains what the real benefit of test coverage measurements is and how to use it for good instead of for evil.
  • Brian Marick: How to Misuse Code Coverage (pdf) – cited a lot by Fowler in his article, this is really a good paper. Marick has participated in the development of several code coverage tools and understands well their limitations. One of the key points is that code coverage tools can discover only one class of test weakness (not testing some paths through your code) but cannot discover that you are missing some code you should have (e.g. when you check only for two of three possible return values). Thus the code coverage metric tells you “this code isn’t well tested, are you sure you don’t to look more into it”? It’s crucial not to write tests so as to increase the code coverage; look at the code and improve the test without any regard for coverage. You may thus decrease the likeliness of both the class of problems.
  • A Year with MongoDB – Kiip has found out that Mongo isn’t the best choice for them (having 240GB, 500+ operations/s, 85M docs and their specific usage of the store) and migrated to the combination of Riak (key-value store) and PostgreSQL. Some of the issues they hit are slow counts and limit/offset queries due to using non-counting B-trees for indexing, memory management that could be more intelligent and tuned for the use to make sure the data needed is indeed in RAM, no built-in support for compressing key names (their size adds up as they’re repeated in each document; you’ve to compress them [user -> u etc.] in the client if you want to), limited concurrency due to process-wide write lock (which becomes a problem if the write’s aren’t short enough w.r.t. number of ops/s, e.g. because data isn’t in RAM and/or the query is complicated), safe settings (waiting for a write to finish, …) off by default, offline-only table compaction (w/o it the disk usage grows unbounded). The lessons learnt for me: Know your storage, its weaknesses and intended way of usage, and make sure it matches your needs.
  • Rudolf Winestock: The Lisp Curse – Lisp’s expressive power is actually a cause of its lack of momentum because it’s so easy to implement anything that people have no need to join forces and thus there are many half-baked (“works-for-me”) solutions for anything – but no complete, generally accepted one. An interesting essay. “Lisp is so powerful that problems which are technical issues in other programming languages are social issues in Lisp.”
  • Understanding JDBC Internals & Timeout Configuration – the article itself could have been written better but it conveys the important information that configuring timeouts for JDBC isn’t trivial because they need to be set correctly at different levels and without a socket timeout set in a driver-specific way it can hang forever if the DB cannot be reached due to network/system failure
  • Circos: An Amazing Tool for Visualizing Big Data – this article is interesting primarily for its combination of Google Analytics API, Neo4J and an unusual data visualization with circular graphs

Tools

  • CRaSH: Extensible shell for the JVM (docs) – a shell that you can embedd into a web server as a WAR, run standalone or attach to a running JVM, connect to it via SSH or Telnet, and use it to execute commands against the JVM. Some commands: configure loggers, control threads, monitor the system (mem, threads, ..), connect/issue queries via JDBC. More commands can be written in Groovy. There is a whole set of commands for working with JCR. Pluggable authentication.

Clojure Corner

Posted in Databases, General, Testing, Tools, Top links of month | Tagged: , , , | Comments Off

Exposing Functionality Over HTTP with Groovy and Ultra-Lightweight HTTP Servers

Posted by Jakub Holý on April 4, 2012

I needed a quick and simple way to enable some users to query a table and figured out that the easiest solution was to use an embedded, ligthweight HTTP server so that the users could type a URL in their browser and get the results. The question was, of course, which server is best for it. I’d like to summarize here the options I’ve discovered – including Gretty, Jetty, Restlet, Jersey and others – and their pros & cons together with complete examples for most of them. I’ve on purpose avoided various frameworks that might support this easily such as Grails because it didn’t feel really lightweight and I needed only a very simple, temporary application.

I used Groovy for its high productivity, especially regarding JDBC – with GSQL I needed only two lines to get the data from a DB in a user-friendly format.

My ideal solution would make it possible to start the server with support for HTTPS and authorization and declare handlers for URLs programatically, in a single file (Groovy script), in just few lines of code. (Very similar to the Gretty solution below + the security stuff.)

Read the rest of this entry »

Posted in j2ee, Languages | Tagged: , , , | 5 Comments »

Groovy Grape: Troubleshooting Failed Download

Posted by Jakub Holý on April 2, 2012

If you use the Grape’s @Grab annotation to get dependencies for your Groovy scripts at runtime and their retrieval fails with the exception “General error during conversion: Error grabbing Grapes — [unresolved dependency: …not found]” and a useless stack trace then you migth want to know that you can configure Ivy to log all the details of what is going on (what it is trying to download, where from, …), for example in the interactive groovysh shell:

groovy:000> org.apache.ivy.util.Message.setDefaultLogger(new org.apache.ivy.util.DefaultMessageLogger(org.apache.ivy.util.Message.MSG_DEBUG))
groovy:000> groovy.grape.Grape.grab(autoDownload: true, group: 'org.eclipse.jetty.orbit', module: 'javax.servlet', version: '3.0.0.v201112011016')
...
WARN: ==== ibiblio: tried
WARN:   http://repo1.maven.org/maven2/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.orbi
...

You can likely also increase the log level by setting the system property ivy.message.logger.level to 4 (debug, see the Ivy Message class.)

(For the list of arguments that grab supports see GrapeIvy, namely the method createGrabRecord [btw, ext and type are ignored unless you also set classifier] this has been fixed)

Posted in General, Languages | Tagged: , | Comments Off