Most interesting links of June ’12

Recommended Readings

Development

• Neal Ford: Evolutionary architecture and emergent design: Emergent design in the wild – discusses why not to do Big Design Up Front and how to recognize the “last responsible moment”. Quote: “It’s puzzling that so many organizations continue to use BDUF in the face of so many failures and underachieving projects.“
• Gojko Adzic: How To Solve “Not Enough Time” – everybody in IT complains about too much work. The solution acording to Gojko? Kill software already made that isn’t needed (ex.: deleted 70% functionality that wasn’t used; remember that maintenance costs grow ~ exponentially with size/complexity). Kill software in making that won’t be needed (know the value –  effect maps). Kill software that was not successful (if you measure the value of SW, you know whether its existence is worth it). Well written and inspiring. I’m looking forward to killing some SW
• Coding Horror: Postponing decisions to the last responsible moment – a brief and pretty good explanation of this key lean principle together with its connection to YAGNI (and some criticism of LRM by A. Cockburn – I agree with the postscript 2)

Special

• Build The (USS) Enterprise – A fascinating site by an engineer who dares to think big yet manages to stay rooted in reality. A huge inspiration for us all! We tend to think too small. Having a great, inspiring goal is what moves us forward. And people are known to have achieve seemingly impossible things nobody else believed in (how many believed Columbus could reach India by by sailing the opposite direcetion? and even though he actually hasn’t, the impacts of his discovery were tremendous)
• Hooking Users In 3 Steps: An Intro to Habit Testing – to be successful with a new web product in this age of distraction, you need your users to build the habit of using the app regularly (think of Twitter, Facebook, newspaper); this post describes how to find your habitual users, understand them, and optimize the application to turn more occassional users into habitual ones (identify habitual users – learn how they use the app & learn what turns random users into “devotees” – optimize the “habit path”)
• Puppet: Serve configuration from a particular Git branch on demand (original: Git Workflow and Puppet Environments) – at Comoyo we use Puppet to configure all our environments and developers need to test their changes before pushing them to the live environment, preferably without interference from other devs. This post by Adrien of PuppetLabs describes how to enable each developer to have her private branch(es) and have Pupet Master serve the config from the branch on demand, using little puppet environments and hooks magic. (Notice that if using Puppet to serve files then you’ll need to have them inside a module, which is a good practice anyway.)

Other

• DZone reference card for Jetty (DZone login required) – useful, brief overview over Jetty XML and programatic configuration, especially useful is the overview of handlers, use of servlets and webapps, SSL support, websockets
• What’s new in Groovy 2.0? – static type checking, Java 7
• Speed, memory and LOC of Python 3 vs. Java 7 [fixed link] (the computer language benchmark game) – Python tends to take more time (though not terribly more) but is quite economical with memory and visibly more productive (3* less lines of code). Ruby is similar but way slower. Scala is little slower and more memory hunrgy but also more productive. Of course performance is rarely the key factor for picking a language on often it doesn’t matter that much.

Favourite Quotes

Goodhart’s Law: once a metric becomes a target, it loses its meaning as a measure.

by WordFriday

Clojure Corner

• What Sucks about Clojure…and Why You’ll Love It Anyway – 40 min talk by Clojure Programming author Chas Emerick. Some of the negative points: Namespaces – complex (use x refer etc.; no package.* import, …), no forward references; Dynamic scope (with-foo …) has subtle, complex behavior and hard to see => avoid when you can; Using STM (effectively) is hard even though it looks easy, it’s overused, think of it rather as in-memory db with superior integr. with Clojure (ex.: what is the right ref granularity? whole word map? small particle?), nondeterministic (which tx will fail/suceed? – can’t reason) => strange error under high load/mix of operations; JVM: long startup time, …; Macros look easy but are hard (don’t compose nicely with the rest); Function composition – hard to find failure root cause.
• Why is Clojure so slow? (2/2012) – an interesting comparison of Clojure and other languages (C, F#, Groovy etc.) and an analysis of its slow start-up time (metadata and docstring building) and slower performance. (Interestingly, ClojureScript runs it 9* faster.) According to the author, it’s slower partly due to the imutable data structures (nothing new here, we knew we have to offer some performance for the increased safety and robustness). Conclusion: “Clojure is a beautiful, powerful and very useful language, but [...] not great for small script-y programs.” Also, the usefulness of the benchmark is limited if you aren’t writing games. Plans for making Clojure leaner and faster are under way.
• Stuart Sierra: Functional Relational Programming with Cascalog – a brief introduction into Cascalog with a valuable background info about Hadoop, MapReduce, relational programming in Clojure. Good links, especially the paper Out of the Tar Pit looks interesting.

Most interesting links of January ’12

Recommended Readings

• Jeff Sutherland: Powerful Strategy for Defect Prevention: Improve the Quality of Your Product – “A classic paper from IBM shows how they systematically reduced defects by analyzing root cause. The cost of implementing this practice is less than the cost of fixing defects that you will have if you do not implement it so it should always be implemented.” – categorize defects by type, severity, component, when introduced; 80% of them will originate in 20% of the code; apply prioritized automated testing (solve always the largest problem first). “In three months, one of our venture companies cut a 4-6 week deployment cycle to 2 weeks with only 120 tests.”
• Ebook draft: Beheading the Software Beast – Relentless restructurings with The Mikado Method (foreword by T. Poppendieck) – the book introduces the Mikado Method for organized, always-staying-green (large-scale) refactorings, especially useful for legacy systems, shows it on a real-world example (30 pages!), discusses various application restructuring techniques, provides practical guidelines for dealing with different sizes of refactorings and teams, discusses in depth technical debt and more. To sum it up in three words: Check it out!
• Daily Routine of a 4 Hour Programmer (well, it’s actually about 4h of focused programming + some hours of the rest) – a very interesting reading with some inspiring ideas. We should all find some time to follow up the field, to reflect on our day and learn from it (kaizen)
• The Agile Testing Quadrants – understanding the different types of tests, their purpose and relation by slicing them by the axis “business facing x technology facing” and the axis “supporting the team x critiquing the product” => unit tests x functional tests x exploratory testing x performance testing (and other). It helps to understand what should be automated, what needs to be manual and helps not to forget all the dimensions of testing.
• Adam Bien: Can stateful Java EE apps scale? – What does “stateless” really mean? “Stateless only means, that the entire state is stored in the database and has to synchronized on every request.” “I start the development of non-trivial (>CRUD) applications with Gateway / PDOs [JH: stateful EJBs exposing JPA entities] and measure the performance and memory consumption continuously.” Some general tips: Don’t split your web server and servlet container, don’t use session replication.
• Brian Tarbox: Just-In-Time Logging – How to remove 90% of worthless logs while still getting detailed logs for cases that matters – the solution is to (1) only add logs for a particular “transaction” with the system into a runtime structure and (2) flush it to the log only if the transaction fails or st. else significant happens with it. The blog also proposes a possible implementation in detail.
• DZone’s Top 10 NoSQL Articles of 2011
• DZone’s Top 5 DevOps Articles of 2011
• Test Driven Infrastructure with Vagrant, Puppet and Guard – this is interesting for me for I’m using Vagrant and Puppet on my project to create and share development environments or their parts and applying test-first approach to it seems interesting as do also the tools, rspec-puppet, cucumber-puppet and Guard (events triggered by file changes) and referenced articels.
• 5+1 Sonar Plugins you must not miss (2012 version) – Timeline Plugin (with Google Visualization Annotated TimeLine), Useless Code Plugin, SIG Maintainability Model Plugin (metrics Analysability, Changeability, Stability, Testability), Quality Index Plugin (1-number health indicator), Technical Debt Plugin

Links to Keep

• Niklas Schlimm: Java EE 6 vs. Spring Framework: A technology decision making process – key factors to consider when deciding between Java EE and Spring (or a mixed application of both) – value of standards, independence/flexibility, production-readiness, downwards compatibility …

Clojure Corner

• ClojureScript One Guide – “ClojureScript One shows you how to use ClojureScript to build single-page, single-language applications in a productive, effective and fun way.”
• Asynchronous workflows in Clojure - true asynchronous (non-blocking) network access in Clojure with Netty/the Lamina project.
• Clojure 2011 Year in Review – a list with important events in the Clojure sphere with links to details – C. 1.3.0, ClojureScript, logic programming with core.logic, clojure-contrib restructuring, birth of 4Clojure and Avout.
• Clojure Atlas – interesting project (alpha version) presenting Clojure documentation in the form of interactive graph of related concepts and functions; it’s far from perfection but I like the concept and consider paying those ~ $25 for the 1.3.0 version when its out (however, the demo is free and it might become open-sourced in 2012)

Most interesting links of October

Recommended Readings

• Steve Yegge’s Execution in the Kingdom of Nouns – I guess you’ve already read this one but if not – it is a well-written and amusing post about why not having functions as first class citizens in Java causes developers to suffer. Highly recommended.
• Reply to Comparing Java Web Frameworks – a very nice and objective response to a recent blog summarizing a JavaOne presentation about the “top 4″ web frameworks. The author argues that based on number of resources such as job trends, StackOverflow questions etc. (however data from each of them on its own is biased in a way) JSF is a very popular framework – and rightly so for even though JSF 1 sucked, JSF 2 is really good (and still improving). Interesting links too (such as What’s new in JSF 2.2?). Corresponds to my belief that GWT and JSF are some of the best frameworks available.
• Using @Nullable – use javax.annotation.Nullable with Guava’s checkNotNull to fail fast when an unexpected null appeares in method arguments
• JavaOne 2011: Migrating Spring Applications to Java EE 6 (slides) – nice (and visually attractive) comparison of JavaEE and Spring and proposal of a migration path. It’s fun and worthy to see.
• xUnitPatterns – one of the elementary sources that anybody interested in testing should read through. Not only it explains all the basic concepts (mocks, stubs, fakes,…) but also many pitfalls to avoid (various test smells such as fragile tests due to Data Sensitivity, Behavior Sensitivity, Overspecified Software [due to mocks] etc.), various strategies (such as for fixture setup), and general testing principles. The materials on the site were turned into the book xUnit Test Patterns: Refactoring Test Code (2007), which is more up-to-date and thus a better source.
• Eclipse tip: Automatically insert at correct position: Semicolon, Braces – in “while(|)” type “true {” to get ”while(true) {|” i.e. the ‘{‘ is moved to the end where it belongs, the same works for ‘;’
• Google Test Analytics – Now in Open Source – introduces Google’s Attributes-Components-Capabilities (ACC) application intended to replace laborous and write&forget test plans with something much more usable and quicker to set up, it’s both a methodology for determining what needs to be tested and a tool for doing so and tracking the progress and high-risk areas (based not just on estimates but also actual data such as test coverage and bug count). The article is a good and brief introduction, you may also want to check a live hosted version and a little more detailed explanation on the project’s wiki.
• JSF and Facelets: build-time vs. render-time (component) tags (2007) – avoid mixing them incorrectly
• StackOverflow: What are the main disadvantages of Java Server Faces 2.0? Answer: The negative image of JSF comes from 1.x, JSF 2 is very good (and 2.2 is expected to be just perfect ). Nice summary and JSF history review.
• Ola Bini: JavaScript in the small – best practices for projects using partly JavaScript – the module pattern (code in the body of an immediately executed function not to polute the global var namespace), handling module dependencies with st. like RequireJS, keeping JS out of HTML, functions generating functions for more readable code, use of many anonymous functions e.g. as a kind of named parameters, testing, open questions.

Talks

• Kent Beck’s JavaZone talk Software G Forces: The Effects of Acceleration is absolutely worth the 1h time. Kent describes how the development process, practices and partly the whole organization have to change as you go from annual to monthly to weekly, daily, hourly deployments. What is a best practice for one of these speeds becomes an impediment for another one – so know where you are. You can get an older version of the slides and there is also a detailed summary of the talk from another event.
• Rich Hickey: Simple Made Easy - Rich, the author of Clojure, argues very well that we should primarily care for our tools, constructs and artifacts to be “simple”, i.e. with minimal complexity, rather than “easy” i.e. not far from our current understanding and skill set. Simple means minimal interleaving – one concept, one task, one role, minimal mixing of who, what, how, when, where, why. While easy tools may make us start faster, only simplicity will make it possible to keep going fast because (growing) comlexity is the main cause of slowness.  And simplicity is a choice – we can create the same programs we do today with the tools of complexity with drastically simpler tools. Rich of course explains what, according to him, are complex tools and their simple(r) alternatives – see below. The start of the 1h talk is little slow but it is worth the time. I agree with him that we should much more thing about the simplicity/complexity of the things we use and create rather than easiness (think ORM).
  Read also Uncle Bob’s affirmative reaction (“All too often we do what’s easy, at the expense of what’s simple. And so we make a mess. [...] doing what is simple as opposed to what is easy is one of the defining characteristics of a software craftsman.”).

Random Notes from Rich’s Simple Made Easy Talk:

There are also better notes by Alex Baranosky and you may want to check a follow-up discussion with some Rich’s answers.

The complex vs. simple toolkit (around 0:31):

COMPLEXITY                             SIMPLICITY
State, objects                            Values
Methods                                   Functions, namespaces
vars                                          Managed refs
Inheritance, switch, matching  Polymorphism a la carte
Syntax                                      Data
Imperative loops, fold              Set functions
Actors                                      Queues
ORM                                         Declarative data manipulation
Conditionals                             Rules
Inconsistency                            Consistency

What each of the complexity constructs mixes (complects) together

CONSTRUCT                        COMPLECTS (MIXES)
State, objects – everything that touches it (for state complects time and value)
Methods – function and state, namespaces (2 classes, same m. name)
Syntax – Meaning, order
Inheritance – Types (ancestors, child)
Switch/matching – Multiple who/what pairs (1.decide who, 2.do what ?)
var(iable)s – Value, time
Imperative loops, fold – what/how (fold – order)
Actors – what/who
ORM – OMG
Conditionals – Why, rest of program (rules what program does are intertw. with the structure and order of the program, distributed all over it)

HE SIMPLICITY TOOLKIT (around 0:44)
CONSTRUCT            GET IT IVA…
Values – Final, persistent collections
Functions – a.k.a. stateless methods
Namespaces – Language support
Data – Maps, arrays, sets, XML, JSON etc.
Polymorphism a la carte – Protocols, Haskell type classes
Managed refs – Clojure/Haskell refs (compose time and value , not mix)
Set functions – Libraries
Queues – Libraries
Declarative data manipulation – SQL/LINQ/Datalog
Rules – Libraries, Prolog
Consistency – Transactions, values

True abstraction isn’t hiding complexity but drawing things away – along one of the dimensions of who, what, when, where, why [policy&rules of the app.], how.
Abstraction => there are things I don’t need – and don’t want – to know.
Why – do explore rules and declarative logic systems.
When, where – when obj. A communicates with obj. B. => put a queue in between them so that A doesn’t need to know where B is; you should use Qs extensively.

Links to Keep

• Incredibly Useful CSS Snippets -  “a list of CSS snippets that will help you minimize headaches, frustration and save your time while writing css” – few float resets, targetting specific browsers & browser hacks, cross-rowser transparency/min height/drop shadow, Google Font API, link styled by file type,

DevOps: Tools and libraries for system monitoring and (time series) data plotting

• Hyperic SIGAR API – open-source library that unifies collection of system-related metrics such as memory, CPU load, processes, file system metrics across most common operating systems
• rrd4j – Java clone of the famous RRDTool, which stores, aggregates and plots time-series data (RRD = round-robin database, i.e. keeps only a given number of samples and thus has a fixed size)
• JRDS “is performance collector, much like cacti or munins”, uses rrd4j. The documentation could be better and it seems to be just a one man project but it might be interesting to look at it.

Clojure Corner

• Alex Miller: Real world Clojure – a summary of experiences with using Clojure in enterprise data integration and analytics products at Revelytix, since early 2011 with a team of 5-10 devs. Some observations: Clojure code is 1-2 order of magnitude smaller than Java. It might take more time to learn than Java but not much. Clojure tooling is acceptable, Emacs is still the best. Debugging tools are unsurprisingly quite inferior to those for Java. Java profiling tools work but it may be hard to interpret the results. “[..]  I’ve come to appreciate the data-centric approach to building software.” Performance has been generally good so far.
• Article series Real World Clojure at World Singles – the series focuses on various aspects of using Clojure and how it was used to solve particular problems at a large dating site that starting to migrate to it in 2010. Very interesting. F. ex. XML generation, multi-environment configuration, tooling (“If Eclipse is your drug of choice, CCW [Counter ClockWise] will be a good way to work with Clojure.”, “Clojure tooling is still pretty young [..]  - but given how much simpler Clojure is than most languages, you may not miss various features as much as you might expect!”)
• StackOverflow: Comparing Clojure books – Programming Clojure, Clojure in Action, The Joy of Clojure, Practical Clojure – which one to pick? A pretty good comparison.
• Clojure is a Get Stuff Done Language – experience report – “For all that people think of Clojure as a “hard” “propeller-head” language, it’s actually designed right from the start not for intellectual purity, but developer productivity.”

Most interesting links of September

Recommended Readings

• J. Yip: It’s Not Just Standing Up: Patterns for Daily Standup Meetings - it isn’t easy to make stand-up meetings short, focused, energizing, and centered around continuous improvements and team spirit. This description of an example good standup, the meeting’s goals, and especially the “patterns” and “bad smells” can be pretty useful to get and keep on the track towards a brighter future. TBD: standup goals: GIFTs, team spirit, appreciation where we go and are.
• M. Poppendieck: Don’t Separate Design from Implementation – according to Mary, (detailed) requirements – being it in the form of (backlog) user stories or any other – represent actually a design of the system, which shouldn’t be done by the amateur product owner/business analyst but by professionals, meaning the developers, based on high-level goals and clear specification of the desired business value. She writes about a project that her factory outsourced and which she could have designed but didn’t – yet it succeeded even though there were no detailed requirements. I’ve also read and unfortunately lost an interesting answer where the author argues that that is only possible if the developers are really experienced in the field. I tend to agree more with Mary though it is of course a question what “high” and “low” level goals/requirements are. But undeniably users/analysts tend to propose solutions disguised as requirements while often missing the technical insight to see possible other and better solutions. We also cannot expect the developers to produce a great SW if the true goals, needs, and business values behind the requested “features” aren’t clearly communicated to them. (The best example – lost source again – is where a developer proposes to the client a simple process change that will solve the problem without writing a single line of code.)
• Mike Cohn: The Forgotten Layer of the Test Automation Pyramid – three levels of testing with increasing number of tests: UI/Service/Unit (or end-to-end instead of UI), each requiring a different approach. Unit tests are best because a failure points directly to its source (with higher level tests you don’t immediately know the cause). The higher in the pyramid, the less tests we should have (e.g. because of their redundancy). It’s important not to forget the middle, service layer – unit tests are too low-level, UI tests too difficult and brittle. Also Gojko in Specification by Examples says that acceptance/BDD tests should run mainly at the service layer because of the UI level issues.
  “Although automated unit testing is wonderful, it can cover only so much of an application’s testing needs. Without service-level testing to fill the gap between unit and user interface testing, all other testing ends up being performed through the user interface, resulting in tests that are expensive to run, expensive to write, and brittle.” [Emphasis JH.]
• Technical Debt and the Lean Startup – Paul Dyson remarks that while quality is an essential concern for projects in established environments, in the case of lean startups the primary goal is to find out whether a product is viable and what it should be like and thus it’s reasonable to accept much higher technical debt by not spending too much time on ensuring scalability, de-duplication etc. – only when the product proves viable should we start to care for its long-evity by emphasizing the quality. But one thing can never miss and that is good test suite because this is the crucial factor that makes letter payment of the technical debt possible without ruining oneself.
• Coding dojo – Real time coding competition with Extreme Startup – an inspiring report about a coding dojo lead by Johannes Brodwall in Bergen’s JUG, the task being the implementation of a server that can respond to questions send over HTTP (that’s all participants know at the beginning – they learn the rest during the iterations)
• Using Code Katas to Improve Programming Skills – why to use code katas + links to different proposed katas
• Kent Beck: Don’t Cross the Beams: Avoiding Interference Between Horizontal and Vertical Refactorings - when to do depth-first (more exploratory) refactoring and when to extend it into breadth (i.e. apply it to several similar objects)

Learning Clojure (maybe not so interesting for those not learning the language)

• Phil Calçado: My Experience With TDD In Clojure (via planetclojure) – nice example of how to decompose a task in functional programming to make it easy to test (via Midje), including useful testing-related links and a discussion of side-effect isolation and the building blocks of functional programs, i.e. function composition using combinators (i.e. functions producing functions)
• How to learn Clojure effectively (via planetclojure) – a very good description of how the task at  4Clojure (though I prefer Clojure koans) should be solved to benefit one’s learning the most plus some general tips on functional thinking
• Clojure open source projects for learning how to code it

A Funny Story about the Pain of Monthly Deployments

The end of our one-month iteration approached and the levels of fear and agitation started to rise. What will happen when we deploy it? Will everything crash? Will all hell break loose? Will there be a flood of midnight severity-one issues? Will the features developed work together or are there going to be unexpected and fatal interactions?

What is funny about this story, you certainly ask? You know these pains of deployment too and there is nothing funny about them.

Well, the funny thing is that I was used to quarterly releases and considered them completely normal and OK. Afterwards, I got used to bi-weekly releases and now I consider even monthly releases way too long (and I actually wouldn’t mind weekly releases). The conclusion? Once you see how easy and better things can be, you will always feel pain when you need to step back.

Most interesting links of May

Recommanded Readings

Acceptance testing / Specification by example:

• Gojko Adzic: Anatomy of a good acceptance test - an example of refactoring a bad acceptance test into a good one – good for learning about pitfalls and how a good one should look like
• Gojko: Top 10 reasons why teams fail with Acceptance Testing – acceptance testing is great and brings lot of value but must not be underestimated; some of the problems are bad collaboration, focusing on “how” instead of “what,” confusing AT with full regression tests. Brief, worth reading.
• Specification by Example: a love story (go directly to the PDF with the story): A nice, made-up story of going from low-level, workflow-based Selenium tests through similar Cucumber ones to true BDD tests describidng clearly what, not how – very well shows the point of specification by example and how it should (and should not) look like

(Enterprise) Java best practices:

•  Clean code, clean logs: 10 brief posts on logging best-practices – nothing really new here for me but in total it is a very good overview that every developer should know
• Make Large Scale Changes Incrementally with Branch By Abstraction – Continuous integration doesn’t work well with branches but as this article shows, you can manage even large-scale refactorings without branches using “branch by abstraction,” an approach reminding me of Fowler’s “strangler application” (an incremental replacement of a legacy system). The idea is: 1. Create an abstraction over the part of code to be changed;  2. Refactor the code to use it; 3. Implement new functionality using the new way / step by step move old functionality too, the abstraction layer delegating either to the new or old implementation … . It may be more work but: 1) your software is always working and deliverable; 2) (side-effect) in the end it will be more decoupled

Git:

• John Wiegley’s Git from the bottom upp (31p, Git 1.5.4.5, PDF) – a useful explanation of the fundamentals of Git, i.e. how it is constructed and how it works, which makes it much easier to understand how to  use it properly (recommended by Pål R.). Reading the The Git Parable first may be a good idea for an easy introduction into the fundamentals, though absolutely not necessary. This document introduces very well the important Git concepts (blob, index, commit, commit names such as branches, reflog) and how they cooperate to provide the rich set of functionality it has. It also explains well the value and usage of rebase. Among others I’ve appreciated the tip to use checkout, branch -m <new-branch> master, branch -D instead of the much more dangerous reset –hard and the tip to use stash / stash apply to create daily backups of your working tree in the reflog (with clearing it with ‘git reflog expire –expire=30.days refs/stash‘ instead of stash clear). Also git diff/log master..[HEAD] for reviewing work done in the current branch and and git diff/log ..master for checking the changes since the last merge/rebase after a fetch are interesting.

Tools:

• The secret power of bookmarklets - bookmarklets are an indispensable tool for every developer who works with web applications (to fill in test data, speed up log in, …), yet I’m sometimes surprised by meeting people who don’t know or use them; this blog explains them nicely, links to some useful ones and some useful tools for building them

Recommended Books

• (*****) Implementing Lean Software Development: From Concept to Cash by Mary Poppendieck, Tom Poppendieck – A great introduction into lean thinking (the values and principles it is build upon), clearly communicated with the help of “war stories”. I absolutely recommend it to anybody interested in lean/agile.
• (**** ) Agile Project Management with Scrum (Microsoft Professional) by Ken Schwaber – Even though you can’t understand Scrum without experiencing it, this book full of war stories will help you to avoid many Scrum implementation pitfalls and to understand its mantra of “the art of the possible” and will show you how to adapt Scrum to various situations. It’s very easy to read thanks to its format of brief case studies organized by topics (team, product owner, …).

Favourite Quotes of the Month

@unclebobmartin: Cleaning code does NOT take time. NOT cleaning code does take time.

Discussion: Agile not suitable for governmental IT?

The recent article Agile will fail GovIT, says corporate lawyer is rather controversial but very valuable. Its value lays not in its claim that agile cannot work in governmental environment, something I quite disagree with, but in its presentation of how (inaccurately) agile is/may be perceived in this environment and of obstacles posed by such an environment to any (and especially to agile) development.

The article reveals a fundamentaly psychological and social issue. It doesn’t question the ability of agile to deliver projects much more successfully than waterfall. The laywer, Alistair Maughan, doesn’t speak at all about projects’ results. He speaks about fear (to bear the consequences of a failure) and constraints present in governmental environment. Thus it’s pointless to argue about benefits of the agile and absurdity of the waterfall approaches. The key to a successful project is to understand those constraints, lift them as much as possible, and create a security structure for both governmental officials and the supplier to be able to work within those constraints safely. We shouldn’t forget that there are also smart people in the government agencies who will gladly accept a methodology that leads to better results as long as they are safe from being denounced on the front side of newspapers as bad public servants when something goes wrong.

Most interesting links of April (renewed)

Only two articles this month:

Computerworld: 22 free tools for data visualization and analysis

- great review if different categories of data analysis and visualization tools. The tools I haven’t known (i.e. excluding R, Google Charts etc.) and found them especially interesting:
Data web apps: Google Refine (data cleansing in a spreadsheet-like UI: clustering, data distribution overview, …), Google Fusion Tables (data => map etc., beta), Impure (rich & interactive data visualization via a drag-and-drop UI reminiscent of Yahoo Pipes; cons: lacking documentation, steep learning curve, check the teaser video).
JS libraries: Exhibit (JavaScript library by MIT for creating interactive visualizations e.g. for articles – incl. maps, timeplots, calendars etc., supporting filtering, sorting, searching), InfoVis Toolit (JS lib for interactive data visualizations; pros: beautiful, cons: choice of visualization types is somewhat limited), Protovis (by Stanford University’s Visualization Group; one of the more popular JS libraries for turning data into visuals, great docs, robust); OpenLayers ( example; customize & display a map, e.g. Open Street Map of Google), Polymaps (interactive maps with overlays)
GIS: OpenHeatMap (webapp, “astonishingly easy to create a color-coded map from many types of location data”)
Other: Timelines with TimeFlow (interesting desktop /java/ app x alpha) or SIMILE Timeline widget (JS); Word clouds: IBM Word-Cloud Generator (free, desktop /java/); Gephi (graph/network visualization & exploration; desktop)

The Evolution of Test Driven Developers

- an entertaining and enlightening article with valuable links to resources that can help you get to the next evolutionary step, one of its benefits is that it helps to understand the true value of the different types of tests (some -> TDD -> Behaviour Driven Development (‘what’ rather than ‘how’) -> Acceptance Test Driven Development) and the shift from a technical to a business perspective along the line

Real-world data prove that Agile, BDD & co. work – lecture by G. Adzic

I’ve attended a very inspirational lecture by Gojko Adzic, organized by the Oslo XP Meetup. Many people including some respectable persons claim that Lean, Agile, and high-level testing based on specifications (whether you call it Agile acceptance testing, Acceptance-test driven development, Example-driven development, Story-testing, Behavior-driven development, or otherwise – let’s call them all Specification by example) do not work.

To prove the contrary, Gojko has collected over 50 case studies of projects that were very successful thanks to using these methods. In his soon-to-be-published book, Specification by Example (download ch1, a review), he investigates what these projects and teams had in common, which was missing in the failed ones. So it’s great for two reasons: It documents how great success you can achieve with Specification by Example and it shows you how to implement it successfully.