The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘management’

The Failure of Governmental IT (Learnings From

Posted by Jakub Holý on October 28, 2013

The failure of Healthcare.Gov has been discussed a lot but the main causes of the failure are unrelated to the project or to technology and apply similarly to other governments / large projects.

According to some reasonable articles, the main problems were:

  1. Byzantine procurement process – those with the best lawyers and experience, not the most capable ones get the job
  2. Size – the project is “too important to fail” – and thus also too big to succeed; +- 55 contractors creating SW with fixed date, integrating hundreds of insurence providers and some 36 states and various agencies; big-bang style of development and deployment
  3. Fragmented responsibility and lack of ability – no one both with enough knowledge and power responsible for the whole project (and lack of the best talent in government IT in general), responsibility spread across tens of contractors and officials likely driven by cover-my-ass motivation (e.g. the procurement officer interested in selecting the cheapest offer that checks all the checkboxes instead of the best one – because who can fire her/him for doing that?)
  4. Niagara Falls of waterfall development, constrained by rules and bureaucracy to immobility – extensive legislation, rules, and security requirements together with a fear/blame-driven organization or not good for agile approaches

BTW, according to L. Hart (below), 0 federal projects over $5M were delivered on time, only 6.4% of those over $10M have succeeded and full 40% of such projects were canceled. So, under those circumstances, Healthcare.Gov is actually a small miracle.


  • Laurence Hart: Healthcare.Gov Fiasco Shows the Problems in Federal IT – insight into the broken procurement and many obstacles any federal IT project faces by a person with rich experience with it
  • Merici Vinton: 9 Things You Should Know Before Debating, From Someone Who Actually Launched a Successful Government Website – an important story: it is possible to launch a successful government website but it requires special effort and approach. The main advice is:
    1. “Never build a website that’s too big to fail; instead, start small” – the CFPB “launched a pretty basic, consumer facing public website in six to eight weeks” then gradually added intake for complains regarding various products, one by one. “We did each rollout in small chunks and built more and more based on what we learned with each integration.”
    2. “Let’s do open source when possible (preferably always).”
    3. “Let’s have in house strategy, design, and tech.” – i.e. do not outsorce those

    Also, involve IT people in the procurement and hiring.

  • Tim Murphy: Could Obama’s Campaign Tech Gurus Fix Let’s Ask ‘Em! – the answer is no, mostly not – due to the sheer size, the procurement process, and all the legislation. Quotes the campaign’s CTO’s twitter: “The ‘secret’ here is that the problems are not about tech at all,” he tweeted on Monday. “It is about procurement. I can’t fix that with my tech chops or my team.”

By the way, the Government Digital Service team in the UK has become “recently” famous for bringing effective IT to the government. It is interesting to read about the UK DS strategy, based on delivery – frequent, iterative, repeatedly successfull.

Thanks to @flowchainsensei and @timoreilly for the links.


Posted in General | Tagged: , , | Comments Off on The Failure of Governmental IT (Learnings From

Most interesting links of Mars ’12

Posted by Jakub Holý on March 31, 2012

Recommended Readings

  • ThoughtWorks Technology Radar 3/2012 – including apps with embedded servlet containers (assess), health check pages for webapp monitoring, testing at the appropriate level (adopt), JavaScript micro-framewors (trial, see, Gradle over Maven (e.g. thanks to flexibility), OpenSocial for data & content sharing between (enterprise) apps (assess), Clojure (before in asses) and CoffeeScript on trial (Scala very close to adopt), JavaScript as a 1st class language (adopt), single-threaded servers with aync I/O (Node.js, Webbit for Java [http/websocket], …; assess).
  • Jez Humble: Four Principles of Low-Risk Software Releases – how to make your releases safer by making them incremental (versioned artifacts instead of overwritting, expand & contract DB scripts, versioned APIs, releasing to a subset of customers first), separating software deployment from releasing it so that end-users can use it (=> you can do smoke tests, canary releasing, dark launching [feature in place but not visible to users, already doing something]; includes feature toggles [toggle on only for somebody, switch off new buggy feature, …]), delivering features in smaller batches (=> more frequently, smaller risk of any individual release thanks to less stuff and easier roll-back/forward), and optimizing for resiliance (=> ability to provision a running production system to a known good state in predictable time – crucial when stuff fails).
  • The Game of Distributed Systems Programming. Which Level Are You? (via Kent Beck) – we start with a naive approach to distributed systems, treating them as just a little different local systems, then (painfully) come to understand the fallacies of distributed programming and start to program explicitely for the distributed environment leveraging asynchronous messaging and (often functional) languages with good support for concurrency and distribution. We suffer by random, subtle, non-deterministic defects and try to separate and restrict non-determinism by becoming purely functional … . Much recommended to anybody dealing with distributed systems (i.e. everybody, nowadays). The discussion is worth reading as well.
  • Shapes Don’t Draw – thought-provoking criticism of inappropriate use of OOP, which leads to bad and inflexible code. Simplification is OK as long as the domain is equally simple – but in the real world shapes do not draw themselves. (And Trades don’t decide their price and certainly shouldn’t reference services and a database.)
  • Capability Im-Maturity Model (via Markus Krüger) – everybody knows CMMI, but it’s useful to know also the negative directions an organization can develop in. Defined by Capt. Tom Schorsch in 1996, building on Anthony Finkelstein’s paper A Software Process Immaturity Model.
  • Cynefin: A Leader’s Framework for Decision Making – an introduction into the Cynefin cognitive framework – the key point is that we encounter 5 types of contexts differing by the predictability of effects and each of them requires a different management style, using the wrong one is a recipe for a disaster. Quote:

    The framework sorts the issues facing leaders into five contexts defined by the nature of the relationship between cause and effect. Four of these—simple, complicated, complex, and chaotic—require leaders to diagnose situations and to act in contextually appropriate ways. The fifth—disorder—applies when it is unclear which of the other four contexts is predominant.

  • Et spørsmål om kompleksitet (Norwegian). Key ideas mixed with my own: Command & control management in the traditional Ford way works very well – but only in stable domains with clear cause-and-effect relationships (i.e. the Simple context of Cynefin). But many tasks today have lot of uncertanity and complexity and deal with creating new, never before seen things. We try to lead projects as if they were automobile factories while often they are more like research – and researchers cannot plan when they will make a breakthrough. Most of the new development of IT systems falls into the Complex context of Cynefin – there is lot of uncertanity, no clear answers, we cannot forsee problems, and have to base our progress on empirical experience and leverage emergence (emergent design, ..).
  • The Economics of Developer Testing – a very interesting reflection on the cost and value of testing and what is enough tests. Tests cost to develop and maintain (and different tests cost differently, the more complex the more expensive). Not having tests costs too – usually quite a lot. To find the right ballance between tests and code and different types of tests we must be aware of their cost and benefits, both short & long term. Worth reading, good links. (Note: We often tend to underestimate the cost of not having good tests. Much more then you might think.)

Links to Keep


Kent Beck answering a question about how much testing to do (highlighted by me):

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Different people will have different testing strategies based on this philosophy, but that seems reasonable to me given the immature state of understanding of how tests can best fit into the inner loop of coding. Ten or twenty years from now we’ll likely have a more universal theory of which tests to write, which tests not to write, and how to tell the difference. In the meantime, experimentation seems in order.

Posted in General, Testing, Top links of month | Tagged: , , , , , | Comments Off on Most interesting links of Mars ’12