Lesson Learned: Don’t Use Low-Level Lib To Test High-Level Code

Summary: Using a fake http library to test logic two levels above HTTP is unnecessarily complex and hard to understand. Fake instead the layer directly below the logic you want to test and verify the low-level HTTP interaction separately. In general: Create thin horizontal slices for unit testing, checking each slice separately with nicely focused and clear unit tests. Then create a coarse-grained vertical (integration-like) test to test across the slices.

The case: I want to test that the method login sends the right parameters and transforms the result as expected. Login invokes post-raw which calls an HTTP method. Originally I have tried to test it by using the library clj-http-fake but it proved to be unnecessarily complex. It would be much better to fake post-raw itself for testing login and test the original post-raw and its HTTP interaction separately, using that library.

Most interesting links of April ’13

Recommended Readings

The top top article

How To Survive a Ground-Up Rewrite Without Losing Your Sanity (recommended by Kent Beck) – sometimes you need to actually rewrite an important part of a system; here we learn about two such rewrites, one which went well and one that failed badly – and what are the important differences.

The pain of a rewrite: “it’s [a major rewrite] going to take insanely longer than you expect” – because: “there’s this endless series of weird crap encoded in the data in surprising ways” and it takes days to convert them, “It’s brutally hard to reduce scope” (you cannot drop features, edge cases), “There turn out to be these other system that use ‘your’ data”.

To succeed you need: 1) Determine clear business-visible wins to justify the effort that will be much higher than expected and to know when to give up / what to postpone; 2) Do it extremely incrementally (<->  Succession) – break it into a series of small, safe steps, each generating a business value and learning of its own thus enabling early and frequent economical tradeoffs (stop, shift priorities, …) – ex.: rewrite a single reports, migrate its data, switch customers to it, go on to the next one – complete slice of functionality => a more realistic estimate soon => reprioritisation; incrementalism requires you to be able to write data both to the old and new system, which is hard but always pays off: “Here’s what I’m going to say: always insert that dual-write layer. Always. It’s a minor, generally somewhat fixed cost that buys you an incredible amount of insurance.” 3) “Abandoning the Project Should Always Be on the Table” (<- known biz value, better estimate based on early feedback).

Some Specific Tactics: Shrink Ray FTW (a graph of how much has been already replaced => motivation), Engineer The Living Hell Out Of Your Migration Scripts (tests, robustness, error handling, restartability), If Your Data Doesn’t Look Weird, You’re Not Looking Hard Enough.

Methodology, agile, lean

• M. Fowler: The New Methodology – a good description of the rise of Agile, the motivation for it, the various Agile methodologies (XP, Lean, Scrum etc.) and what is required to be able to apply an agile approach. Main points: Agile is adaptive (vs. predictive) and relies heavily on people and their judgement and skills (vs. treating them as same, replacable units) – which also leads to the need of leadership instead of (command&control) management. Discusses unpredictability of requirements and scope, foolishness of separating design and implementation, difficulty of measurement of SW development, continuous improvement etc. Quotes: “However letting go of predictability doesn’t mean you have to revert to uncontrollable chaos. Instead you need a process that can give you control over an unpredictability. That’s what adaptivity is all about.”
• The Toyota concept of ‘respect for people’ – many state that they respect their workers but fail to really understand what it means; it is not about freedom of act, it is about a mutual respect, leveraging the strengths of each other: worker’s experience and insight and manager’s broader overview, as demonstrated by the problem-solving dialog and challenges (problem – root cause – solution – measure of success, the manager challenging the worker’s answers). Also a nice example how the evaluation of individual performance leads to a much worse system and high turnover compared to a whole-oriented company.
• Fixed Bid Agile Without Cognitive Dissonance – a refreshing take on fixed-scope projects and Agile; yes, they are bad but sometimes the client has no other choice so what best we can make out of it? The core advice: Agree “a pragmatic change management protocol (along with a contingency built into the pricing)” (push for lower initial requirements granularity, customer involvement, flexibility of functionality) => “you can gain significant agile benefits for clients who wouldn’t otherwise accept them”.
• Agile Atlas: Scrum – a good description of Scrum and its values, roles, artifacts, and activities

Learning, psychology, estimates

• How Developers Stop Learning: Rise of the Expert Beginner – sometimes you meet people with experience-indicating titles that are actually little competent, perhaps leading incompetent IT departments. Why? They, unchallenged by competent peers or broader IT community, came to believe that they are “experts” while actually being only little more advanced beginners, better than their beginner colleagues but still lacking any understanding of the big picture and the knowledge of what they do not know, trapped in the “unconscious incompetence” stage. The post explains this in a more detail and is followed up an explanation how it can lead to the rise of a mediocre SW group in “How Software Groups Rot: Legacy of the Expert Beginner“.
• Coding, Fast and Slow: Developers and the Psychology of Overconfidence (via @peterskeide) – why are we so bad at estimating (inherent complexity of SW vs. our overconfidence) and why it cannot be fixed. We can learn to somehow estimate tasks of few hours length (less complex, plenty of practice opportunities). The question is: “how you can your dev team generate a ton of value, even though you can not make meaningful long-term estimates?”
• Cognitive Overhead, Or Why Your Product Isn’t As Simple As You Think (via @JiriJerabek) – to make apps more accessible to users, we try to make them simple – but “simple” might be different from what you expect. The important thing is not less steps, less features, less elements, but lower cognitive overhead, i.e. “how many logical connections or jumps your brain has to make in order to understand or contextualize the thing you’re looking at.” Good examples of unexpectadly high / pleasantly low cognitive overhead, some tips, even suprising ones such as make people do more (to be more involved in the process – e.g. bump their phones), slow down your product.

Other

• Economies of Scala – a case for using Scala over Java, supported by data: many capable developers want to use it but there are few opportunities for them – and getting developers is one of the main challenges.
• A canonical Repository test – a nice standard way to test a “DAO”; highlights: use of  FEST assert 2 for clean and nice checks, no unimportant details in the test (f.ex. details of the test data hidden in randomPerson() and randomOder(Person)).
• How To Think Like An Engineer – some nice ideas such as: “Build A Simple First Version: With People, Not Code” – “Technology is not always the best solution, because technology is not always the simplest solution.”, i.e. don’t automate everything from the start (examples from Netflix, Amazon); “Rather than trying to do everything at once, break down the functions of your company into smaller goals.” – and focus at one at a time
• Economies Of Scale As A Service (do not mix up with Scala! )- an interesting description of the trend away from ownership to the rental of important resources (servers, manufacturing capabilities, personal cars, …) and the resulting changes in the society, business, and industry
• Troy Hunt: Our password hashing has no clothes (or the much shorter though biased How To Safely Store A Password) – MD5 and SHA are not safe enough due to brute-force attack enabled by GPUs, irrespective key size; it’s crucial to use hashing algorithms designed for passwords (and thus sufficiently slow) – f.ex. bcrypt, or PBKDF2 or the newer scrypt.
• Everything about Java 8 – a well-made summary of what should come in Java 8, based on the current state, discussing the finer points: static and default (non-static, overridable) methods on interfaces, lambdas (do I need to mentione that?!) and method references (String::valueOf, Object::toString, myVar::toString, ArrayList::new); good discussion of the various use cases and limitations of lambdas (capturing x non-c., ..); java.util.stream for functional operations on value streams (filter, map, reduce etc.); java.time inspired by Joda, more concurrency utilities (e.g. CompletableFuture for chaining futures); String.join (finally!), Optional ~ Scala’s Option & more; yummy!
• How To Keep Your Best Programmers – what motivates capable programmers to stay/leave? The author lists some common reasons and concludes that, ultimately, all are linked to the desire for autonomy, mastery, or purpose. However he goes further and proposes that, to keep talented devs, you must offer them an appealing narrative (regarding their actions and a result, related to autonomy/mastery/purpose) and reaffirm/update it frequently; ex.: “With the work that we’re giving you over the next few months, you’re going to become the foremost NoSQL expert in our organization.” “At any point, both you and the developers on your team should know their narratives.” – so that they will be “constant points of job satisfaction and purpose.”

Clojure Corner

• Clojure Data Analysis Cookbook review – “The book provides a collection of recipes for accomplishing common tasks associated with analyzing different types of data sets. It starts out by showing how to read data from a variety of sources such as JSON, CSV, and JDBC. [..] how to sanitize the collected data and sample large data sets. [..] a number of different strategies for processing it.” How to present them with ClojureScript and  NVD3 (D3.js components). “Some of the highlights include using the Clojure STM, parallel processing of the data, including useful tricks for partitioning, using reducers, and distributed processing with Hadoop and Casalog.”

Favorite Quotes

once again, trying to do it *and* do it right was too much all at once, resulting in little progress and little learning.

- Kent Beck’s tweet 2013-04-16

A true agile development process can be recognized by its continual evolution:

A project that begins using an adaptive process won’t have the same process a year later. Over time, the team will find what works for them, and alter the process to fit.

- Martin Fowler in The New Methodology

The Value and Perils of Performance Benchmarks in the Wake of TechEmpower’s Web Framework Benchmark

The TechEmpower’s Web Framework Benchmark is quite interesting but the comments following it at HackerNews are even more so. That is at least the constructively critical ones that highlight many of the issues with benchmarks while also reminding us of their value. One could formulate the benchmark paradox:

Benchmarks are important for rational technological choices yet it is very hard if not impossible to perform them in a sensible way.

I would like to record here some of the important points, mainly as a future reference for myself for whenever I will be dealing with benchmarking.

Most interesting links of January ’13

Recommended Readings

Various

• Dustin Marx: Significant Software Development Developments of 2012 – Groovy 2.0 with static typing, rise of Git[Hub], NoSQL, mobile development (iOS etc.), Scala and Typesafe stack 2.0, big data, HTML5, security (Java issues etc.), cloud, DevOps.
• 20 Kick-ass programming quotes – including Bill Gates’ “Measuring programming progress by lines of code is like measuring aircraft building progress by weight.”,  B.W. Kernighan’s “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”, Martin Golding’s “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” (my favorite)
• How to Have a Year that Matters (via @gbrindusa) – do you want to just survive and collect possessions or do you want to make a difference? Some questions everybody should pose to him/herself.
• Expression Language Injection – security defect in applications using JSP EL that can sometimes leads to double evaluation of the expressions and thus makes it possible to execute data supplied by the user in request parameters etc. as expressions, affects e.g. unpatched Spring 2.x and 3.

Languages etc.

• HN discussion about Scala 2.10 – compilation speed and whether it matters, comparison of the speed and type system with Haskell and OCaml, problems with incremental compilation (dependency cycles, fragile base class), some speed up tips such as factoring out subprojects, the pros and cons of implicits etc.
• Blog Mechanical Sympathy – interesting posts and performance tests regarding “writing software which works in harmony with the underlying hardware to gain great performance” such as Memory Access Patterns Are Important and Compact Off-Heap Structures/Tuples In Java.
• Neal Ford: Functional thinking: Why functional programming is on the rise – Why you should care about functional programming, even if you don’t plan to change languages any time soon – N. Ford explains the advantages of FP and why FP concepts are spreading into other languages (higher abstractions enabling focus on the results over steps and ceding control to the language, more reusability on a finer level (higher-order functions etc.), few generic data structures with many operations -> better composability, “new” and different tool such as lazy collections, shaping the language towards the problem instead of vice versa, aligning with trends such as immutability)
• Neal Ford: Java.next: The Java.next languages Leveraging Groovy, Scala, and Clojure in an increasingly polyglot world – a comparison of these languages with focus on what they are [not] suitable for, exploration of their paradigms (static vs. dynamic typing, imperative vs. functional)

SW development

• How to Completely Fail at BDD – a story of an enthusiastic developer who tried to make everyone’s life better by introducing automated BDD tests and failed due to differences in culture (and inability to change thinking from the traditional testing), a surprising lack of interest in the tool and learning how to write good tests: “Culturally, my current team just isn’t ready or interested in something like this.” Morale: It is hard to change people, good ideas are not enough.
• M. Feathers: Refactoring is Sloppy – refactoring is often prioritized out of regular development and refactoring sprints/stories aren’t popular due to past failures etc. An counter-intuitive way to get refactoring in is to imagine, during planning, what the code would need to be like to make it easy to implement a story. Then create a task for making it so before the story itself and assign it to somebody else then the story (to force a degree of scrutiny and communication). “Like anything else in process, this is medicine.  It’s not meant to be ‘the way that people do things for all time’ [..]” – i.e. intended for use when you can’t fit refactoring in otherwise. It may also make the cost of the current bad code more visible. Read also the commits (f.ex. the mikado method case).
• Cyber-dojo: A great way to practice TDD together. Compare your read-green cycle and development over time with other teams. Purposefully minimalistic editor, a number of prepared tdd tasks.
• On the Dark Side of “Craftsmanship” – an interesting and provoking article. Some developers, the software labouers, want to get work done and go home, they haven’t the motivation and energy to continualy spend time improving themselves. There is nothing wrong with that and we shouldn’t disparge them because of that. We shouldn’t divide people into craftsmen and the bad ones. A summary of and response to the varied reactions follows up in More on “Craftsmanship”. The author is right that we can’t expect everybody to spend nights improving her/his programming skills. Still they should not produce code of poor quality (with few exceptions) since maintaining such code costs a lot. There should be time for enough quality in a 9-5 day and people should be provided with enough guidance and education to be able to write decent code. (Though I’m not sure how feasible it is, how much effort it takes to become an acceptable developer.) Does the increased cost of writing (an learning to write) good code overweight the cost of working with bad code? That is an eternal discussion.

Cloud, web, big data etc.

• Whom the Gods Would Destroy, They First Give Real-time Analytics (via Leon) – a very reasonable argument against real-time analytics: yes, we want real-time operational metrics but “analytics” only makes sense on a sensible amount of data (for the sake of statistical significance etc.) RT analytics could easily provide misguided results.
  CAP Twelve Years Later: How the “Rules” Have Changed (tl;dr, via @_dagi) – an in-depth discussion of the CAP theorem and the simplification (2 out of 3) that it makes; there are many more nuances. By Eric Brewer, a professor of computer science at the University of California, Berkeley, and vice president of infrastructure at Google.
• ROCA: Resource-oriented Client Architecture – “A collection of simple recommendations for decent Web application frontends.” Server-side: true REST, no session state, working back/refresh etc. Client: semantic HTML independent of layout, progressive enhancement (usable with older browsers), usable without JS (all logic on the server) etc. Certainly not suitable for all types of apps but worthwile to consider the principles and compare them with your needs.

Clojure Corner

• Distributed Actors in Clojure (5/2012) – a discussion of options for Akka-like stuff in Clojure. Akka is great but “interfacing to Akka from Clojure is not nice, and certainly not idiomatic”. Though there have been some new Akka-Clojure libs since that (akka-clojure, okku).

Tools

• Vaurien, the Chaos TCP Proxy (via @bsvingen) – an extensible proxy that you can control from your tests to simulate network failure or problems such as delays on 20% of the requests; great for testing how an application behaves when facing failures or difficulties with its dependencies. It supports the protocols tcp, http, redis, memcache.
• Wvanbergen’s request-log-analyzer for Apache, MySQL, PostgreSQL, Rails and more (via Zarko) – generates a performance report from a supported access log to point out requests that might need optimizing
• Working Effectively With iTerm2 (Mac) – good tips in the body and comments

Favorite Quotes

A very good (though not very scientific) definition of project success applicable for distinguishing truly agile from process-driven projects:

[..] a project is successful if:

• Something was delivered and put to use
• The project members, sponsors and users are basically happy with the outcome of the project

- Johannes Brodwall in “How do we become Agile?” and why it doesn’t matter, inspired by Alistair Cockburn

(Notice there isn’t a single word about being “on time and budget”.)

Most interesting links of October ’12

Recommended Readings

• David Veksler: Some lesser-known truths about programming – things newcomers into the field of IT don’t know and don’t expect, true and an interesting read. Not backed by good data but anyway. F.ex.: “[..] a programmer spends about 10-20% of his time writing code [..] much of the other 90% thinking, researching, and experimenting”. “A good programmer is ten times more productive than an average programmer. A great programmer is 20-100 times more productive than the average [..]” “Bad programmers write code which lacks conceptual integrity, non-redundancy, hierarchy, and patterns, and so is very difficult to refactor.” “Continuous change leads to software rot, which erodes the conceptual integrity of the original design.” “A 2004 study found that most software projects (51%) will fail in a critical aspect, and 15% will fail totally.”
• Brett L. Schuchert: Modern Mocking Tools and Black Magic – An example of power corrupting – interesting for two reasons: a good analysis of a poorly written piece of code and discussion of the code injection black magic (JMockIt) vs. actually breaking dependencies to enable tests.  The author presents a typical example of low-quality method (mixing multiple concerns, mixing different levels of abstractions, untestable due to a hardcoded use of an external call) and discusses ways to improve it and to make it testable. Recommended to read.
• It’s Not About the Unit Tests – Learning from iOS Developers: iOS developers don’t do much testing yet they manage to produce high quality. How is that possible? The key isn’t testing itself, but caring for the code. (Of course, iOS is little special: small apps, no legacy, a powerful platform that does lot for the apps, very visual apps.) “It’s not about the practices. It’s about the spirit and intent behind them, and how they are applied.” (M. Fowler had a similar observation about a team that used mock-based testing exclusively and thus lacked integration tests yet all worked. [I've lost the link to the post and would be grateful for it])
• Java Code Quality Tools – Overview – brief descriptions of 44 quality-related tools including some interesting tools and Eclipse plugins I didn’t know or knew but forgot. F.ex. analysis of dependencies with JBoss Tattletale or JarAnalyzer, Clirr to check libraries for source and binary backwards compatibility, JDiff generates JavaDoc-based report of removed/added/changed in an API. Spoon – read and check or transform Java code. Java PathFinder (NASA) – special JVM capable of checking all execution path to discover concurrency defects etc.

Tools

• DirB, Directory Bookmarks for Bash (home) – moving efficiently among favourite directories (s <name> to create a bookmark for pwd, g <bookmark | relative/abs dir path> to enter a dir (=> works both for bookmarks and as a replacement for cd); also support for relative path bookmarks & more; sl lists bookmakrs in the last used order) (You might also want to check out Autojump, described in Dec 11; bashmarks is another similar project. Another similar project is rupa’s z and j2 and the fish clone z-fish)

Clojure Corner

•  Jon Pither: Clojure at a Bank – Moving from Java -  the justification (productivity, dynamism, FP a better match for the domain) and process behind moving from Java to Clojure with a monolithic 1M LOC Spring/Hibernate app. (Random quotes: “I had used some dynamical languages before and it was quite obvious that we were essentially forcing lots of schema and type definition on to a problem domain that just didn’t want or need it.” “[..] it [dependency injection] just looks redundant in retrospect now that I’m working 95% with FP code.”) There is also a EuroClojure talk about their experiences one year later (35 min).
• Prismatic’s “Graph” at Strange Loop – an interesting desing problem, its solution, and a resulting OSS library. The problem: How to break a large function into independently usable small ones that might depend on each other without ever needing to recompute a value once the function producing is called. The solution: Graph – “Graph is a simple, declarative abstraction to express compositional structure.” (Enabling explicit declaration of data dependencies and pluging in different implementations.)
• The Oblong: Blog about 2/3 D game programming in Clojure, starting from scratch (w/o an engine); interesting experiences
• Ironclad: Steam Legions – Clojure game development battle report (the game on Github)
• Building the Wishlisted.org webapp in Clojure – experiences from learning Clojure for real by building a webapp in Noir
• Clojure vs. Scala smackdown (“Just kidding with the title of this post “) – a short post with interesting discussion. Dmitri Sotnikov’s opinion resonates with me: “I found that for me Clojure wins on simplicity and consistency. While it looks more alien initially, once you learn the basics, you just reuse the same patterns everywhere.” Some more comments: “One major concern was maintainability, since it’s fairly easy to write very dense code. This turned out to not be a problem in practice. Because Clojure code is written as a tree, refactoring it is very easy.” REPL seems to be a big win (applies to Scala too). Scala’s type system might get tedious and learning its quirks takes time but there is lot of potential and both have they strong sides.
• Code Fatigue – discussion of the advantages of learning, using, and combining the (many) standard Clojure functions instead of a “basic solution” using recursion etc. The argument is in favor of higher-level code with less complexity in the form of branching, recursion, nested expressions etc. and thus less mental fatigue.

Favorite Quotes

A classic test only cares about the final state – not how that state was derived. Mockist tests are thus more coupled to the implementation [emphasis mine] of a method. Changing the nature of calls to collaborators usually cause a mockist test to break.

- Martin Fowler in his classical Mocks Aren’t Stubs

I’m afraid of code. When I see a big pile of code, I get scared . Some classes and method make me cry. I had troubles explaining why I prefer short pieces of code keeping the same level of abstraction, cohesive and loosely coupled. The following quote captures the essence – improved communication.

One way to improve communication is to reduce the need for it and the same can be said for code. [...] Since we tend to read code more than write it, anything we can do to reduce the need to read code is time well invested in the life of a project.

- Brett L. Schuchert in Modern Mocking Tools and Black Magic – An example of power corrupting justifying extraction of code into another class or method

Do You Know Why You Are Testing?! (On The Principles Underlying TDD)

Kent Beck in his recent post Functional TDD: A Clash of Cultures summarizes well the key principles and benefits that underlie test-driven development. I think it is really worthwhile becoming aware of and thinking over these foundation stones of TDD (and testing in general). Knowing them enables you to apply TDD in the most effective way with respect to a particular context to gain the maximum of these benefits. People that do not really understand the value of their tests and TDD tend to write hard to maintain tests of limited (and occasionally even negative) value. Are you one of them?

Most interesting links of September ’12

Recommended Readings

• Johannes Brodwall: This dependency injection madness must end! – it’s very valuable to hear well-founded arguments against any popular belief and Dependency Injection is one of these. “I have started disliking the consequence of this strategy very much: All coupling in my system becomes implicit and harder to understand. I have instead reverted to using design patterns like the Singleton pattern, but with a slight twist.”
• Computer Programmers Learn Tough Lesson in Sharing – WSJ.com – A balanced presentation of pair-programming including both benefits and issues. A key point: It is a skill that must be learned (to respect the other one, give her space, be aware of how your behavior is perceived by her, …).
• Kent Beck: Functional TDD: A Clash of Cultures – TDD has been developed for object-oriented languages and applying it to a functional language with strong type brings interesting challenges. Also a good summary of the benefits of TDD: double checking of the logic (by the implementation and the, preferably quite different, test), solution decomposition (focus on part of the problem, once solve be sure it stays solved),  automatic checking of correctness, outside in design (API first, implementation after that). Plus the pleasant experience of the continuous cycle of tension (failing test) – relief (green).
• Paul Callaghan: Thinking Functionally with Haskell: Types? Tests? We Need a New Word – Powerful type systems eliminate possibility of defects thus venturing into the domain of testing – what can they offer and where the new border and symbiosis between types and tests will be?
• Tales from the Ops Side: Black Friday – an interesting and exciting view into the life of operations engineers one day when all went wrong. Key learnings: Many interdependant components are difficult to reason about; good monitoring and communication are crucial. The post refers to an interesting concept of Recovery-Oriented Computing, i.e. failures are inevitable and their prediction is nearly impossible thus we must focus on making the systems able to survive failures (e.g. vi damage containment, automatic fault detection, component-level restartability).
• Groovy: The road map for the popular JVM language – why was Groovy created (as Java companion focused on productivity), key changes in Groovy 2.0 (more suport for static typing, Java 7, modularity with speed as a side-effect) and in the future Groovy 3.0 (invokedynamic everywhere, more Groovy written in itself).
• Martin Fowler: Key Points from NoSQL Distilled – an overview of why NoSQL, data models, distribution models, consistency, map-reduce, polyglot persistence, criteries for choosing a database.
• You’re a Top Developer! – a surprising hypothesis that “90% of all developers never read a programming blog, never have any side projects to learn something new, and never spend any time or effort outside work hours to improve”. However I haven’t seen any data to back that up (the author only quotes  Peopleware) and the author doesn’t propose any explanation for the fact. I’d really like to know if it is true and why it is so.

Business & Agile

• Experimentation Is The New Planning – “You have no idea what’s going to happen to your industry. That’s why you build your organization into an engine of possibility.” We need “to  continually develop options and explore possibilities” to survive in the ever-changing conditions. Successful strategies emerge from the many ongoing experiments. However, “For emergent strategy to be successful, there must be enough autonomy, freedom, and slack in the system for people and resources to connect in a peer-to-peer way”.
• The Buy-vs-Build Shift (part 1) – Buy to reduce risk of failure (however true agile development – with frequent deliveries and feedback-driven direction – may be cheaper and more importantly can tailor the product to the actual needs) and to avoid inefficiecy of development (but it doesn’t need to be so with agile). “[..] in projects with long cycle times (years) there is a tendency for the business to be somewhat speculative and request all functionality that they can think of [..] With prioritised iterative delivery the business can halt a project when all features that are actually needed have been completed. [..] it does reduce the amount of features that are implemented, and based on my experience, quite substantially so.” Today’s development with e.g. TDD, powerful IDEs supporting automated refactoring, powerful development/production machine, the all-knowing Internet may be much more efficient.
• European entrepreneurs – Les misérables – A good analysis of why it is much more difficult to be an entrepreneur in Europe than in USA (the strong negative impact of a business failure, lack of local investors, cost of firing people) and the decline of European entrepreneurship since 19th century/WW1.
• U.S. Government Accountability Office: Effective Practices and Federal Challenges in Applying Agile Methods – US government considers agile effective; description of the useful practices and of challenges

Tools

• Command line tools for the Google Data APIs – create calendar tasks with reminders, list [today's] task; edit a doc in a local editor (vim, …), get/upload docs; send photos to Picasa, … .

Clojure Corner

• 10-minute Emacs for Clojure – getting started with Emacs for Clojure – install & config & basic usage for Emacs newbies (though no REPL integration yet)
• Keep IT Simply Simple: First month @Runa Inc. – Clojure shop in Silicon Valley – brief post about using Clojure in the wild. Some points: TDD works splendidly; frameworks are not necessary; Clojure can be really fast (<= type hinting, memoziation, performant data structures + occasional Java code)
• Blackstag: Why Clojure? The author describes the set of reasons that have led him to Clojure – and those that actually made him stick with it. “[..] what I like the most about Clojure is that it brings together the best of what many languages have to offer while not forcing it all upon me and, in doing so, has provided a good balance between power and flexibility.” “With Clojure I accomplish more and have found a greater sense of happiness with the work I am doing.”

Programming Like Kent Beck

Republished from blog.iterate.no with the permission of my co-authors Stig Bergestad and Krzysztof Grodzicki.

Three of us, namely Stig, Krzysztof, and Jakub, have had the pleasure of spending a week with Kent Beck during Iterate Code Camp 2012, working together on a project and learning programming best practices. We would like to share the valuable lessons that we have learnt and that made us better programmers (or so we would like to think at least).

(Unit) Testing Swiss Knife: All the Tools You Wanted to Know

I love testing. And I like productivity. There are many tools and libraries that make writing tests easier, more convenient, more fun. I would like to introduce here those that I found the most useful during the years, from selected advanced features of JUnit to assertion libraries, powerful behavior/fault injection, testing of database-related code, and finally to boosting your testing productivity hundredfold with Groovy.

This post accompanies my JavaZone 2012 lightning talk and goes more in depth and introduces additional tools and tips.

Help, My Code Isn’t Testable! Do I Need to Fix the Design?

Our code is often untestable because there is no easy way to “sense¹” the results in a good way and because the code depends on external data/functionality without making it possible to replace or modify these during a test (it’s missing a seam², i.e. a place where the behavior of the code can be changed without modifying the code itself). In such cases the best thing to do is to fix the design to make the code testable instead of trying to write a brittle and slow integration test. Let’s see an example of such code and how to fix it.