Archive for the ‘Testing’ Category
Posted by Jakub Holý on October 31, 2012
- David Veksler: Some lesser-known truths about programming – things newcomers into the field of IT don’t know and don’t expect, true and an interesting read. Not backed by good data but anyway. F.ex.: “[..] a programmer spends about 10-20% of his time writing code [..] much of the other 90% thinking, researching, and experimenting”. “A good programmer is ten times more productive than an average programmer. A great programmer is 20-100 times more productive than the average [..]” “Bad programmers write code which lacks conceptual integrity, non-redundancy, hierarchy, and patterns, and so is very difficult to refactor.” “Continuous change leads to software rot, which erodes the conceptual integrity of the original design.” “A 2004 study found that most software projects (51%) will fail in a critical aspect, and 15% will fail totally.”
- Brett L. Schuchert: Modern Mocking Tools and Black Magic – An example of power corrupting – interesting for two reasons: a good analysis of a poorly written piece of code and discussion of the code injection black magic (JMockIt) vs. actually breaking dependencies to enable tests. The author presents a typical example of low-quality method (mixing multiple concerns, mixing different levels of abstractions, untestable due to a hardcoded use of an external call) and discusses ways to improve it and to make it testable. Recommended to read.
- It’s Not About the Unit Tests – Learning from iOS Developers: iOS developers don’t do much testing yet they manage to produce high quality. How is that possible? The key isn’t testing itself, but caring for the code. (Of course, iOS is little special: small apps, no legacy, a powerful platform that does lot for the apps, very visual apps.) “It’s not about the practices. It’s about the spirit and intent behind them, and how they are applied.” (M. Fowler had a similar observation about a team that used mock-based testing exclusively and thus lacked integration tests yet all worked. [I’ve lost the link to the post and would be grateful for it])
- Java Code Quality Tools – Overview – brief descriptions of 44 quality-related tools including some interesting tools and Eclipse plugins I didn’t know or knew but forgot. F.ex. analysis of dependencies with JBoss Tattletale or JarAnalyzer, Clirr to check libraries for source and binary backwards compatibility, JDiff generates JavaDoc-based report of removed/added/changed in an API. Spoon – read and check or transform Java code. Java PathFinder (NASA) – special JVM capable of checking all execution path to discover concurrency defects etc.
- DirB, Directory Bookmarks for Bash (home) – moving efficiently among favourite directories (
s <name> to create a bookmark for pwd,
g <bookmark | relative/abs dir path> to enter a dir (=> works both for bookmarks and as a replacement for cd); also support for relative path bookmarks & more;
sl lists bookmakrs in the last used order) (You might also want to check out Autojump, described in Dec 11; bashmarks is another similar project. Another similar project is rupa’s z and j2 and the fish clone z-fish)
- Jon Pither: Clojure at a Bank – Moving from Java – the justification (productivity, dynamism, FP a better match for the domain) and process behind moving from Java to Clojure with a monolithic 1M LOC Spring/Hibernate app. (Random quotes: “I had used some dynamical languages before and it was quite obvious that we were essentially forcing lots of schema and type definition on to a problem domain that just didn’t want or need it.” “[..] it [dependency injection] just looks redundant in retrospect now that I’m working 95% with FP code.”) There is also a EuroClojure talk about their experiences one year later (35 min).
- Prismatic’s “Graph” at Strange Loop – an interesting desing problem, its solution, and a resulting OSS library. The problem: How to break a large function into independently usable small ones that might depend on each other without ever needing to recompute a value once the function producing is called. The solution: Graph – “Graph is a simple, declarative abstraction to express compositional structure.” (Enabling explicit declaration of data dependencies and pluging in different implementations.)
- The Oblong: Blog about 2/3 D game programming in Clojure, starting from scratch (w/o an engine); interesting experiences
- Ironclad: Steam Legions – Clojure game development battle report (the game on Github)
- Building the Wishlisted.org webapp in Clojure – experiences from learning Clojure for real by building a webapp in Noir
- Clojure vs. Scala smackdown (“Just kidding with the title of this post :-)”) – a short post with interesting discussion. Dmitri Sotnikov’s opinion resonates with me: “I found that for me Clojure wins on simplicity and consistency. While it looks more alien initially, once you learn the basics, you just reuse the same patterns everywhere.” Some more comments: “One major concern was maintainability, since it’s fairly easy to write very dense code. This turned out to not be a problem in practice. Because Clojure code is written as a tree, refactoring it is very easy.” REPL seems to be a big win (applies to Scala too). Scala’s type system might get tedious and learning its quirks takes time but there is lot of potential and both have they strong sides.
- Code Fatigue – discussion of the advantages of learning, using, and combining the (many) standard Clojure functions instead of a “basic solution” using recursion etc. The argument is in favor of higher-level code with less complexity in the form of branching, recursion, nested expressions etc. and thus less mental fatigue.
A classic test only cares about the final state – not how that state was derived. Mockist tests are thus more coupled to the implementation [emphasis mine] of a method. Changing the nature of calls to collaborators usually cause a mockist test to break.
– Martin Fowler in his classical Mocks Aren’t Stubs
I’m afraid of code. When I see a big pile of code, I get scared ;-). Some classes and method make me cry. I had troubles explaining why I prefer short pieces of code keeping the same level of abstraction, cohesive and loosely coupled. The following quote captures the essence – improved communication.
One way to improve communication is to reduce the need for it and the same can be said for code. […] Since we tend to read code more than write it, anything we can do to reduce the need to read code is time well invested in the life of a project.
– Brett L. Schuchert in Modern Mocking Tools and Black Magic – An example of power corrupting justifying extraction of code into another class or method
Posted in General, Languages, Testing, Tools, Top links of month | Tagged: bash, clojure, java, legacy, productivity, quality, scala | Comments Off on Most interesting links of October ’12
Posted by Jakub Holý on October 27, 2012
Kent Beck in his recent post Functional TDD: A Clash of Cultures summarizes well the key principles and benefits that underlie test-driven development. I think it is really worthwhile becoming aware of and thinking over these foundation stones of TDD (and testing in general). Knowing them enables you to apply TDD in the most effective way with respect to a particular context to gain the maximum of these benefits. People that do not really understand the value of their tests and TDD tend to write hard to maintain tests of limited (and occasionally even negative) value. Are you one of them?
Read the rest of this entry »
Posted in Testing | Tagged: kent beck, opinion, tdd | Comments Off on Do You Know Why You Are Testing?! (On The Principles Underlying TDD)
Posted by Jakub Holý on September 30, 2012
- Johannes Brodwall: This dependency injection madness must end! – it’s very valuable to hear well-founded arguments against any popular belief and Dependency Injection is one of these. “I have started disliking the consequence of this strategy very much: All coupling in my system becomes implicit and harder to understand. I have instead reverted to using design patterns like the Singleton pattern, but with a slight twist.”
- Computer Programmers Learn Tough Lesson in Sharing – WSJ.com – A balanced presentation of pair-programming including both benefits and issues. A key point: It is a skill that must be learned (to respect the other one, give her space, be aware of how your behavior is perceived by her, …).
- Kent Beck: Functional TDD: A Clash of Cultures – TDD has been developed for object-oriented languages and applying it to a functional language with strong type brings interesting challenges. Also a good summary of the benefits of TDD: double checking of the logic (by the implementation and the, preferably quite different, test), solution decomposition (focus on part of the problem, once solve be sure it stays solved), automatic checking of correctness, outside in design (API first, implementation after that). Plus the pleasant experience of the continuous cycle of tension (failing test) – relief (green).
- Paul Callaghan: Thinking Functionally with Haskell: Types? Tests? We Need a New Word – Powerful type systems eliminate possibility of defects thus venturing into the domain of testing – what can they offer and where the new border and symbiosis between types and tests will be?
- Tales from the Ops Side: Black Friday – an interesting and exciting view into the life of operations engineers one day when all went wrong. Key learnings: Many interdependant components are difficult to reason about; good monitoring and communication are crucial. The post refers to an interesting concept of Recovery-Oriented Computing, i.e. failures are inevitable and their prediction is nearly impossible thus we must focus on making the systems able to survive failures (e.g. vi damage containment, automatic fault detection, component-level restartability).
- Groovy: The road map for the popular JVM language – why was Groovy created (as Java companion focused on productivity), key changes in Groovy 2.0 (more suport for static typing, Java 7, modularity with speed as a side-effect) and in the future Groovy 3.0 (invokedynamic everywhere, more Groovy written in itself).
- Martin Fowler: Key Points from NoSQL Distilled – an overview of why NoSQL, data models, distribution models, consistency, map-reduce, polyglot persistence, criteries for choosing a database.
- You’re a Top Developer! – a surprising hypothesis that “90% of all developers never read a programming blog, never have any side projects to learn something new, and never spend any time or effort outside work hours to improve”. However I haven’t seen any data to back that up (the author only quotes Peopleware) and the author doesn’t propose any explanation for the fact. I’d really like to know if it is true and why it is so.
Business & Agile
- Experimentation Is The New Planning – “You have no idea what’s going to happen to your industry. That’s why you build your organization into an engine of possibility.” We need “to continually develop options and explore possibilities” to survive in the ever-changing conditions. Successful strategies emerge from the many ongoing experiments. However, “For emergent strategy to be successful, there must be enough autonomy, freedom, and slack in the system for people and resources to connect in a peer-to-peer way”.
- The Buy-vs-Build Shift (part 1) – Buy to reduce risk of failure (however true agile development – with frequent deliveries and feedback-driven direction – may be cheaper and more importantly can tailor the product to the actual needs) and to avoid inefficiecy of development (but it doesn’t need to be so with agile). “[..] in projects with long cycle times (years) there is a tendency for the business to be somewhat speculative and request all functionality that they can think of [..] With prioritised iterative delivery the business can halt a project when all features that are actually needed have been completed. [..] it does reduce the amount of features that are implemented, and based on my experience, quite substantially so.” Today’s development with e.g. TDD, powerful IDEs supporting automated refactoring, powerful development/production machine, the all-knowing Internet may be much more efficient.
- European entrepreneurs – Les misérables – A good analysis of why it is much more difficult to be an entrepreneur in Europe than in USA (the strong negative impact of a business failure, lack of local investors, cost of firing people) and the decline of European entrepreneurship since 19th century/WW1.
- U.S. Government Accountability Office: Effective Practices and Federal Challenges in Applying Agile Methods – US government considers agile effective; description of the useful practices and of challenges
- 10-minute Emacs for Clojure – getting started with Emacs for Clojure – install & config & basic usage for Emacs newbies (though no REPL integration yet)
- Keep IT Simply Simple: First month @Runa Inc. – Clojure shop in Silicon Valley – brief post about using Clojure in the wild. Some points: TDD works splendidly; frameworks are not necessary; Clojure can be really fast (<= type hinting, memoziation, performant data structures + occasional Java code)
- Blackstag: Why Clojure? The author describes the set of reasons that have led him to Clojure – and those that actually made him stick with it. “[..] what I like the most about Clojure is that it brings together the best of what many languages have to offer while not forcing it all upon me and, in doing so, has provided a good balance between power and flexibility.” “With Clojure I accomplish more and have found a greater sense of happiness with the work I am doing.”
Posted in General, Testing, Top links of month | Tagged: agile, clojure, fp, groovy, haskell, nosql, spring, tdd, xp | Comments Off on Most interesting links of September ’12
Posted by Jakub Holý on September 12, 2012
Republished from blog.iterate.no with the permission of my co-authors Stig Bergestad and Krzysztof Grodzicki.
Three of us, namely Stig, Krzysztof, and Jakub, have had the pleasure of spending a week with Kent Beck during Iterate Code Camp 2012, working together on a project and learning programming best practices. We would like to share the valuable lessons that we have learnt and that made us better programmers (or so we would like to think at least).
Read the rest of this entry »
Posted in General, Languages, Testing | Tagged: best practices, CleanCode, craftsmanship, java, kent beck, opinion, tdd | Comments Off on Programming Like Kent Beck
Posted by Jakub Holý on September 9, 2012
I love testing. And I like productivity. There are many tools and libraries that make writing tests easier, more convenient, more fun. I would like to introduce here those that I found the most useful during the years, from selected advanced features of JUnit to assertion libraries, powerful behavior/fault injection, testing of database-related code, and finally to boosting your testing productivity hundredfold with Groovy.
This post accompanies my JavaZone 2012 lightning talk and goes more in depth and introduces additional tools and tips.
Read the rest of this entry »
Posted in Languages, Testing, Tools | Tagged: java, JavaZone, junit | 5 Comments »
Posted by Jakub Holý on September 9, 2012
Our code is often untestable because there is no easy way to “sense1” the results in a good way and because the code depends on external data/functionality without making it possible to replace or modify these during a test (it’s missing a seam2, i.e. a place where the behavior of the code can be changed without modifying the code itself). In such cases the best thing to do is to fix the design to make the code testable instead of trying to write a brittle and slow integration test. Let’s see an example of such code and how to fix it.
Read the rest of this entry »
Posted in Testing | Tagged: CleanCode, design, legacy, testability | Comments Off on Help, My Code Isn’t Testable! Do I Need to Fix the Design?
Posted by Jakub Holý on July 2, 2012
When defining a Zabbix (1.8.2) trigger e.g. to inform you that there are errors in a log file, how do you verify that it is correct? As somebody recommended in a forum, you can use a Calculated Item with a similar expression (the syntax is little different from triggers). Contrary to triggers, the value of a calculated item is easy to see and the historical values are stored so you can check how it evolved. If your trigger expression is complex the you can create multiple calculated items, one for each subexpression.
Read the rest of this entry »
Posted in Testing, Tools | Tagged: monitoring, ops, zabbix | Comments Off on Testing Zabbix Trigger Expressions
Posted by Jakub Holý on June 30, 2012
- Neal Ford: Evolutionary architecture and emergent design: Emergent design in the wild – discusses why not to do Big Design Up Front and how to recognize the “last responsible moment”. Quote: “It’s puzzling that so many organizations continue to use BDUF in the face of so many failures and underachieving projects.“
- Gojko Adzic: How To Solve “Not Enough Time” – everybody in IT complains about too much work. The solution acording to Gojko? Kill software already made that isn’t needed (ex.: deleted 70% functionality that wasn’t used; remember that maintenance costs grow ~ exponentially with size/complexity). Kill software in making that won’t be needed (know the value – effect maps). Kill software that was not successful (if you measure the value of SW, you know whether its existence is worth it). Well written and inspiring. I’m looking forward to killing some SW :-)
- Coding Horror: Postponing decisions to the last responsible moment – a brief and pretty good explanation of this key lean principle together with its connection to YAGNI (and some criticism of LRM by A. Cockburn – I agree with the postscript 2)
- Build The (USS) Enterprise – A fascinating site by an engineer who dares to think big yet manages to stay rooted in reality. A huge inspiration for us all! We tend to think too small. Having a great, inspiring goal is what moves us forward. And people are known to have achieve seemingly impossible things nobody else believed in (how many believed Columbus could reach India by by sailing the opposite direcetion? and even though he actually hasn’t, the impacts of his discovery were tremendous)
- Hooking Users In 3 Steps: An Intro to Habit Testing – to be successful with a new web product in this age of distraction, you need your users to build the habit of using the app regularly (think of Twitter, Facebook, newspaper); this post describes how to find your habitual users, understand them, and optimize the application to turn more occassional users into habitual ones (identify habitual users – learn how they use the app & learn what turns random users into “devotees” – optimize the “habit path”)
- Puppet: Serve configuration from a particular Git branch on demand (original: Git Workflow and Puppet Environments) – at Comoyo we use Puppet to configure all our environments and developers need to test their changes before pushing them to the live environment, preferably without interference from other devs. This post by Adrien of PuppetLabs describes how to enable each developer to have her private branch(es) and have Pupet Master serve the config from the branch on demand, using little puppet environments and hooks magic. (Notice that if using Puppet to serve files then you’ll need to have them inside a module, which is a good practice anyway.)
- DZone reference card for Jetty (DZone login required) – useful, brief overview over Jetty XML and programatic configuration, especially useful is the overview of handlers, use of servlets and webapps, SSL support, websockets
- What’s new in Groovy 2.0? – static type checking, Java 7
- Speed, memory and LOC of Python 3 vs. Java 7 [fixed link] (the computer language benchmark game) – Python tends to take more time (though not terribly more) but is quite economical with memory and visibly more productive (3* less lines of code). Ruby is similar but way slower. Scala is little slower and more memory hunrgy but also more productive. Of course performance is rarely the key factor for picking a language on often it doesn’t matter that much.
Goodhart’s Law: once a metric becomes a target, it loses its meaning as a measure.
- What Sucks about Clojure…and Why You’ll Love It Anyway – 40 min talk by Clojure Programming author Chas Emerick. Some of the negative points: Namespaces – complex (use x refer etc.; no package.* import, …), no forward references; Dynamic scope (with-foo …) has subtle, complex behavior and hard to see => avoid when you can; Using STM (effectively) is hard even though it looks easy, it’s overused, think of it rather as in-memory db with superior integr. with Clojure (ex.: what is the right ref granularity? whole word map? small particle?), nondeterministic (which tx will fail/suceed? – can’t reason) => strange error under high load/mix of operations; JVM: long startup time, …; Macros look easy but are hard (don’t compose nicely with the rest); Function composition – hard to find failure root cause.
- Why is Clojure so slow? (2/2012) – an interesting comparison of Clojure and other languages (C, F#, Groovy etc.) and an analysis of its slow start-up time (metadata and docstring building) and slower performance. (Interestingly, ClojureScript runs it 9* faster.) According to the author, it’s slower partly due to the imutable data structures (nothing new here, we knew we have to offer some performance for the increased safety and robustness). Conclusion: “Clojure is a beautiful, powerful and very useful language, but […] not great for small script-y programs.” Also, the usefulness of the benchmark is limited if you aren’t writing games. Plans for making Clojure leaner and faster are under way.
- Stuart Sierra: Functional Relational Programming with Cascalog – a brief introduction into Cascalog with a valuable background info about Hadoop, MapReduce, relational programming in Clojure. Good links, especially the paper Out of the Tar Pit looks interesting.
Posted in General, Testing, Tools, Top links of month | Tagged: agile, clojure, design, DevOps, groovy, inspiration, lean, leanstartup, performance, puppet, python | Comments Off on Most interesting links of June ’12
Posted by Jakub Holý on May 31, 2012
This was a rich month, bringing some hope for ORM, providing a peep-hole into the bright and awesome future with in-browser video and other cool web-stuff presented at WebRebels 2012 and IDEs providing immediate feedback and visualisation. There were valuable articles about simplicity and quality in software and good talks about the lean startup (i.e. enabling innovation) and other topics.
- M.Fowler: ORM Hate – Why ORM is actually a good solution – a very valuable article where Fowler opposes the popular trend of criticising Object-Relational Mappers such as Hibernate. Yes, using an ORM is difficult and a leaky abstraction – but that’s because the problem of mapping from a rich in-memory object model to a relational store is inherently difficult (and you need to do it with or w/o an ORM tool) and because those last 10-20% of DB access require human intelligence. If you can avoid the need for ORM by using the relational model also in memory or by using a NoSQL database with a data model that fits your in-memory model then it’s great to do so but often you can’t and then using ORM is the best solution. You certainly don’t want to code your own “lightweight” ORM.
- Communicating Sequential Processes: Theory for reasoning about concurrent, interacting processes – an inspirational reading about a much better way to do concurrency than Java threads; “… [CSP] is a language for describing patterns of interaction between concurrent objects. It is supported by an elegant, mathematical theory, a set of proof tools, and an extensive literature.” The beauty is that thanks to the theory behind, you can actually reason about the interactions and verify their correctness, contrary to the feared mess of Java threads. CSP is broadly similar to the popular Actors model and is implemented in Occam while it also influenced Erlang’s concurrency model and Go. The library JSCP brings it to Java. I guess we’re better of using Actors due to their popularity and maturity though the mathematical backing of CSP with the potential of formal proofs of correctness is indeed attractive. Any of the two is better than using threads directly because:
The monitor-threads model provided by Java, whilst easy to understand, proves very difficult to apply safely in any system above a modest level of complexity. One problem is that monitor methods are tightly interdependent, so that their semantics compose in non-trivial ways […]
- Rich Hickey introduces the Reducers library: simplicity in practice – a beautiful example of simplifying something by taking appart all the unrelated but mingled concerns and focus only on those really needed. Whether you’re interested in Clojure or not, you should read the beginning of the post where Hickey explains how the current collection functions based on first (returns 1st element) and rest (returns the remaining ones) mix too many things (ordering, output representation, etc.) and how this “new super-generalized and minimal abstraction for collections” avoids that and thus provides e.g. for doing things in parallel and composing transformation without producing intermediate collections. Beautiful! (PS: I’ve blogged about more examples of pursuing simplicity & gaining power.)
- M. Fowler: Cannot Measure Productivity – a thoughful discussion of why the productivity of programmers is hard/impossible to measure (i.e. you should concentrate on measuring other, more useful metrics) “[..] false measures only make things worse.”
- Gojko Adzic: Redefining software quality – an obligatory read that introduces a holistic view of SW quality and the quality pyramid. The key idea is that there are multiple, vertically organized facets of quality and once a more basic facet is saturated, you should move and and concentrate on the next facet and level of quality. The quality pyramid: Deployable & functionally OK > Performant & secure > Usable > Useful > Successful. Once a particular level is satisfied, it is wasteful to put more effort into it and you’ll bring much more value to the customer by focusing on the next higher level. Gojko: “Yet from what I see most software teams invest, build and test only at the lowest two levels, gold-plating things without a way to explain why that is bad.”
- Is Pair Programming for Me? – the author, who claims to have taught pair programming to 200+500 people, points out that pair programming is a skill that must be (consciously) learned, or actually a number of inter-personal skills. He also describes the cycle people go through when learning it, including a temporary downswing in productivity and negative view of pairing (therefore people should do it at least for 3-4 weeks to overcome the problems and gain the benefits).
- Bret Victor: Inventing on Principle (55 min, see at least the first 5 min) – very inspiring! Victor firmly believes that “creators need an immediate connection to what they create” and demonstrates how this can be achieve when coding image rendering, a game, an algorithm, when designing a circuit. After watching it for few minutes you will think: How could we have been working with such crappy tools without realizing how limited they are? Fortunately people started to apply the idea of an immediate connection between code and the result, f.ex. in LightTable and Bikeshed‘s IDE. On the other hand, there is an evidence that this may be too hard with the current programming languages.
- Eric Ries: Evangelizing for the Lean Startup – entertaining and enriching introduction into an approach for bringing innovation to life – f.ex. in startups – withou failing unnecessary, demonstrated on the example of the author’s failed and successful startup. Many innovators fail because they don’t realize that their key challenge is that they don’t now neither the problem (who are our customers and what they need) nor the solution (the product to satisfy the need) and thus what they need to do is to experiment and learn in the shortest cycles possible. If you wander what the buzz about lean startup is or how to build innovations, this is the ultimite source you should watch. The video has 1h but the first 20-30 min will give you a sufficient overview. The key points summarized by the Iterate lean guru Anders Haugeto:
– There is only one way to measure progress: Progress == The amount of things you have learned from your real customers
– Hence, you need to work a continuous loop to build, measure and learn as fast as possible. Typical iterations, like sprints, are too long, hence inefficient
– Until you have an established product, even recognized engineering practices like TDD, sprints, refactoring, and all the XP-stuff are less important than this feedback cycle
– Even the perfect agile method is nothing, if you’re using it to build the wrong product: How can you know you are heading in the right direction?
Links to Keep
- E. King: Maximizing the Value of Your Stand-up – interesting techniques to try out at your stand-ups – Speed Scrum, Pass-the-Conch Scrum (passing a token randomly to define the order), Time-Box Scrum, Challenge Scrum (the team may ask 1 question each presenter), Impediments-Only Scrum, Award Scrum (reward for best articulation of his/her information), Business Value-Focused Scrum, No-Board Scrum, Whiteboard Scrum, Buddy Scrum (report for sb. else)
- puppet-lint – check code style of your Puppet files
- Guard – cross-platform tool that can watch for file changes and execute actions (“guards”) when a file is changed, useful e.g. to execute tests/style checks only on the files being modified. Includes support for many testing/checking tools and multiple notification means such as Growl.
- ThreadLogic – Thread dump analyzer that understands common patterns found in application servers and enabling the definition of custom patterns. Supports Sun, IBM, and JRockit.
- Dumbster – mock SMTP server for unit testing (start in @Before, get sent messages in the test, stop afterwards)
Kai Thomas Gilb, in a talk proposal for JavaZone 2012:
Accurate estimation is impossible for complex technical projects, but keeping to agreed budgets, and deadlines is achievable by using feedback and change.
- StackOverflow: Clojure Performance Benchmarks – links to various discussions and benchmarks (beware that older results and facts are likely to be outdated). And of course you must keep in mind that 1) benchmark only measure what they measure, e.g. the outcomes cannot be generalized and that 2) benchmark results aren’t relevant for your problem unless you’re doing exactly the kind of operations being benchmarked (e.g. who cares that X is 100 ms slower if your code spends 1s waiting for a XML file download?) (Craig Andera had a pretty good experience report from webapp performance testing including what (not) to do)
- A good wrapup of the EuroClojure conference by Deon Garrett. Such a pity I missed it!
- Goldberg (at GitHub) – Johann Sebastian Bach’s Goldberg Variations in Overtone by @ctford; using Overtone and Clojure to build up mathematical and functional definitions of canons. Deon Garrett: “Go right now and download the code from Chris’ talk. If you don’t know Clojure, use this as an excuse to learn it – it’s that good.”
Rich Hickey interviewed by M. Fogus:
Reducing incidental complexity is a primary focus of Clojure, and you could dig into how it does that in every area.
In particular, the use of objects to represent simple informational data is almost criminal in its generation of per-piece-of-information micro-languages, i.e. the class methods, versus far more powerful, declarative, and generic methods like relational algebra. Inventing a class with its own interface to hold a piece of information is like inventing a new language to write every short story. This is anti-reuse, and, I think, results in an explosion of code in typical OO applications.
Posted in Databases, General, Testing, Tools, Top links of month | Tagged: agile, clojure, concurrency, design, html5, leanstartup, ORM, scrum, smtp, trends | Comments Off on Most interesting links of May ’12
Posted by Jakub Holý on April 30, 2012
- V. Duarte: Story Points Considered Harmful – Or why the future of estimation is really in our past… (also as 1h video) – thoughtful and data-backed claim that there is a much cheaper way for estimating work throughput than estimating each story in story points (SP) and that is simply counting the stories. Even though their sizes differ, over (not that much) longer periods, where it really matters, these differences will even out. The author argues that estimating in number of stories provides the same reliability and benefits as SP and is much easier. (Keep in mind that estimation is just an attempt at predicting the future and humans are proved to be terrible at doing that; why to pretend that we can do it?) I’d recommand this to anybody doing Scrum and similar.
- M. Fowler: Test Coverage – it’s obvious that increasing test coverage for the sake of test coverage it’s a nonsense but some people still need to be reminded of it :-). Fowler explains what the real benefit of test coverage measurements is and how to use it for good instead of for evil.
- Brian Marick: How to Misuse Code Coverage (pdf) – cited a lot by Fowler in his article, this is really a good paper. Marick has participated in the development of several code coverage tools and understands well their limitations. One of the key points is that code coverage tools can discover only one class of test weakness (not testing some paths through your code) but cannot discover that you are missing some code you should have (e.g. when you check only for two of three possible return values). Thus the code coverage metric tells you “this code isn’t well tested, are you sure you don’t to look more into it”? It’s crucial not to write tests so as to increase the code coverage; look at the code and improve the test without any regard for coverage. You may thus decrease the likeliness of both the class of problems.
- A Year with MongoDB – Kiip has found out that Mongo isn’t the best choice for them (having 240GB, 500+ operations/s, 85M docs and their specific usage of the store) and migrated to the combination of Riak (key-value store) and PostgreSQL. Some of the issues they hit are slow counts and limit/offset queries due to using non-counting B-trees for indexing, memory management that could be more intelligent and tuned for the use to make sure the data needed is indeed in RAM, no built-in support for compressing key names (their size adds up as they’re repeated in each document; you’ve to compress them [user -> u etc.] in the client if you want to), limited concurrency due to process-wide write lock (which becomes a problem if the write’s aren’t short enough w.r.t. number of ops/s, e.g. because data isn’t in RAM and/or the query is complicated), safe settings (waiting for a write to finish, …) off by default, offline-only table compaction (w/o it the disk usage grows unbounded). The lessons learnt for me: Know your storage, its weaknesses and intended way of usage, and make sure it matches your needs.
- Rudolf Winestock: The Lisp Curse – Lisp’s expressive power is actually a cause of its lack of momentum because it’s so easy to implement anything that people have no need to join forces and thus there are many half-baked (“works-for-me”) solutions for anything – but no complete, generally accepted one. An interesting essay. “Lisp is so powerful that problems which are technical issues in other programming languages are social issues in Lisp.”
- Understanding JDBC Internals & Timeout Configuration – the article itself could have been written better but it conveys the important information that configuring timeouts for JDBC isn’t trivial because they need to be set correctly at different levels and without a socket timeout set in a driver-specific way it can hang forever if the DB cannot be reached due to network/system failure
- Circos: An Amazing Tool for Visualizing Big Data – this article is interesting primarily for its combination of Google Analytics API, Neo4J and an unusual data visualization with circular graphs
- CRaSH: Extensible shell for the JVM (docs) – a shell that you can embedd into a web server as a WAR, run standalone or attach to a running JVM, connect to it via SSH or Telnet, and use it to execute commands against the JVM. Some commands: configure loggers, control threads, monitor the system (mem, threads, ..), connect/issue queries via JDBC. More commands can be written in Groovy. There is a whole set of commands for working with JCR. Pluggable authentication.
Posted in Databases, General, Testing, Tools, Top links of month | Tagged: agile, bigdata, nosql, Testing | Comments Off on Most interesting links of April ’12