The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘design’

Most interesting links of July ’14

Posted by Jakub Holý on July 31, 2014

Recommended Readings

  • Video: The Unreasonable Effectiveness of Dynamic Typing for Practical Programs – a static-typing zealot turned friend of dynamic typing under the experience of real-world projects and problems shares thoughts about the limits of type systems (f.ex. both energy and torque are measured in N*m yet cannot be combined) and their cost: according to the Hanenberg’s experiment about static and dynamic typing => the time required to handle the time chacker > time to debug the errors that it would have caught. According to a review of issues at GitHub, only 2% of reported issues for JS, Clojure, Python, and Ruby are type errors and for a large, closed-source Python project type/name/attribute errors were 1%. “I have come to believe that tests are a much better investment [than static typing].” Rigorous type system/model => limited applicability (due to different needs) <=> modelling some things with types doesn’t cut it. “Are the costs of static typing offset by a few percent fewer defects? Is agility more important than reliability?” “Static types are anti-modular” – too a tight coupling. “Static type checking comes at the expense of complexity, brittleness and a tendency to monoliths.
    (Personally I miss static typing – but that is perhaps due to having relied on it for so long.)
  • ThoughtWorks Tech Radar July 2014 (pdf): f.ex. Ansible in Adapt, Masterless Chef/Puppet in Trial, Machine image as a build artifact: Trial, PostgreSQL for NoSQL: Trial, Adopt Dropwizard (Rest 4 Java), Go lang, Reactive Extensions across langs [JH: RxJava, RxJS, ..]; Asses Property-based (generative) testing, … . Other highlights: Mapbox (open-source mapping platform), OpenID Connect as a less complex and thus promising alternative to SAML/generic OAuth, Pacto/Pact for Consumer-Driven Contracts (contract => simulate consumers/stubb producers => test your REST clients against the contract so that the rest of tests can assume it is correct and use a stubbed client), Swagger for REST documentation.
  • The madness of layered architecture – a nice critique of over-designed “enterprise” apps, why that is a problem (SRP, cost of code, unclear where to do a change, ….), why it is different from the successful layered network stack of Ethernet/IP/TCP/… (because in an app, all layers are on the same level of abstraction); bottom line: do not add a layer unless you have a really good reason (hint: the advice of a consultant/speaker does not count as one)
  • Key Takeaway Points and Lessons Learned from QCon New York 2014 (viz @RiczWest) – “[..] deep insights into real-world architectures and state of the art software development practices, from a practioner’s perspective.” – architectures of Fb, Foursquare etc., continuous delivery, creating culture, real world functional programming, … .
  • Questioning the Lambda Architecture (J. Kreps of LinkedIn) – maintaining the same processing in two very different systems (one batch, one stream & real-time) is a maintenance nightmare => improve the RT/stream processing to handle re-processing and thus both (using e.g. Kafka to store the data and thus be able to re-play them)
  • Google: Checklist for mobile website improvement
  • Google Dataflow and the transition from batch to stream processing – G. Dataflow might not be a Hadoop killer due to requiring that the data are in the Google Cloud but the trend is clear, going away from batch processing to more stream-oriented processing with tools like Spark, Flume etc. that are faster thanks to using memory better and more flexible thanks to not being limited to the rigitd two-stage model of map-reduce. (Reportedly, Google – the one that made Map-Reduce popular – doesn’t use it anymore.)
  • OS X: Extract JDK to folder, without running installer

Society, economics, people

  • HBR: The Power of Meeting Your Employees’ Needs – people feel better, perform better, are more engaged and likely to stay longer (=> profitability) when 4 basic needs are met: physical [energy] renewal (=> give opportunity, encourage to take a nap or do whatever that helps), value – feeling of being valued by the company, ability to focus, purpose (i.e. serving something larger than ourselves). “What’s surprising about our survey’s results is how dramatically and positively getting these needs met is correlated with every variable that influences performance. It would be statistically significant if meeting a given need correlated with a rise of even one or two percentage points in a performance variable such as engagement, or retention. Instead, we found that meeting even one of the four core needs had a dramatic impact on every performance variable we studied. [..] when all four needs are met, the effect on engagement rises from 50% for one need, to 125%. Engagement, in turn, has been positively correlated with profitability. [..] employers with the most engaged employees were 22% more profitable than those with the least engaged employees.
    [..] those who were encouraged to take intermittent breaks reported they were 50% more engaged, more than twice as likely to stay with the company, and twice as healthy overall. Valuing and encouraging renewal requires no financial investment. What it does require is a willingness among leaders to test their longstanding assumption that that performance is best measured by the number of hours employees puts in – and the more continuous the better — rather than by the value they generate, however they choose to do their work.
  • The Pitchforks Are Coming… For Us Plutocrats – increasing inequality will eventually lead to the collapse of the sysem (at least so does teach the history). It is people – primarily the middle class – that are the source of the wealth of the society, they produce and also consume most. Thus it is necessary to support them …
  • Why the U.S. Corporate World Became ‘A Bull Market for Corruption’ – Enron, GM, Goldman Sachs, … – we hear more and more the names of large corporations in the context of negligence and misues of their customers and investors. It seems that leadership (in the lead by example sense) has died out as well as the feeling of responsibility when one wields power over her customers/investors/markets. Instead, we have the me-first and  money at any cost thinking. Organizations are designed to shield higher-ups from responsibility (meetings with no records…). High pay for short term gains, failure to punish high ranking people.
  • (US) This is what happened when I drove my Mercedes to pick up food stamps – the experience of life in poverty after dropping down from $125k to $25k/year in two months due to childbirth, real estate market crash, and loss of a job. “Using the coupons was even worse. The stares, the faux concern, the pity, the outrage — I hated it. [..] That’s the funny thing about being poor. Everyone has an opinion on it, and everyone feels entitled to share. [..] Poverty is a circumstance, not a value judgment. I still have to remind myself sometimes that I was my harshest critic. That the judgment of the disadvantaged comes not just from conservative politicians and Internet trolls. It came from me, even as I was living it.

Clojure Corner

  • Isomorphic Clojure[Script], part I – enjoying all the benefits of Single-Page Apps while avoiding their drawbacks (SEO, slower page load, accessibility etc.) – a SPA that can be pre-rendered by the server. Using Om/React, JDK8 with the Nashorn JS engine, core.async, Sente (bi-dirrectional HTTP/WS communication over core.async) and Clojure in the JVM, ClojureScript in Nashorn in the JVM, and ClojureScript in the browser. Example app: Omelette.
  • clj-crud: a relatively feature-complete example of a Clojure web (4/2014; GitHub) – using Component, Liberator (REST), Datascript + Quiescent (=> React.js), Enlive, Friend etc. including couple of unit-test and ui-test libraries
  • Conclujon: Acceptance testing tool (α), Clojure reimplementation of Concordion, a beautifully simple ADD tool
  • dynalint: human-friendly error messages during dev – Clojure typically provides little helpful and sometimes confusing error messages thrown from somewhere deep in the implementation, such as “Don’t know how to create ISeq from: java.lang.Long at clojure.lang.RT.seqFrom” while we want st. like “First argument to clojure.core/first must be seqable: 1 (instance of class java.lang.Long” – and that’s what Dynalint does. In the tradition of defensive programming, it adds checks and good error messages to Vars at runtime. You typically run it only during dev, triggering it from the REPL.
  • Grimoire (Reid McKenzie) – a more up-to-date replacement for ClojureDocs
  • Adam Bard’s Top Clojure Articles for beginners and intermediate Clojure devs – f.ex. Five Mistakes Clojure Newbies Make, Acceptable Error Handling in Clojure, Clojure Reducers for Mortals
  • J. Wilk: Isolating External Dependencies in Clojure – a nice overview of the options and their pros and cons – with-redefs, alter-var-root, Midje (using alter-var-root in a more controlled manner), higher-order-functions (#1!) etc.
  • philandstuff’s detailed notes from Euroclojure 2014

Tools/Libs

  • NixOS (via @bodil) – a new interesting “purely functional” Linux distribution – system configuration is fully declarative (think of Puppet/Chef) and it is always trivial to roll back, you can have multiple versions of a package, users can install non-global SW
  • InfluxDB – time series, metrics, and events DB that scales; contrary to Graphite it can store richer data than Graphite and its single value; additional highlights: authorization for individual data, roll-up/clean old data, https API. Written in Go.

Posted in General, Languages, Top links of month | Tagged: , , , , , , , , , | Comments Off

Most interesting links of December ’13

Posted by Jakub Holý on December 31, 2013

Recommended Readings

Society

  • HBR: Want to Build Resilience? Kill the Complexity – a highly interesting, thought provoking article relevant both to technology in particular and the society in general; f.ex.: more security features are bad for they make us behave less safely (risk compensation) and are more fragile w.r.t. unexpected events. “Complexity is a clear and present danger to both firms and the global financial system: it makes both much harder to manage, govern, audit, regulate and support effectively in times of crisis. [..] Combine complex, Robust-Yet-Fragile systems, risk-compensating human psyches, and risk-homeostatic organizational cultures, and you inevitably get catastrophes of all kinds: meltdowns, economic crises, and the like.” The solution to future financial crisis is primarily not more regulation but simplification of the system – to make it easier to police, tougher to game. We also need to decrease interconnectednes (of banks etc.), one of the primary sources of complexity. Also a great example of US Army combatting complex, high-risk situations by employing “devil’s advocates / professional skeptics” trained to help “avoid the perils of overconfidence, strategic brittleness, and groupthink. The goal is to respectfully help leaders in complex situations unearth untested assumptions, consider alternative interpretations and “think like the other”“.
  • The Dark Side of Technology – technologies provide great opportunities – but also risks we should be aware of – they create a world of mounting performance pressure for all of us (individuals, companies, states), accelerate the rate of change, increasing uncertanity (=> risk of Taleb’s black swans). “All of this mounting pressure has an understandable but very dangerous consequence. It draws out and intensifies certain cognitive biases [..]” – magnify our perception of risk, shrink our time horizons, foster a more and more reactive approach to the world, the “if you win, I will lose” view, erode our ability to trust anyone – and “combined effect of these cognitive biases increases the temptation to use these new digital infrastructures in a dysfunctional way: surveillance and control in all aspects of our economic, social and political life.” => “significantly increase[d] the likelihood of an economic, social and political backlash, driven by an unholy alliance between those who have power today and those who have achieved some modest degree of income and success.
    Complexity theory: the more connected a system is, the more vulnerable it becomes to cascades of disruptive information/action.
  • What Do Government Agencies Have Against 23andMe, Uber, and Airbnb? – innovative startups do not fit into established rules and thus bureaucrats do not know how to handle them and resort to their favourite weapon: saying no, i.e. enforcing rules that harm them (f.ex. France recently passed a law that requires Uber etc. drivers to wait 15 min before picking up a customer so that established taxi services have it easier; wot?!)
  • Nonviolent communication in action – wonderful stories about NVC being applied in difficult situations with a great success

Tech

  • D. Nolen: The Future of JavaScript MVC Frameworks – highly recommended thought food – about React.js, disadvantages of event-based UI, benefits of immutability, performance, the ClojureScript React wrapper Om  – “I simply don’t believe in event oriented MVC systems – the flame graph above says it all. [...] Hopefully this gives fans of the current crop of JS MVCs and even people who believe in just using plain JavaScript and jQuery some food for thought. I’ve shown that a compile to JavaScript language that uses slower data structures ends up faster than a reasonably fast competitor for rich user interfaces. To top it off Om TodoMVC with the same bells and whistles as everyone else weighs in at ~260 lines of code
  • Quora: Michael Wolfe’s answer to Engineering Management: Why are software development task estimations regularly off by a factor of 2-3? – a wonderful story explaining to a layman why estimation is hard, on the example of a hike from SF to LA
  • Style Guide for JavaScript/Node.js by Felix Geisendörfer, recommended by a respectable web dev; nothing groudn breaking I suppose but great start for a team’s standards
  • Johannes Brodwall: Why I stopped using Spring [IoC] – worth to read criticism of Spring by a respected and experienced architect and developer; summary – dependency injection is good bug “magical” frameworks decrease understandability and encourage unnecessarily complex code => smaller code, , easier to navigate and understand and easier to test
  • Misunderstanding technical debt – a brief discussion of the various forms of tech. debt (crappy code x misaligned design and problem domain x competence debt)
  • Tension and Flaws Before Health Website Crash – surprising lack of understanding and tensions between the government and contractors on HealthCare.gov – “a huge gap between the administration’s grand hopes and the practicalities of building a website that could function on opening day” – also terribly decision making, shifting requirements (what news!), management’s lack of decision power, CGI’s blame-shifting. A nice horror story. The former head knew that they should “greatly simplify the site’s functions” – but the current head wasn’t able to “talk them out of it”.
  • The Log: What every software engineer should know about real-time data’s unifying abstraction – logs are everywhere and especially important in distributed apps – DB logs, append-only logs, transaction logs – “You can’t fully understand databases, NoSQL stores, key value stores, replication, paxos, hadoop, version control, or almost any software system without understanding logs” – I have only read a small part but it looks useful
  • What I Wish I Knew When Learning Haskell tl;dr
  • Better Than Unit Tests – a good overview of testing approaches beyond unit tests – including “Automated Contract Testing” (ability to define a contract for a web service, use it to test it and to simulate it; see Internet of Strings for more info), Property-based Testing (test generic properties using random data/calls as with Quickcheck), Fault Injection (run on multiple VMs, simulate network failures), Simulation Testing as with Simulant.
  • Use #NoEstimates to create options and deliver value reliably – a brief post with an example of an estimation-based vs. no-estimates project (i.e. more focus on delivering early, discovery)
  • How Google Sold Its Engineers on Management – managers may be useful after all :-); a report about Google’s research into management and subsequent (sometimes radical) improvements in management style/skills and people satisfaction; I love that Google hasn’t HR but “people ops”
  • Roy Osherove: Technical Disobedience – take nothing for granted, don’t let the system/process stop you, be creative about finding ways to improve your team’s productivity; there always is a way. Nice examples.
  • Uncle Bob: Extreme Programming, a Reflection – a reflection on changes in the past ~ 14 years since XP that have seen many of the “extreme” practices becoming mainstream
  • The Anti-Meeting Manifesto – essentially a checklist and tips for limitting meetings to minimum

Other

Talks

  • Pete Hunt: React: Rethinking best practices (JSConf 2013, 30 min) – one of the most interesting talks about frontend development, design, and performance I have heard this year, highly recommended. Facebook’s React JavaScript framework  is a fresh and innovative challenger in the MVC field. It is worthwile to learn why they parted ways with the popular approach of templates (spoiler: concern separation, cohesion x coupling, performance). Their approach with virtual DOM enables some cool things (run in Node, provide HTML5-like events in any browser with consistent behavior, …). Key: templates are actually tightly coupled to display logic (controllers) via the model view tailored for them (i.e. Controller must know what specific data & in what form View needs) => follow cohesion and keep them together componets, separate from other components and back-end code. Also, state changing over time at many places is hard => re-render the whole app rather than in-place updates. Also, the ClojureScript Om wrapper enables even more performance optimizations thanks to immutable data structures etc.
  • David Pollak: Some musings on Scala and Clojure by a long time Scala dude (46 min) – a subjective but balanced comparison of Scala and Clojure and their strengths/weaknesses by the author of the Scala Lift framework (doing Scala since 2006, Clojure since 2013)

Clojure Corner

Tools/Libs

  • Apache Sirona – a new monitoring tool in the Apache incubator – “a simple but extensible monitoring solution for Java applications” with support for HTTP, JDBC, JAX-RS, CDI, ehcache, with data published e.g. to Graphite or Square Cube. It is still very new.
  • GenieJS – Ctrl-Space to popup a command-prompt for your web page, inspired by Alfred (type ‘ to see all possible commands)

Favourite Quotes

A good #agile team considers their backlog inaccurate. It is merely a list of assumptions that must be tested & refined by shipping product
- @mick maguire 12/10

Ada Lovelace (1st program), Grace Hopper (1st compiler), Adele Goldberg (1st OO language), why would anyone think women aren’t in computing?
@Dan North 12/11

There will always be a shortage of talented, self-motivated creative professionals who will unquestioningly follow orders.
@Thomas K Nilsson 12/7

Estimation paradox = If something unpredictable happens, predict how long it will take to fix it
me 12/7

IT systems can be inspired by AK-47 a.k.a. Kalashnikov. The rifle was purposefully designed to be simple and to be tolerant to imperfections in most parts; as a result, it required essentially no maintenance and was extremely reliable.
– summarized from Roman Pichlík’s Odkaz Michaila Kalašnikova softwarovému vývoji

Posted in General, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , , , , | Comments Off

Bad Code: Are We Thinking Too Little?

Posted by Jakub Holý on December 31, 2013

Do we not think enough when coding? Do we jump to the first solution without really considering the problem, without trying to analyze and decompose it and understand the components and orthogonal forces invovled? Is that the cause of bad code (together with time press) and the reason why we typically see a “patchvolution” rather than evolution (of design)?

For example I want a certain item on my list shown grayed out because it has been marked for removal or is currently being edited and I therefore add a flag called isDisabled. But if I really thought about it, I would likely call it based on the purpose rather than display, e.g. isBeing Edited. And I have often observed that I/we tend to jump to the first acceptable solution without trying to consider other, (radically) different and perhaps better alternatives. That is easily explained with our inborn intelectual laziness and we certainly can agree that we should not overthink things and that we need to ship but still, shouldn’t we try to think a little more?

The Clojure community has been very inspiring for me in this regard. There is a strong focus on spending more time on the problem than the solution to really understand it, and on separating the different concerns involved and adressing them separately, as well as on achieving simplicity. One of the manifestation is the strong preference of small, focused, composable libraries over frameworks. F.ex. it took couple of years for Clojure to get support for named arguments – but the result – destructuring – is something much more powerful, that now pervades the whole languages (of course, this is a language, not an app). When you listen to Rich Hickey talking f.ex. about core.async (vs. actors, Reactive Extensions etc.) you see that the man thought deeply about the problem, alternatives, and their pros and cons.

May be we should spend little more time with our problems before jumping to solutions, no matter how much we like to solve things. Perhaps we would end up with a much better code than we typically have, and thus considerably lower maintenance costs.

Ref: Simple Made Easy, The Clojure Philosophy from Joy of Clojure.

Posted in General | Tagged: , , | Comments Off

How I Learned to Avoid Magical Dependency Injection And Love Plain Java

Posted by Jakub Holý on November 27, 2013

A short story about the complexity of magical frameworks and dependency injection with a happy ending, featuring Resteasy, CDI, and JBoss.

Once upon time, I have created a JAX-RS webservice that needed to supply data to a user’s session. I wanted to be fancy and thus created a @Singleton class for the exchange of information between the two (since only a user request serving code can legally access her session, a global data exchange is needed). However sharing the singleton between the REST service and JSF handler wasn’t so easy:

  • Originally, the singleton was generic: OneTimeMailbox<T> – but this is not supported by CDI so I had to create a derived class (annotated with @Named @Singleton)
  • While everything worked in my Arquillian test, at runtime I got NullPointerException because the @Inject-ed mailbox was null in the service, for reasons unclear. According to the internets, CDI and JAX-RS do not blend well unless you use ugly tricks such as annotating your service with @RequestScoped (didn’t help me) or use JBoss’ resteasy-cdi module.

Finally I got fed up by all the complexity standing in my way and reverted to plain old Java singleton (OneTimeMailbox.getInstance()) while making testing possible with multiple instances by having a setter an alternative constructor taking the mailbox on each class using it (the service and JSF bean) (using a constructor might be even better).

Result? Actually better testability and simpler code.

Bjørn Borud and Johannes Brodwall were right – plain old Java is better than magical frameworks and magical DI is evil. (Though they would diapprove of JBoss and likely prefered if I used a plain servlet instead of JAX-RS for my very simple case.)

Update: As pointed out by Daniel Kolman now and others previously, dependency injection itself isn’t bad (though some would argue), it is only magic DI that is a problem. You can well do DI yourself using plain old Java – see Bakksjø: The inverse of IoC is Control, Perry: Do-It-Yourself Dependency Injection (pdf; quote: “[..] shows how dependency injection can be accomplished without any framework. The same benefits provided by frameworks can be realized using “do-it-yourself” (DIY) handcrafted code.“; recommended by Google’s test master Miško Hevery who is a fan of DI because it helps with testability).

Posted in j2ee, Testing, Uncategorized | Tagged: , , , , | 6 Comments »

Most interesting links of October ’13

Posted by Jakub Holý on October 31, 2013

Recommended Readings

  • Google engineers insist 20% time is not dead—it’s just turned into 120% time – it is interesting to see how has this evolved; “I have done many engineering/coding 20% projects and other non-engineering projects, with probably 20-40% producing “real” results (which over 7 years I think has been more than worth it for the company). But these projects are generally not rewarded.” [highlight mine]
  • The Worst Daily Scrum Ever – a story whose bad part is a too common reality; if energy is low, nobody asks for / offers help, and people only report status / plans then you are doing the daily scrum wrong and should stop now (but it also documents a nice example of a good, effective scrum)
  • Why Responsive Design is a Waste of Time – a refreshingly critical take on responsive design; the author now aknowledges that it is sometimes worth the pain but the points are still valid – responsive design requires (lot of) extra work, the attempt to create a one-size-fits-all site of course adds considerable complexity (having two separate simple frontends might be better than one that is too complex), also many sites are good enough as they are (especially taking into account the capabilities of mobile browsers)
  • How to lose $172,222 a second for 45 minutes – i.e. your bugs are likely not so serious after all :-) A financial company screwed big and ended up bankrupt. The cause? Chaotic DevOps, not removing old unused code, reusing a feature flag instead of creating a new one, lack of monitoring. The story in short: They deployed new trading code but failed to notice that it has not been deployed to one of the 8 servers; due to the flag reuse, the old, 10 years unused code has been activated instead. Due to the lack of monitoring they did not notice the cause, tried to roll back while leaving the flag enabled thus effectively activating the bad code on all the servers. => have proper automated and self-checking deployments, delete old code, do not repurpose old switches.
  • 40 Inappropriate Actions to Take Against an Unlocked (Windows) PC – good tips for promoting security and having fun at the same time; I shall keep this at hand :-)
  • How to go about ‘proving’ why dynamically typed languages are better – a cultivated and interesting discussion; as argueed, thinking in this direction is itself wrong and in different contexts, different languages will be more appropriate. I also like Phil Lord’s “Programming is a highly fashion-centric occupation for any number of reasons.” and “For me, the main advantage is that you are not forced to build complex hierarchies just to support the type system ([..]), and that having only a few abstractions makes it worthwhile adding lots of functions operating over them.” and L. Petit’s “IMHO, the question is irrelevant. It implicitly assumes that statically typed vs dynamically typed is a black / white choice, and that either ‘static wins over dynamic’ or ‘dynamic wins over static’ will be a true statement whatever the context.” Also a good observation that types are only a subset of function contract enforcement and one of possible implementations.
  • The Failure of Governmental IT (Learnings From HealthCare.gov) – links to a few really good articles about the problems with governmental IT in general and my summary of them
  • Inside the Arctic Circle, Where Your Facebook Data Lives – the designs of data centers used to be proprietary secrets until Fb developed its own and open-sourced them, enabling many Asian manufactures to start creating cheaper datacenters and thus started a revolution in this domain. Facebook’s data centers are not general purpose but suitable ot the kind of work they need, but it is still widely applicable. Cool to see how they use natural conditions to get energy needs down and make HW that fits best their needs – that is what I call innovation!
  • Academia.edu (via @RiczWest) – a rich source of free research papers – just register as an independant researcher; also lean/agile/systems thinking and other interesting topics
  • Writing Code? Know Your Boundaries – an inspiring way of thinking; we use many technologies in combination (HTML, CSS, JS, SQL, server-side language, …) and “the risk for picking the wrong tool for the job is strongest near the boundaries“; a discussion of the aforementioned boundaries with examples follows, e.g.: “Avoid putting HTML in JavaScript strings for ‘poor man’s templating‘”, messing up SQL with html (“SELECT '<strong>' + Username + '</strong>' FROM Users“), CSS+HTML: using inline styles, SQL+server-side: string concatenation to create dynamic SQL queries, “writing dynamic JavaScript in a string on the server“. I shall keep this in mind!
  • Johannes Brodwall: A canonical web test – a simple web app end-to-end smoke test – using an embedded Jetty, a test DB (preferably in-memory), WebDriver to test it (simple: browser.get(“/people”), assertThat(browser.findElement(<person id>.contains(<person’s name>)); simple, nice, useful

Learning

  • LearnGitBranching – an online game to learn branching & rebase in git; use the menu in the lower-right corner to navigate between the levels etc. You can also execute commands “show goal”, “hint”, “level” to navigate around; pretty cool and great for learning the command line commands

Society & people

Not a typical topic I share here but really worth it this time.

  • The ocean is broken – a saddening story worth reading to learn what does your tuna sandwitch cost and where does all the plastic we use end up. From a sailing trip from Melbourne to US where there were plenty of fish (and birds) 10 years ago – and 2 this year, killed to a noticable degree by huge fishing ships that catch tuna – and kill and throw away all the other “junk” fish. Nowadays fish are replaced by plastic and other waste that actually prevents usage of the engine unless somebody can watch for dangerous nets and ropes leftovers. Earth, where are you falling to?
  • The Guardian: Why have young people in Japan stopped having sex? – sad and interesting to observe what happens when the system is set up so that people “can’t be bothered” to have inter-sexual relationships, partnership, and children. Japan needs a good deal of systems thinking to fix its broken society where women do not want children because it would cost them their career and neither men nor women are willing to subjects themselves to the social pressure and demands associated with relationships.
  • The Guardian: 29 million people enslaved, says first global index on slavery – welcome to the 21st century! The leading slave countries are India (14M), China (3M), Pakistan (2M). Also, slaves are building the world cup stadion in Qatar.
  • They’re Taking Over! – how we managed to destroy sea ecosystems and helped the now unstoppable return of jellyfish – Jellyfish are evidently very veried and extemely resilient and have been hold at bay only by rather complex ecosystems that we managed to destabilize so much that Jellyfish are on their way back to ruling all the sees again (destroying the rests of the ecosystems – i.e. fish – on the way); a sad future for the sea, Earht, and us

Clojure Corner

Tools/Libs

Mac:

  • WhiteHat Aviator – A Safer Web Browser – WhiteHat, a well-known security company, has released a browser that aims at improving privacy by preventing user tracking (f.ex. but not sending referral URL) and blocking ads even at the cost of occassional slight discomfort, i.e. something that the mainstream browsers are not interested in. So far for OS X only.
  • EnvPane – a preference pane for environment variables for Mac OS X 10.8 (Mountain Lion) – set env. vars for GUI/terminal apps, no need to log out upon change

Favorite Quotes

Weinberg: Bureaucracy is what we do when we no longer remember why we are doing it
- via Ben Simo, no source specified so it may be fake but anyway it is valid

Posted in General, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , , | 4 Comments »

Most interesting links of September ’13

Posted by Jakub Holý on September 30, 2013

Recommended Readings

  • Stuff The Internet Says On Scalability For September 13, 2013 – a collection of interesting performance related articles with summaries (via @_dagi)
  • Can you copy a culture? The NUMMI story (audio/transcript) – how the GM factory with the worst workforce has been turned around via a good application of Toyota Production System – “a truly inspiring story of human potential and how systems can be designed to bring the best or worst of of people.” And how GM failed to learn from it and to copy Toyota’s culture.
  • The Reactive Manifesto – why to write reactive SW – “Reactive applications represent a balanced approach to addressing a wide range of contemporary challenges in software development. Building on an event-driven, message-based foundation, they provide the tools needed to ensure scalability and resilience. On top of this they support rich, responsive user interactions. We expect that a rapidly increasing number of systems will follow this blueprint in the years ahead.
  • Frameworkless JavaScript – Why Angular, Ember, or Backbone don’t work for us [Moot discussion platform] (via JavaScriptWeekly) Me: Frameworks are not always evil, but are likely overused and there are good cases when rolling your own solution is the best way. Why in Moot? Because the want a minimal API (no framework methods), small code size, small and familiar code base, no dependency hell and external package updates, no lock-in to technology that will be gone in few years, need WebSockets not REST. “Moot uses native pushState for managing URLs, John Resig’s “micro templating” for views, and internal communication between model and views happens with a custom event library. There is no router or automatic data-binding.” The looked at Angular, Ember, Backbone. “As a result of our combined perfectionism and minimalism, Moot is an extremely lightweight, manageable, and independent web application [..]
  • NYT: Eiji Toyoda, Promoter of the Toyota Way and Engineer of Its Growth, Dies at 100 – learn about the life of one of the founders of lean thinking
  • Gojko Adzic: How we solved our #1 product management problem – valuable experience of false assumptions, learning from users, and a much helpful UI remake: even if you build a product to scratch your itch, you have to test it with real users

Big data

  • Don’t use Hadoop – your data isn’t that big – a great post about the downside of Hadoop and that there are much better options (large disks, large RAM, Pandas/R/Postgres) for data up to few TBs. “In addition to being more difficult to code for, Hadoop will also nearly always be slower than the simpler alternatives.”
  • Gartner On Big Data: Everyone’s Doing It, No One Knows Why – golf talk / hype -driven initiatives FTW! “According to a recent Gartner report, 64% of enterprises surveyed indicate that they’re deploying or planning Big Data projects. Yet even more acknowledge that they still don’t know what to do with Big Data.”
  • What makes Spark exciting – why it might be a good replacement for Hive/Hadoop, based on experiences with H/H: “Hive has served us well for quite a while now. [...]  That said, it has gotten to the point where Hive is more frequently invoked in negative contexts (“damn it, Hive”) than positive. (Primarily due to being hard to test, hard to extend.)” “We had heard about Spark, but did not start trying it until being so impressed by the Spark presentation at AWS re:Invent [..] that we wanted to learn more. [..] Spark, either intentionally or serendipitously, addresses both of Hive’s primary shortcomings, and turns them into huge strengths. (Easy to test, extend.) [..] I find the codebase small and very easy to read, [..] –which is a nice consolation compared to the daunting codebases of Hadoop and Hive.” Cons.: Spark is only pre-1.0, the author hasn’t yet tested it heavily.
  • 10 Ways to Make Your Office Fun To Work In – because we spend there plenty of our time so why not have a pleasant/cosy, inspiring environment? Some tips: plants, not-your-boring-enteprprise-look-and-feel, open it to the nature (I want this!), design it as home, not office, provide play space (I am too into work to want to play but having a resting place for a nap is st. I’d love).

Books

Other

  • Stanford engineers build computer using carbon nanotube technology (via @RiczWest)
  • NYT: The Banality of Systemic Evil – a good article about human tendency to “obey the system” thus potentially causing evil – and thus the need to resist the system, as heroic individuals such as Snowden, Hammond, Schwartz, Manning. See the famous Eichmann in Jerusalem for how “doing your job” can create evil – “[..] what happens when people play their “proper” roles within a system, following prescribed conduct with respect to that system, while remaining blind to the moral consequences of what the system was doing — or at least compartmentalizing and ignoring those consequences.” (Tip: The book Moral Mazes explores the ethics of decision making within several corporate bureaucracies => mid-managers rules of life: (1) never go around your boss, (2) tell the boss what she wants to hear, (3) drop what she wants dropped, (4) anticipate what the boss wants so that she doesn’t need to act as a boss to get it, (5) do not report something the boss does not want reported, cover it up; the the job & keep your mouth shut.) “The bureaucracy was telling him [Snowden] to shut up and move on (in accord with the five rules in “Moral Mazes”), but Snowden felt that doing so was morally wrong.” “[..] there can be no expectation that the system will act morally of its own accord. Systems are optimized for their own survival and preventing the system from doing evil may well require breaking with organizational niceties, protocols or laws.
  • Fairphone – “A seriously cool smartphone that puts social values first” (likely the only one not built by poorly paid workers and creating too much ecological burden), for just €325. You can see detailed cost breakdown, list of suppliers, specs, and essentially everything. This is, in my opinion, super cool! Go and read the story!

Clojure Corner

  • Amazonica – “A comprehensive Clojure client for the entire Amazon AWS api.”
  • Talk Ritz, The Missing Clojure Tooling (40min, 9/2013) – thanks to this I finally understood how to use Ritz but it still seems not to work well, f.ex. setting a breakpoint always reported “Set 0 breakpoints” (lein ritz/middleware 0.7.0, nrepl-ritz.el 0.7.1); according to callen, debug-repl is simpler and nicer if you only care about local vars and evaluation. To try ritz: use M-x nrepl-ritz-jack-in, then M-x nrepl-ritz-break-on-exception, exec. f.ex. “(/ 1 0)”. In the poped-up buffer, t or enter to show frame locals, e to eval a code in the context of the frame etc. If you managed to trigger the debug buffer through a breakpoint, the actions lists would contain STEP etc. (See fun. nrepl-ritz-line-breakpoint)
  • C. Grand’s spreadmap – “library to turn Excel spreadsheets in persistent reactive associative structures” => access content via map functions; changing a value updates formula cells using it
  • Alembic Reloads your Leiningen project.clj Dependencies – add a dependency to your project.clj w/o needing to restart your REPL (just call (alembic.still/load-project), provided you have it in your lein dependencies). Limitations: cannot remove deps or change versions.
  • Defeating stack overflows – techniques for transforming mutually recursive calls etc. into something that won’t blow the stack – “Priming the pump” (memoize subresults first), core.async
  • Google Groups: Clean Architecture for Functional Programming – How do the Clean Architecture and the Clean Code best practices apply to FP (Clojure/Haskell)? Some points: OOP isn’t worse than FP, only people do class-oriented programming instead; OO better e.g. for UIs, combining them (func. core, imperative shell) can be sometimes best. Some clean arch. patterns are actually more like functions – “Interactors and Presenters, for example, do not maintain any state of their own.  Even those objects that do imply some kind of state, such as entities and gateways, keep that state hidden behind boundaries and present a functional interface instead.
  • night-vision: Handy, super light weight debugging utility – add it to your lein profile and then call (night-vision.goggles/introspect-ns! '<name of ns>) and it will print each entry/exit of a function within the scope of the namespace with the argument/return values
  • Nil Punning (Or Null Pointers Considered Not So Bad) – a great post about why nil in Clojure is not bad contrary to Java’s null (because it is actually an object, you can call functions on it, treat it as false/empty list/map/set, most core functions work on it)

Tools/Libs

Posted in General, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , , , | Comments Off

Lesson Learned: Don’t Use Low-Level Lib To Test High-Level Code

Posted by Jakub Holý on May 21, 2013

Summary: Using a fake http library to test logic two levels above HTTP is unnecessarily complex and hard to understand. Fake instead the layer directly below the logic you want to test and verify the low-level HTTP interaction separately. In general: Create thin horizontal slices for unit testing, checking each slice separately with nicely focused and clear unit tests. Then create a coarse-grained vertical (integration-like) test to test across the slices.

The case: I want to test that the method login sends the right parameters and transforms the result as expected. Login invokes post-raw which calls an HTTP method. Originally I have tried to test it by using the library clj-http-fake but it proved to be unnecessarily complex. It would be much better to fake post-raw itself for testing login and test the original post-raw and its HTTP interaction separately, using that library.

Read the rest of this entry »

Posted in Testing | Tagged: , , | Comments Off

Most interesting links of Mars ’13

Posted by Jakub Holý on March 31, 2013

Recommended Readings

A lot of stuff this month since I have finally got time to review some older articles. Quite a few articles by Fowler. Few really great (yet short) talks on agile & SW development.

Top

  • Agile in a Nutshell (originally Agile Product Ownership in a Nutshell) by Henrik Kniberg – the best explanation of the agile development process ever, in just 15 minutes and with wonderful animation; every developer should see this. Some highlights: the most important task of product owner is to say NO so that backlog doesn’t grow infinitely; at start, the estimates of size and value will suck and that’s OK because the value is in the conversation, not in the numbers (that are anyway just relative); the goal is to maximize outcome (value), not output (# features). Compromises between short-term vs. long-term goals, knowledge vs. customer value building etc. Build the right thing (PO) x build it right (devs) x build it fast (SM). Technical debt x sustainable pace. As I said – you MUST see it.
  • Martin Fowler: The Value of Software Design (talk, 22 min, from 0:45:00 til 1:07; Feb 2013) – a balanced argument for the value of good software design and internal code quality based on paying off by enabling us to keep our development speed. Discusses the DesignStaminaHypothesis (bad design => rapid decline of development speed), TechnicalDebt, TechnicalDebtQuadrant (Prudent x Reckless, Deliberate x Inadvertent), TradableQualityHypothesis. According to the experience of Fowler and others, the good design payoff point “it’s weeks, not months.”
  • What Does It Take To Become A Grandmaster Developer? – great post about cognition and learning, valuable references, quotes from an interesting study of good vs. mediocre developers. We have mental capacity for ~7 chunks of information => great performers recognize patterns and see and understand thus higher-level chunks and have many “chunks” (patterns encountered previously) readily available. You need deliberate effort to learn more chunks – especially initially but you must always try to get out of your comfort zone to grow. Experienced collegues can help a lot in acending the learning curve.

Agile, organization, innovation, project management

  • How to Prioritize a User Story Map – we all know that we should prioritize features by their value, risk, and lack of knowledge and that we should slice the features thin so that they fit into short iteration and can be deployed soon to produce feedback, right? Here we see a nice example of what happens if not done so and how to do feature slicing better.
  • Bob Marshall: Rightshifting – according to the author, 80% of knowledge work organizations are very ineffective, wasting resources on non-value-adding activites; only few are effective, even fewer highly effective. Rightshifting is the attempt at shiting them to the right, towards higher effectiveness. Links to a few videos explaining it more. Related: Steve McConnell’s Business Case for Better Software Practices, referring to a study by SEI; “The actual distribution of software effectiveness is asymmetric. Most organizations perform much closer to the worst practice than to the best.” – the best performing 10 times better then the worst/average (productivity, speed, defects, value)
  • On Antifragility in Systems and Organizational Architecture – introduces the concept of antifragility, based on Nassim Taleb’s book Antifragile that compares fragile, robust, and antifragile systems and organizational structures (which is also applicable to SW systems); robust = resists change (unless too large); antifragile: learn, adapt; closely related to DevOps and continous delivery
  • M. Fowler: PurposeOfEstimation – many Agilist disdain estimation, this is a balanced view: “estimation is valuable when it helps you make a significant decision.” (F.ex. when deciding what we (don’t) have resources for or when in need of coordinating related activities.) It is evil when used as commitments that people are forced to stick to and blamed for not managing to do so. “Above all be wary of anyone who tells you they [estimates] are always needed, or never needed.” A. Ferguson: “[..] it is poor project management (whether by project managers or other team members) that results in a client who thinks estimates are fixed, or that raw estimates = actual effort/duration”.
  • Ron Jeffries: Estimation is Evil – discusses the problems estimates can cause, issues with requirements gathering up front and their volatility, transparency and politics. Very valuable, highly recommended. See the “favorite quotes” at the bottom of this post. Also contains an interesting lesson learnt from the failed Chrysler C3 project: don’t try to build a grand new system to replace and fix the old one, fix one problem at a time – worth reading for this alone.
  • Interview with Steve Blank: Why Big Companies Can’t Innovate – the 2013 list of the world’s 50 most innovative companies has only a few large, established firms (those that have built innovation into its DNA such as Apple and Google). Established companies are less innovative because they focus in their existing business model, have risk-aversion (while there are many failures on the way to a new business model); finally “the people who are best suited to search for new business models and conduct iterative experiments usually are not the same managers who succeed at running existing business units.” – and thus aren’t given the chance. “[..]  the process of starting a new business [..] is fundamentally different from running an existing one. So if you want your company to grow organically, then you need to organize your efforts around these differences.”

Architecture & Ops

  • M. Fowler: Schemalessness + NoSQL and Consistency (20 + 20 min) two short, very good, balanced talks about NoSQL. He explains schemalessness and consistency and points out common misunderstanding about them so if you are into NoSQL, watch it.
  • What Powers Instagram: Hundreds of Instances, Dozens of Technologies (2012) – interesting high-level overview of the Instagram infrastructure based on AWS and Python (25 XL instances running Django/Gunicorn behind ELB with 3 Nginxes, sharded PostgreSQL with streaming replication on 12 QXL mem instances with software raid and XFS to freeze when snapshoting, media in S3, Redis, Solr for geo-search, Memcached. Gearman for task queues, pyapns for notifications. Munin for monitoring.)
  • The Netflix API Optimization Story – how Netflix redesigned its APIs to improve performance, reduce chattiness, and power product development and experimentation. The common REST API has become a development bottleneck and a lowest common denominator solution (w.r.t. supporting various clients). The main changes were: usage Hystrix for fault tolerance, each device team managing their own end-points in any JVM languges (primarily Groovy) and re-using common APIs (i.e. pushing some device-specific code to the server) => able to experiemnt more quickly, using the Functional Reactive Programming Model and asynchronous APIs (to abstract away thread-safety and parallel execution implementation details from the device teams so that code can execute sync. or async. without them needing to know).
  • Me: Overview of current monitoring libs for Java – Netflix’ Servo, Yammer’s Metrics, JavaMelody, JavaSimon.
  • Debug Servlets, or ‘HTTP Won; Use It’ – expose all debugging info of your services over HTTP – it makes debugging much simpler. We do a part of it and it really helps. Expose config (values, where they come from), logs, log configuration, JMX (setting it up otherwise not trivial), version, build number, git hash, server time (timezones tricky), metrics, stack dumps, app-specific status (Hadoop: live nodes, data size etc.). The author recommends JavaMelody to collect & visualize many common metrics. Not on security: Make sure to hide passwords and make the endpoints visible only internally. (Tip: consider Jolokia for exposing JMX over HTTP, see below.)
  • JVM Crash/Core Dump Analysis – 3 common categories of  JVM crash causes (JVM/JIT/JNI) and how to recognize and troubleshoot them

Other

  • How to lose wight in the browser:  The definitive front-end performance guide – a site by a number of experts from Twitter, Opera, Google, and other places with best practices for performant web sites (HTML, CSS, JS, jQuery, images). Ex.: styles up top, scripts down bottom; minify your html, css and JS; async script loading; combine css/JS files into one; cache array lengths while looping; use css sprites for icons.
  • Luke Stevens: The harsh truth about HTML5′s structural semantics (part 1) – “HTML’s structural elements — article, section, nav and aside — are, at first glance, some of the easiest parts of the HTML5 specification to understand and implement. However, they’re actually some of the most poorly specified, poorly understood, and poorly implemented parts of HTML5.” Interesting: The “research” leading to their establishment was quite random, ignoring a crucial source of information (css IDs).
  • Marco Emmanuel Patiño: Six non-technical books every programmer should read – 1. Team Geek: A Software Developer’s Guide to Working Well with Others (-> effective communication and collaboration), 2. The Pragmatic Programmer: From Journeyman to Master, 3. The Passionate Programmer: Creating a Remarkable Career in Software Development, 4. Clean Code: A Handbook of Agile Software Craftsmanship, 5. 97 Things Every Programmer Should Know: Collective Wisdom from the Experts (online), 6. Code Simplicity: The Fundamentals of Software.
    • Related: Top 5 Java programming books – Best of lot (actually 8) – 1) Head First Java, 2) Effective Java, 3) Thinking in Java, 4) Head First Design Pattern, 5) Concurrency Practice in Java, 6)Java performance, 7) Java Puzzlers, 8) Head First Object Oriented Analysis and Design.
  • Humans as slaves of chemistry: America’s Real Criminal Element – Lead – a fascinating article about how whole nations can be seriously influenced by a single chemical substance. Aside of that it is also fascinating to observe how we tend to search for causes in our domain of expertise (police, sociologists, …) and of interest while denying other possible causes, no matter how strong are the proofs. If the facts presented are true, then the fivefold increase in serious crimes in (not only) America since 60s has been caused by the increase of lead in the environment (pushing many people over the edge of ocassional violent loss of control). How many social problems in the world have similar industrial causes? Are we careful enough with what we let into our air and bodies?

Languages

  • newcoder.io: Learning more Python via projects – an excellent next step when you have learned Python syntax via LPHW or similar; in this tutorial series you will be building real-world apps while learning more of Python. You will play with, Data Visualization, APIs, Web Scraping, Networks, GUI.
  • Brian McCallister: Go is PHP for the Backend – a very good explanation why you might want to use Go and that you have to first learn “the Go way” to avoid insanity, since it is very opinionated and different from what you might be used to. Some pros: “native code, UNIX friendly, higher level then C, lower level then Python or Ruby, garbage collected, strongly typed, good performance, good concurrency support, etc.”
  • The Neophyte’s Guide to Scala 1 to 15 (list) – a good follow-up on the Cursera FP in Scala course, a series of blog posts exploring some topics more in depth. F.ex.: extracotrs (unapply, for pattern-matching), the broad applicability of pattern matching, pattern matching anonymous functions & partial functions #4, usiong Option idiomaticly #5, nice FP error handling with the Try type #6, Futures, etc. Higly recommended! Thx to Jakob Lind

Libs

  • Jolokia is remote JMX with JSON over HTTP: a REST API bridged to JMX, with support for security, fine-grained access control, bulk operations. Especially useful if you either 1)  need to perform bulk operations (e.g. get multiple values) or 2) want to access them from something that doesn’t support JMX. JSON is in general very easy to use and navigate. You can install Jolokia as a WAR (or mebedd its Servlet), a JVM agent, or attach it on-the-fly to a running JVM.
  • The Appeal and Controversy of ZeroMQ – why to use 0MQ? It is a messaging library that focuses on performance, decentralization and simplicity, solving some really hard problems (sending async. messages w/o locks, distribuing to specific subscribers) and providing a simple API. Main pros: decentralized (no central broker), many languages; cons: no security (but you can use it over SSH).

Talks

  • Tim O’Reilly: Create More Value Than You Capture (30 min + questions) – build apps that matter, that change how we do things. Thinking just about money is bad. Try to make the society better, smart, create more value than you capture, solve important problems, help people. Ex. startups: Uber, Square, Code for America.
  • TED: Bruce Feiler: Agile programming — for your family (20 min) – an inspirational talk, based on positive experience from multiple families, about applying the agile thinking and values to make our families happier by empowering the children (enlist them in their upbringing, deciding on goals, rewards, punishments), letting them know who they are, being adaptive, having regular “retrospectives” (that eventually become cherrished memories). Backed by research. Did you know that the #1 wish of children isn’t that parents spend more time with them but that they are less stressed?

Clojure Corner

  • What’s new in Clojure 1.5.x – reducers, new threading macros (cond->, as->, some->, ..), various improvements, improved performance, erro messages, doc strings, bug fixes
  • Stuart Sierra: On the Perils of Dynamic Scope – summary: don’t create macros like with-connection binding to a thread-local var; make all methods take the resource as a parameter – thus the user has the freedom to decide when to close the resource and isn’t limited to a single thread and can use lazy sequences
  • Logic programming is overrated – core.logic is essentially only a complex DSL for doing an exhaustive search and there is already a nice, clean tool for that: the for comprehension. A logic puzzle can be much more clearly and also efficiently using for. But it is not completely useless – logic programming is good e.g. for running programs backwards, unification is important for writing type checkers, and the new constraint programming piece has good potential. Read also Logic Programming is Underrated, which provides a faster core.logic solution than for-comprehension and provides some pointers rgarding the practical usefulness of core.logic.
  • Prismatic – Graph: Abstractions for Structured Computation – How to reduce the complexity overhead in large, real-world, FP systems by decoupling what is done from how it is executed. Graph is a Clojure library enabling a declarative way to describe how data flows between (mostly pure) functions => “It allows us to formalize the informal structure of good FP code, and enables higher-order abstractions over these structures that can help stamp out many persistent forms of complexity overhead.” By decoupling the description of how data flows and the actual execution, we can execute it in different ways (parallelized, with memoization, lazy/eager) and apply various interceptors (for logging etc.). See especially the part “Graph and complexity overhead.”
  • Mike Anderson: Game development in Clojure : Alchemy 7DRL post-mortem (and the previous 7 daily updates, Alchemy @ GitHub) – an interesting report about game making in Clojure during 7 days, in as functional and immutable style as possible while keeping it sufficiently fast. How do you represent & handle statuf game objects, the world map, game state? The design of the game, what was easy and what hard with Clojure. Tl;dr: search it for “Some parting thoughts” (Clojure productive, immutability hard but pays off, prototype objects great, more typing would have helped). “Making everything immutable in Clojure is harder than it would have been in an OOP language like Java where everything can be encapsulated in mutable classes. In particular, the state update functions are tricky to make both correct and performant. The payoff is big however: in terms of the simplicity and effectiveness later on, and in the conceptual clarity being able to treat the entire game state as an immutable value”. Having REPL is a big win.
  • Refactoring Java using Clojure with the Eclipse Java development tools (JDT) (operation on AST nodes, i.e. little too low level; the Eclipse Refactoring API might be better)
  • Clojure at a Bank – [Our] Clojure Code Immaturity – experiences with going from Java to Clojure: 1) too few comments, too short names => hard to learn the code; 2) not knowing clojure.core well enough => reimplementing (if-let, juxt, …); 3) structure, comment, split up your namespaces well, navigating more complicated then in Java IDEs; 4) reasonably used Macros, Protocols, Defrecords payed off;
  • Datomic for Five Year Olds – explaining the key characteristics of Datomic compared to relational and NoSQL DBs (schema, architecture, programmability/language); doesn’t go into details of how it works (e.g. how does Datomic determine what subset of the DB to cache locally and what if it is few GBs); Honey Badger’s 2012 talk Exploring Datomic: a database deconstructed explores the architecture and technical details much more

Tools

  • Vagrant 1.1.0 is out (what’s new?), with support for VMWare Fusion and AWS VM backends in addition to VirtualBox – use the same config to create, provision, stop, destroy and connect to a virtual machine locally or in the cloud (with limited support for shared folders, I’d suppose). V. 1.1 is backwards compatible aside of plugins, upgrade to new config optional.
  • Animated presentations: ArtRage (drawing program, also for iPad), Wacom Intuos 5 (drawing tablet), Screenflow (screen & audio capture) – used for the Agile in a Nutshell (Agile Product Ownership in a Nutshell) mentioned above
  • ckjm — Chidamber and Kemerer Java Metrics (via Neal Ford) – a command-line tool (also Maven/Ant plugin) to compute some metrics, outputting text or XML for further processing; the metrics: WMC: Weighted methods per class (cyclomatic complexity of its methods), DIT: Depth of Inheritance Tree, NOC: Number of Children, CBO: Coupling between object classes, RFC: Response for a Class, LCOM: Lack of cohesion in methods, NPM: Number of Public Methods, Ca: afferent coupling.
  • Bulletproof Demos: Record & Replay built into Chrome – ever got a failure while demonstrating a web app though it has worked moments ago? No more! You Chrome to record your requests and responses and let its cache handle them during the real demonstration. (Mac: stop Chrome, to record run open -a “Google Chrome” –args –record-mode, to replay run open -a “Google Chrome” –args –playback-mode. Linux: google-chrome –record-mode and –playback-mode. Win.: run chrome <arg>)
  • UserTesting.com (via Ash Maurya, the author of Running Lean) – on-demand usability testing; they have a large base of test users, can select those matching your criteria and unleash them upon your site guided by a script your provide, watch videos of their actions while they verbalize their thinking process, recieve written answers from them, talk to them.
  • MindMup.comopensource, free mind-mapping in the cloud by Gojko Adzic & co., with main focus on productivity. Store private maps in Goolge Drive, support for mobile devices, keyboard shortcuts. No registration needed.

Favorite Quotes

Once we estimated a project to require 9 man-months but were later told that we do not understand a thing and it may not take more then 4. At the end it took over 25 and still wasn’t done.

- paraphrasing my collegue Kim Leskovski

On collecting requirements up front:

At the very beginning, we know less about our project than we’ll ever know again. This is the worst possible moment to be making firm decisions about what we “require.”

- Ron Jeffries in Estimation is Evil: Overcoming the Estimation Obsession

On the estimate of project delivery date at its initial phase:

It’s based on an unrealistic list of requirements, using weak estimates, made at the moment of maximum ignorance, by people who are always optimistic about their own abilities.
- ibid

On planning and requirements (the Chrysler’s C3 payroll project, having a payroll expert and a team familiar with the domain):

This was one of the best-planned projects I’ve ever seen, and even so, at least one third of the requirements were added, removed, or substantially changed.
- ibid

[..] a line of code is a liability, not an asset [..]
- Jez Humble in Why Software Development Methodologies Suck

Agile is not something you do, it is something you are.
- Huib Schoots in Creating my own flow
with personal kanban, Agile Record Feb 2013

Posted in General, SW development, Tools, Top links of month | Tagged: , , , , , , , , , , , , , , , , | Comments Off

Most interesting links of February ’13

Posted by Jakub Holý on February 28, 2013

Recommended Readings

  • Øyvind Bakksjø: The inverse of IoC is Control – a well-founded criticism of the (over)use of inversion-of-control containers such as Spring and Guice. Bakksjø isn’t against dependency injection in general but he prefers to do it manually, instead of magically, in a main method, using Java instead of an obscure DSL/XML. The dependencies are typically known at compile time so why postpone assembling (and defect discovery) to runtime? Why hide how parts fit together into the non-transparent magic of IoC? He has many good points though I believe there are cases when some automation of the wiring process is valuable (think of scope-dependant beans, diff. deps in diff. environments etc.) Comment by B. Borud: “Explicit wiring you can read from start to finish beats vague declarative shit that you may be able to figure out if you pay close attention.” Check out also the comments at HN.
  • J. B. Rainsberger: Demystifying the Dependency Inversion Principle – a very good explanation of the value of dependency injection in the terms of general good code principles (promoting abstractions and postponing determination of details to the latest moment/client)
  • CodingHorror: Coding Without Comments (2008) – a very good explanation if how to treat comments (similar what Clean Code says): write code so that it is self-explanatory (using good method and variable names), if necessary, add comments that explain why (while the code shows what/how).
  • Frustrations and Aspirations of a Software Craftsman – what makes us happy or unhappy on a project? (I could add some too.) Bad: bureaucracy, old/unfit technology, lack of autonomy and credibility, uninteresting domain, demotivated people, blaming and highly competitive environment, arrogant and unskilled people, … => it is mostly about people. Good: projects where there is passion, craftsmanship, friendship and trust.
  • Jetty-9 goes fast with Mechanical Sympathy – interesting how the run-time behavior might differ from what we would expect and how knowing the hardware can improve performance. Here:  false sharing of a blocking queue’s head/tail pointers and locks (close => same CPU cache row => updating one invalidates the other), using trie backed directly by IO buffers for faster String lookups etc. Result (all microbenchmark disclaimers): jetty-9 out performs jetty-8 by 30% faster and 50% less YG garbage.

Czech

  • Daniel Kolman: Jak správně pojmenovat test – a good description of different approaches to name and organize tests w.r.t. the experience of the developer – starting with “testWorks1″ through one method-one test with “testMyMethodName” through a more thoughtful naming scheme such as Roy Osherove’s UnitOfWork_StateUnderTest_ExpectedBehavior (e.g. ndexOf_containsSearchedString_returnsCorrectIndex) to tests as a living documentation: tests represent a list of features of the class, each name is a sentence having the class under test as its subject (e.g. [list] holdsItemsInTheOrderTheyWereAdded). Advantages of the latter: readability, documentation, it forces us to think what actually is still the responsability of the class and what isn’t. “Stačí názvům testů věnovat dostatečnou pozornost a ony vám samy napoví, kdy už toho třída dělá moc a měli byste se zamyslet, jak kód lépe rozdělit.”

Non-IT

  • Why You Should Work With Someone You Hate – working with people we like and do not have conflicts with is nice but we are likely to have similar views and ideas and miss the broader picture. Working with somebody that drives you crazy while being able to respect each other is very valuable because it opens you to different views and forces you to really defend yours. Recommended!
  • Kent Beck: Focusing Talks: Three Questions – some good advices for performing technical talks and avoiding putting too much into the talk. Ask 1) Who is the audience? – be as specific and narrow as possible to be able to tune and cut the content; “Better to reach one person than zero.” 2) What is my one startling sentence? – one that makes the audience take notice; “Picking one sentence, I encourage focus and empathy in myself. Now I have one person’s attention. What do I say?” 3) What would I like the audience to do? “‘To what purpose?’ helps me discard interesting-but-tangential points, to, as Peter Jackson put it, move the ring.” “The more clear and uncompromising I can be with the three questions, though, the easier all the rest of it becomes.” “Talk about less. And less. Less than that.”

Clojure Corner

  • Clojure’s Reader is Unsafe (for untrusted strings) – don’t use read-string to read untrusted strings – even if you set *read-eval* to false, it can still be persuaded to execute arbitrary code (hopefully fixed in Clojure 1.5). The right, save way is to use clojure.tools.reader.edn/read-string.

Tools

Posted in General, Tools, Top links of month | Tagged: , , | Comments Off

The Sprinting Centipede Strategy: How to Improve Software Without Breaking It

Posted by Jakub Holý on January 14, 2013

Re-published from blog.iterate.no.

Our code has been broken for weeks. Compiler errors, failing tests, incorrect behavior plagued our team. Why? Because we have been struck by a Blind Frog Leap. By doing multiple concurrent changes to a key component in the hope of improving it, we have leaped far away from its ugly but stable and working state into the marshes of brokenness. Our best intentions have brought havoc upon us, something expected to be a few man-days work has paralized us for over a month until the changes were finally reverted (for the time being).

Lessons learned: Avoid Frog Leaps. Follow instead Kent Beck’s strategy of Sprinting Centipede – proceed in small, safe steps, that don’t break the code. Deploy it to production often, preferably daily, to force yourself to really small and really safe changes. Do not change multiple unrelated things at the same time. Don’t assume that you know how the code works. Don’t assume that your intended change is a simple one. Test thoroughly (and don’t trust your test suite overly). Let the computer give you feedback and hard facts about your changes – by running tests, by executing the code, by running the code in production.

Read the rest of this entry »

Posted in General, SW development | Tagged: , , , | Comments Off