The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘scala’

Most interesting links of December ’13

Posted by Jakub Holý on December 31, 2013

Recommended Readings

Society

  • HBR: Want to Build Resilience? Kill the Complexity – a highly interesting, thought provoking article relevant both to technology in particular and the society in general; f.ex.: more security features are bad for they make us behave less safely (risk compensation) and are more fragile w.r.t. unexpected events. “Complexity is a clear and present danger to both firms and the global financial system: it makes both much harder to manage, govern, audit, regulate and support effectively in times of crisis. [..] Combine complex, Robust-Yet-Fragile systems, risk-compensating human psyches, and risk-homeostatic organizational cultures, and you inevitably get catastrophes of all kinds: meltdowns, economic crises, and the like.” The solution to future financial crisis is primarily not more regulation but simplification of the system – to make it easier to police, tougher to game. We also need to decrease interconnectednes (of banks etc.), one of the primary sources of complexity. Also a great example of US Army combatting complex, high-risk situations by employing “devil’s advocates / professional skeptics” trained to help “avoid the perils of overconfidence, strategic brittleness, and groupthink. The goal is to respectfully help leaders in complex situations unearth untested assumptions, consider alternative interpretations and “think like the other”“.
  • The Dark Side of Technology – technologies provide great opportunities – but also risks we should be aware of – they create a world of mounting performance pressure for all of us (individuals, companies, states), accelerate the rate of change, increasing uncertanity (=> risk of Taleb’s black swans). “All of this mounting pressure has an understandable but very dangerous consequence. It draws out and intensifies certain cognitive biases [..]” – magnify our perception of risk, shrink our time horizons, foster a more and more reactive approach to the world, the “if you win, I will lose” view, erode our ability to trust anyone – and “combined effect of these cognitive biases increases the temptation to use these new digital infrastructures in a dysfunctional way: surveillance and control in all aspects of our economic, social and political life.” => “significantly increase[d] the likelihood of an economic, social and political backlash, driven by an unholy alliance between those who have power today and those who have achieved some modest degree of income and success.
    Complexity theory: the more connected a system is, the more vulnerable it becomes to cascades of disruptive information/action.
  • What Do Government Agencies Have Against 23andMe, Uber, and Airbnb? – innovative startups do not fit into established rules and thus bureaucrats do not know how to handle them and resort to their favourite weapon: saying no, i.e. enforcing rules that harm them (f.ex. France recently passed a law that requires Uber etc. drivers to wait 15 min before picking up a customer so that established taxi services have it easier; wot?!)
  • Nonviolent communication in action – wonderful stories about NVC being applied in difficult situations with a great success

Tech

  • D. Nolen: The Future of JavaScript MVC Frameworks – highly recommended thought food – about React.js, disadvantages of event-based UI, benefits of immutability, performance, the ClojureScript React wrapper Om  – “I simply don’t believe in event oriented MVC systems – the flame graph above says it all. […] Hopefully this gives fans of the current crop of JS MVCs and even people who believe in just using plain JavaScript and jQuery some food for thought. I’ve shown that a compile to JavaScript language that uses slower data structures ends up faster than a reasonably fast competitor for rich user interfaces. To top it off Om TodoMVC with the same bells and whistles as everyone else weighs in at ~260 lines of code
  • Quora: Michael Wolfe’s answer to Engineering Management: Why are software development task estimations regularly off by a factor of 2-3? – a wonderful story explaining to a layman why estimation is hard, on the example of a hike from SF to LA
  • Style Guide for JavaScript/Node.js by Felix Geisendörfer, recommended by a respectable web dev; nothing groudn breaking I suppose but great start for a team’s standards
  • Johannes Brodwall: Why I stopped using Spring [IoC] – worth to read criticism of Spring by a respected and experienced architect and developer; summary – dependency injection is good bug “magical” frameworks decrease understandability and encourage unnecessarily complex code => smaller code, , easier to navigate and understand and easier to test
  • Misunderstanding technical debt – a brief discussion of the various forms of tech. debt (crappy code x misaligned design and problem domain x competence debt)
  • Tension and Flaws Before Health Website Crash – surprising lack of understanding and tensions between the government and contractors on HealthCare.gov – “a huge gap between the administration’s grand hopes and the practicalities of building a website that could function on opening day” – also terribly decision making, shifting requirements (what news!), management’s lack of decision power, CGI’s blame-shifting. A nice horror story. The former head knew that they should “greatly simplify the site’s functions” – but the current head wasn’t able to “talk them out of it”.
  • The Log: What every software engineer should know about real-time data’s unifying abstraction – logs are everywhere and especially important in distributed apps – DB logs, append-only logs, transaction logs – “You can’t fully understand databases, NoSQL stores, key value stores, replication, paxos, hadoop, version control, or almost any software system without understanding logs” – I have only read a small part but it looks useful
  • What I Wish I Knew When Learning Haskell tl;dr
  • Better Than Unit Tests – a good overview of testing approaches beyond unit tests – including “Automated Contract Testing” (ability to define a contract for a web service, use it to test it and to simulate it; see Internet of Strings for more info), Property-based Testing (test generic properties using random data/calls as with Quickcheck), Fault Injection (run on multiple VMs, simulate network failures), Simulation Testing as with Simulant.
  • Use #NoEstimates to create options and deliver value reliably – a brief post with an example of an estimation-based vs. no-estimates project (i.e. more focus on delivering early, discovery)
  • How Google Sold Its Engineers on Management – managers may be useful after all :-); a report about Google’s research into management and subsequent (sometimes radical) improvements in management style/skills and people satisfaction; I love that Google hasn’t HR but “people ops”
  • Roy Osherove: Technical Disobedience – take nothing for granted, don’t let the system/process stop you, be creative about finding ways to improve your team’s productivity; there always is a way. Nice examples.
  • Uncle Bob: Extreme Programming, a Reflection – a reflection on changes in the past ~ 14 years since XP that have seen many of the “extreme” practices becoming mainstream
  • The Anti-Meeting Manifesto – essentially a checklist and tips for limitting meetings to minimum

Other

Talks

  • Pete Hunt: React: Rethinking best practices (JSConf 2013, 30 min) – one of the most interesting talks about frontend development, design, and performance I have heard this year, highly recommended. Facebook’s React JavaScript framework  is a fresh and innovative challenger in the MVC field. It is worthwile to learn why they parted ways with the popular approach of templates (spoiler: concern separation, cohesion x coupling, performance). Their approach with virtual DOM enables some cool things (run in Node, provide HTML5-like events in any browser with consistent behavior, …). Key: templates are actually tightly coupled to display logic (controllers) via the model view tailored for them (i.e. Controller must know what specific data & in what form View needs) => follow cohesion and keep them together componets, separate from other components and back-end code. Also, state changing over time at many places is hard => re-render the whole app rather than in-place updates. Also, the ClojureScript Om wrapper enables even more performance optimizations thanks to immutable data structures etc.
  • David Pollak: Some musings on Scala and Clojure by a long time Scala dude (46 min) – a subjective but balanced comparison of Scala and Clojure and their strengths/weaknesses by the author of the Scala Lift framework (doing Scala since 2006, Clojure since 2013)

Clojure Corner

Tools/Libs

  • Apache Sirona – a new monitoring tool in the Apache incubator – “a simple but extensible monitoring solution for Java applications” with support for HTTP, JDBC, JAX-RS, CDI, ehcache, with data published e.g. to Graphite or Square Cube. It is still very new.
  • GenieJS – Ctrl-Space to popup a command-prompt for your web page, inspired by Alfred (type ‘ to see all possible commands)

Favourite Quotes

A good #agile team considers their backlog inaccurate. It is merely a list of assumptions that must be tested & refined by shipping product
- @mick maguire 12/10

Ada Lovelace (1st program), Grace Hopper (1st compiler), Adele Goldberg (1st OO language), why would anyone think women aren’t in computing?
@Dan North 12/11

There will always be a shortage of talented, self-motivated creative professionals who will unquestioningly follow orders.
@Thomas K Nilsson 12/7

Estimation paradox = If something unpredictable happens, predict how long it will take to fix it
me 12/7

IT systems can be inspired by AK-47 a.k.a. Kalashnikov. The rifle was purposefully designed to be simple and to be tolerant to imperfections in most parts; as a result, it required essentially no maintenance and was extremely reliable.
– summarized from Roman Pichlík’s Odkaz Michaila Kalašnikova softwarovému vývoji

Posted in General, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , , , , | Comments Off

Most interesting links of November ’13

Posted by Jakub Holý on November 30, 2013

Recommended Readings

Some interesting topics this time despite me spending lot of time on the Principles of Reactive Programming course: Java x Node.js, REST x other future-proof architectures, scary legacy code. Of course, also plenty of Clojure.

People, organizations, teams, development:

  • Chris Argyris (1923-2013): An Appreciation – Thinkers 50 – recently departed Ch. Argyris is a person whose work you should know, if a bit interested in learning and organizations and how they (dis)function; and since we all work in organizations and want our work to be pleasant, this means all of us. We all want to work in orgs that do double-loop learning, i.e. they actually evovle as they learn. “Argyris argued that organizations depend fundamentally on people and that personal development is and can be related to work.” Now stop and go read it!
  • Bob Marshall: The Antimatter Principle – “the only principle we need for becoming wildly effective at collaborative knowledge work” – can be summarized as “attend to folks’ needs” (importantly, including your own) => find out what people actually need, interpret their behavior (including anger, seemingly irrational or stupid requests etc.) in terms of needs; mastering this will make you excell in communication and effective work. Read the post to find out more.
  • It’s a state of mind: some practical indicators on doing agile vs. being agile – are you agile or are just “doing agile”? Read on ti find out, if you dare! F.ex. “Non Agile teams have a process that slows the review of the changes.” Cocnlusion: “An Agile mindset is just that – a state of mind, a set of values. Its a constant questioning, and an opening up to possibilities. Its a predisposition to produce great things.
  • Johannes Brodwall:  Humble architects – how to avoid being an architect that does more harm than good, as so many out there? Some tips: Don’t assume stupidity, Be aware that you might be wrong, Be careful with technology (i.e. simplicity beats everything; applies so much to us developers too!), Consistency isn’t as important as you think (or beware context), Tactical reuse in across systems is suboptimization (i.e. reuse has a cost), separate between (coding) rules and dogma (i.e. is that way unsafe, incomprehensible, or just a heresy w.r.t. a dogma?) Very valuable insights into creating good technical solutions and teams that work.
  • Liz Keogh’s The Dream Team Nightmare: a Review – a very good review of this adventure-style book about coaching “agile” teams through (around?) common pitfalls, provides a good base for deciding whether you shall read the book (Liz essentially says yes)
  • Fibonacci Kittens: One Idea One Commit – a short story of coming from biannual releases to frequent release of individual features; I link to this primarily to spread optimism, if this company managed it then, perhaps, we other can too?
  • The Eternal Struggle Between Business and Programmers – “Business: More features! Now! Programmers: More refactoring! Now!” How can we resolve this eternal conflict of needs? This post reveals how the two parties can find a common ground and mutual understanding (beware, everybody must give up on something) and thus work together rather than against each other.

Coding, architecture, legacy

  • Why the future is NOT RESTful – always refreshing to read something against the mainstream; “REST is not fit for the next generation of smart client applications because it has not been designed for smart clients.” According to the author, a smart client app stack needs: “1. persistence (storage and query), 2. documents/orm (conversion to tree-like datastructures), 3. data authorization (once authenticated), 4. pub/sub (websocket communications), 5. client db (client-side caching and querying), 6. templating (presentation level)” Meteor.js has nearly all but #3 thanks to mongodb (1+2), dpp (4), mongo on the client (5), spark (6). The author considers a similar but Clojure-based stack (with Datomic, Angular etc.) and looks at authorization possibilities. “Secured, personalised, CRUD operations are the future to a more simplified web.” We may agree or not but it certainly is worth reading.
  • Michael Feathers (of Working Effectively With Legacy fame): Unconditional Programming – “Over and over again, I find that better code has fewer if-statements, fewer switches, and fewer loops. Often this happens because developers are using languages with better abstractions. [..] The problem with control structures is that they often make it easy to modify code in bad ways.” Includes a nice example of replacing if-else with much more readable and safer code.
  • The Quality of Embedded Software, or the Mess Has Happened – an interesting and scary read about terrible spaghetti code (and hardware) that makes some Toyotas to accelerate when the driver tries to break; 11,000 global variables, the key function showing so high cyclomatix complexity that “makes it impossible not only to test but even maintain this program in any way.” Then 80k violations of the car industry coding standard cannot surprise. And a safety control that does not work. Interesting that a great manufacturer may have so terrible IT (and Toyota isn’t the only one).
  • The string type is broken – the String type is M. Feathers’ favourite example of a leaky abstraction – most languages fail to process/split/count less common Unicode characters properly, the fact that String is implemented as a series of bytes leaks through (UTF-16 langs like Java); worth reading to be aware of the limitations

Languages

  • Why I’m Productive In Clojure – some interesting points about simplicity, consciousness, interactive development, power without overwhelming fatures, etc. “With it [Clojure] I can always easily derive a solution to a particular problem from a small set of general patterns. [..] However, the number of ways that these concepts can be combined to solve all manner of problems appears to be inexhaustible.
  • Node.js at PayPal – PayPal is switching from Java to Node.js (among others to promote language consistency) and, as a part of that, has implemented the same app in Node and Java; results: Node was done earlier, had less code, performed better (though, as Daniel Kvasnička pointed out, “Comparing Node.js and servlet-based archs is not fair… compare Node with @vertx_project and you’ll get a whole different story ;)”; also, as Charles Nutter said, “The @PayPal numbers for their Java to Node move are absurd. A JVM app doing 1.8 pages/s isn’t slow…it’s broken.“)
  • IBM: Developing mobile apps with Node.js and MongoDB, Part 1: A team’s methods and results – also IBM has implemented the same (REST) app once with Java and DB2, once with Node and Mongo where Node+Mongo required less work and performed better; one of the great wins was having JSON as a native structure everywhere instead of transforming from/to it so Mongo is, in my opinion, an important factor in this particular case
  • Dynamics of Programming: Benefits of Scala in CS1 – reasons for and experiences with using Scala in an introductory computer science course, worth reading; some of the advantages over Java are consistency (.asInt on String and Double vs. casting/parsing, no “primitive” types), REPL with time inference good for learning, functional style enables focus on what rather than how; quite persuasive arguments

Security

  • The New Threat: Targeted Internet Traffic Misdirection – did you know that internet traffic to any site can be made to go through a particular server without anybody noticing? This has been observed repeatedly in the wild, for banks and other sites. Rather make sure you use strong encryption (NSA-approved, of course ;)).
  • A (relatively easy to understand) primer on elliptic curve cryptography – Everything you wanted to know about the next generation of public key crypto – you cannot just read this, you have to study it, which I did not; but it looks good and I guess the time will come when I will come back to it

Other

Clojure Corner

  • Stuart Sierra’s Component – a library for making it easier to implement Stuart’s reloadable workflow; a component is something that can be started, stopped, and depend on other components so that it is easier to do interactive REPL development
  • Logan Linn: Clojure/conj 2013 – a pretty good overview of the conference
  • Caribou – “the kernel of usefulness that has emerged from years of this basic practice“- a new Clojure web framework – seems to be interesting
  • Results of the 2013 State of Clojure & ClojureScript survey and drill-down into what features people want – the most interesting fact is how many more participants use Clojure in production than last year and perhaps also the relatively wide adoption of Datomic among the respondents. Light Table has become the 3rd most popular dev. env., after Emacs and Vim. Some of the most mentioned language features requested were types (=> core.typed, Prismatic’s Schema), better error reporting (=> slingshotdireclj-stacktraceio.aviso:pretty, etc.), debuger (though progress is being made)
  • Book: Clojure High Performance Programming
  • Improving Clojure Feedback : Stack Traces – making Clojure stacktraces more usable by filtering out noise and linking to relevant content – io.aviso:pretty,  io.aviso:twixt
  • Clojure Dev discussion: Hashing strategies – Executive summary – “In Clojure, however, it is far more common than in Java to use longs, vectors, sets, maps and compound objects comprised of those components (e.g., a map from vectors of longs to sets) as keys in other hash maps.  It appears that Java’s hash strategy is not well-tuned for this kind of usage.  Clojure’s hashing for longs, vectors, sets, and maps each suffer from some weaknesses that can multiply together to create a crippling number of collisions.” Ex.: An implementation of N-Queens took forever, spending most time on hash collisions. But be calm, smart Clojurians are working on a solution.
  • Datomic Pro Starter Edition – Datomic with all storages, Datomic Console, a year of updates, full Datomic programming model; limitations: no HA transactor, no integrated memcached, max 2 peers and 1 transactor

Tools/Libs

  • AirPair.com – a new site that enables developers to get help from other devs via remote pairing, code review, code mentoring etc. – a good opportunity to get help / help others (and earn something); I haven’t tried it but it sounds pretty interesting

MongoDB web stacks

  • Meteor: JS frontend + MongoDB backend with changes in the DB pushed live to the clients, i.e. MongoDB is used both as the “application server” and storage. It seems great for apps where users need to collaborate in real-time with each other, certainly great for quick proof of concepts etc.; worth checking out; it also comes with free (at least for start?) hosting so really good for prototyping – “an open-source platform for building top-quality web apps in a fraction of the time.” The intro screencast will give you a good overview (10 min).
  • Mean.io – MEAN (Mongo, Express, Angular, Node) stack Boilerplate – frontend, backend and storage using the same language and some of the most popular technologies (not that popular = best fit for you :)); it seems to be very new but since it just glues together 4 popular and documented technologies, that should not be an obstacle. There is an intro on the MongoDB blog.

Other

  • BusyBox (get latest win binary from Tigress.co.uk) – reportedly a better POSIX env for Windows than gow, Cygwin, et al.

Favourite Quotes

[..] no organization should exist unless it is “of service” to its employees, its customers, its community.
- @Tom_Peters 28/11

I hope you’ll agree that there is a certain amount of irony involved in having to write repetitive code
Dmitri Sotnikov in Why I’m Productive In Clojure

Happy teams are productive teams but:

Morale is 95% a function of the prevailing system (the way the work works). Which in turn is a function of the prevailing collective mindset
@flowchainsensei Nov 10th

Posted in General, Languages, Top links of month | Tagged: , , , , , , , , , , | Comments Off

Most interesting links of January ’13

Posted by Jakub Holý on January 31, 2013

Recommended Readings

Various

  • Dustin Marx: Significant Software Development Developments of 2012 – Groovy 2.0 with static typing, rise of Git[Hub], NoSQL, mobile development (iOS etc.), Scala and Typesafe stack 2.0, big data, HTML5, security (Java issues etc.), cloud, DevOps.
  • 20 Kick-ass programming quotes – including Bill Gates’ “Measuring programming progress by lines of code is like measuring aircraft building progress by weight.”,  B.W. Kernighan’s “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”, Martin Golding’s “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” (my favorite)
  • How to Have a Year that Matters (via @gbrindusa) – do you want to just survive and collect possessions or do you want to make a difference? Some questions everybody should pose to him/herself.
  • Expression Language Injection – security defect in applications using JSP EL that can sometimes leads to double evaluation of the expressions and thus makes it possible to execute data supplied by the user in request parameters etc. as expressions, affects e.g. unpatched Spring 2.x and 3.

Languages etc.

  • HN discussion about Scala 2.10 – compilation speed and whether it matters, comparison of the speed and type system with Haskell and OCaml, problems with incremental compilation (dependency cycles, fragile base class), some speed up tips such as factoring out subprojects, the pros and cons of implicits etc.
  • Blog Mechanical Sympathy – interesting posts and performance tests regarding “writing software which works in harmony with the underlying hardware to gain great performance” such as Memory Access Patterns Are Important and Compact Off-Heap Structures/Tuples In Java.
  • Neal Ford: Functional thinking: Why functional programming is on the rise – Why you should care about functional programming, even if you don’t plan to change languages any time soon – N. Ford explains the advantages of FP and why FP concepts are spreading into other languages (higher abstractions enabling focus on the results over steps and ceding control to the language, more reusability on a finer level (higher-order functions etc.), few generic data structures with many operations -> better composability, “new” and different tool such as lazy collections, shaping the language towards the problem instead of vice versa, aligning with trends such as immutability)
  • Neal Ford: Java.next: The Java.next languages Leveraging Groovy, Scala, and Clojure in an increasingly polyglot world – a comparison of these languages with focus on what they are [not] suitable for, exploration of their paradigms (static vs. dynamic typing, imperative vs. functional)

SW development

  • How to Completely Fail at BDD – a story of an enthusiastic developer who tried to make everyone’s life better by introducing automated BDD tests and failed due to differences in culture (and inability to change thinking from the traditional testing), a surprising lack of interest in the tool and learning how to write good tests: “Culturally, my current team just isn’t ready or interested in something like this.” Morale: It is hard to change people, good ideas are not enough.
  • M. Feathers: Refactoring is Sloppy – refactoring is often prioritized out of regular development and refactoring sprints/stories aren’t popular due to past failures etc. An counter-intuitive way to get refactoring in is to imagine, during planning, what the code would need to be like to make it easy to implement a story. Then create a task for making it so before the story itself and assign it to somebody else then the story (to force a degree of scrutiny and communication). “Like anything else in process, this is medicine.  It’s not meant to be ‘the way that people do things for all time’ [..]” – i.e. intended for use when you can’t fit refactoring in otherwise. It may also make the cost of the current bad code more visible. Read also the commits (f.ex. the mikado method case).
  • Cyber-dojo: A great way to practice TDD together. Compare your read-green cycle and development over time with other teams. Purposefully minimalistic editor, a number of prepared tdd tasks.
  • On the Dark Side of “Craftsmanship” – an interesting and provoking article. Some developers, the software labouers, want to get work done and go home, they haven’t the motivation and energy to continualy spend time improving themselves. There is nothing wrong with that and we shouldn’t disparge them because of that. We shouldn’t divide people into craftsmen and the bad ones. A summary of and response to the varied reactions follows up in More on “Craftsmanship”. The author is right that we can’t expect everybody to spend nights improving her/his programming skills. Still they should not produce code of poor quality (with few exceptions) since maintaining such code costs a lot. There should be time for enough quality in a 9-5 day and people should be provided with enough guidance and education to be able to write decent code. (Though I’m not sure how feasible it is, how much effort it takes to become an acceptable developer.) Does the increased cost of writing (an learning to write) good code overweight the cost of working with bad code? That is an eternal discussion.

Cloud, web, big data etc.

  • Whom the Gods Would Destroy, They First Give Real-time Analytics (via Leon) – a very reasonable argument against real-time analytics: yes, we want real-time operational metrics but “analytics” only makes sense on a sensible amount of data (for the sake of statistical significance etc.) RT analytics could easily provide misguided results.
    CAP Twelve Years Later: How the “Rules” Have Changed (tl;dr, via @_dagi) – an in-depth discussion of the CAP theorem and the simplification (2 out of 3) that it makes; there are many more nuances. By Eric Brewer, a professor of computer science at the University of California, Berkeley, and vice president of infrastructure at Google.
  • ROCA: Resource-oriented Client Architecture – “A collection of simple recommendations for decent Web application frontends.” Server-side: true REST, no session state, working back/refresh etc. Client: semantic HTML independent of layout, progressive enhancement (usable with older browsers), usable without JS (all logic on the server) etc. Certainly not suitable for all types of apps but worthwile to consider the principles and compare them with your needs.

Clojure Corner

Tools

  • Vaurien, the Chaos TCP Proxy (via @bsvingen) – an extensible proxy that you can control from your tests to simulate network failure or problems such as delays on 20% of the requests; great for testing how an application behaves when facing failures or difficulties with its dependencies. It supports the protocols tcp, http, redis, memcache.
  • Wvanbergen’s request-log-analyzer for Apache, MySQL, PostgreSQL, Rails and more (via Zarko) – generates a performance report from a supported access log to point out requests that might need optimizing
  • Working Effectively With iTerm2 (Mac) – good tips in the body and comments

Favorite Quotes

A very good (though not very scientific) definition of project success applicable for distinguishing truly agile from process-driven projects:

[..] a project is successful if:

  • Something was delivered and put to use
  • The project members, sponsors and users are basically happy with the outcome of the project

- Johannes Brodwall in “How do we become Agile?” and why it doesn’t matter, inspired by Alistair Cockburn

(Notice there isn’t a single word about being “on time and budget”.)

Posted in General, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Most interesting links of December ’12

Posted by Jakub Holý on December 31, 2012

Recommended Readings

Software development

  • Kent Beck: When Worse Is Better: Incrementally Escaping Local Maxima – Kent reintroduces his Sprinting Centipede strategy (“reduce the cost of each change as much as possible so as to enable small changes to be chained together nearly continuously” => “From the outside it is clear that big changes are happening, even though from the inside it’s clear that no individual change is large or risky.”) and advices how to deal with situations where improvements have reached a local maxima by making the design temporarily intentionally worse (f.ex. by inlining all the ugly methods or writing to both the old and the new data store); strongly recommended
    • Related: Efficient Incremental Change – transmute risk into time by doing small, safe steps, then optimize your ability to make these steps quickly and thus being able to achieve large changes
  • Researchers: It is not profitable to outsource development – the Scandinavian research organisation SINTEF ICT has studied the effects of outsourcing and discovered that often it is more expensive than in-country development due to hidden costs caused by worse communication and cultural differences (f.ex. Indians tend not to ask questions and work based on their, often incomplete, understanding) and very high people turn-over; even after the true cost is discovered, companies irrationally stay there. However it is possible to succeed, in some cases.
  • Bjørn Borud: Tractor pulling and software engineering – very valuable and pragmatic advices on producing good software (i.e. avoiding accumulating so much crap that the software just stops progressing). Don’t think only about the happy path. Simplify. Write for other developers, i.e. avoid too “smart” solutions, test & document, dp actually think about design and its implication w.r.t performance etc. Awake the scientist in you: “Do things because you know they work, not because it happens to be the hip thing to do.”
    (Note: I see the good intention behind “design for the weakest programmer you can think of” but plase don’t take it too far! Software should be primarily simple, not necessarily easy.
  • Know your feedback loop – why and how to optimize it – to succeed, we need to learn faster; the only way to do that is to optimize our feedback loops, i.e. shorten the path our assumptions travel before they are (in)validated, whether about our code, business functionality, or the whole project idea. Conscise, valuable.
  • Code quality is the least important reason to pair program – the author argues, based on his experience, other benefits of pair programming are more important than code quality: “[..]  the most important reasons why we pair: it contributes to an amazing company culture, it’s the best way to bring new developers up to speed, and it provides a great way to share knowledge across the development team.”
  • You Can’t Refactor Your Way Out of Every Problem – refactoring can’t help you if the design is fundamentally wrong, you need to rewrite it; know when it can or cannot help and act accordingly (related to how much design is needed upfront since some design decision cannot be reverted/improved upon)

Languages

  • Josh Bloch: Java – the good, bad and ugly parts (video, 15 min); summary: right design decisions (VM, GC, threads, dynamic linking, OOP, static typing, exceptions, …), some bad details (signed byte, lossy long-> double, == doesn’t cal .equals, ability to call overriden methods from constructors, …); Mr. Bloch has also given a longer talk examining the evolution of Java from 1.0 to 1.7 in The Evolution of Java: Past, Present, and Future.
  • True Scala complexity – a thoughtful criticism of the complexity of Scala, based on code samples; “[it is true that] Scala is a language with a smaller number of orthogonal features than found in many other languages. […] However, the problem is that each feature has substantial depth, intersecting in numerous ways that are riddled with nuances, limitations, exceptions and rules to remember. It doesn’t take long to bump into these edges, of which there are many.”; however, its possible to avoid many of the problems mentioned by resorting to less smart, more clumsy and verbose Java-like code; also, the author still likes Scala.
  • Scala or Java? Exploring myths and facts (3/2012) – a balanced view of Scala’s strengths and weaknesses; “[..] the same features that makes Scala expressive can also lead to performance problems and complexity. This article details where this balance needs to be considered.” Topics: productivity, complexity, concurrency support, language extensibility, Java interoperability, quality of tooling, speed, backward compatibility. Plenty of useful links.

Big data & Cloud:

  • Dean Wampler’s slides from Beyond Map Reduce – 1) Hadoop Map Reduce is the EJB 2 of big data but there are better APIs such as Cascading with Scala/Clojure wrappers; there are also “alternative” solutions like Spark and Storm; 2) functional/relational programming with simple data structures (lists, sets, maps etc.) is much more suitable for big data than OOP (for we do mostly stateless data transformations)
  • Apache HBase vs Apache Cassandra – comparison sheet – if you want to decide between the two
  • Optimizing MongoDB on AWS – 20 min talk about the current state of the art. Simplicity: Mongo AMIs by 10gen, Cloudformation template etc. Stability & perf.: new storage options – EBS with provisioned IOPS volumes (high I/O) + EBS Optimized Instances (dedicated throughput to EBS), High IO instances (hi1.4xlarge – SSD)); comparison of throughput (number of operations, MBs) of these storages; tips for filesystem config. Scalability: scale horizontally and vertically, shrink as needed.
  • Getting Real About Distributed System Reliability by Jay Kreps, the author of the Voldemort DB: distributed software is NOT somehow innately reliable; a common mistake is to consider only probability of independent failures but failures typically are dependent (e.g. network problems affect the whole data center, not a single machine); the theoretical reliability “[..] is an upper bound on reliability but one that you could never, never approach in practice”; “For example Google has a fantastic paper that gives empirical numbers on system failures in Bigtable and GFS and reports empirical data on groups of failures that show rates several orders of magnitude higher than the independence assumption would predict. This is what one of the best system and operations teams in the world can get: your numbers may be far worse.” The new systems are far less mature (=> mor bugs, worse monitoring, less experience) and thus less reliable (it takes a decade for a FS to become mature, likely similar here). Distributed systems are of course more complex to configure and operate. “I have come around to the view that the real core difficulty of these systems is operations, not architecture or design.” Some nice examples of failures.

Other

  • Talks To Help You Become A Better Front-End Engineer In 2013 (tl;dr) – topics such as mobile web development, modern web devel. workflow, current/upcoming featrues of CSS3, ECMAScript 6, CSS preprocessors (LESS etc.), how to write maintainable JS, modular CSS, responsive design, JS debugging, offline webapps, CSS profiling and speed tracer, JS testing
  • On Being A Senior Engineer – valuable insights into what makes an engineer “senior” (i.e. mature; from the field of web operations but applies to IT in general): mature engineers seek out constructive criticism of their designs, understand the non-technical areas of how they are perceived (=> assertive, nice to work with etc.), understand that not all of their projects are filled with rockstar-on-stage work, anticipate the impact of their code (on resource usage, others’ ability to understand & extend it etc.), lift the skills and expertise of those around them, make their trade-offs explicit when making decisions, do not practice “Cover Your Ass Engineering,” are able to view the project from another person’s (stakeholder’s) perspective, are aware of cognitive biases (such as the Planning Fallacy), practice egoless programming, know the importance of (sometimes irrational) feelings people have.

Clojure Corner

  • Polymorphism in Clojure – Tim Ewald’s 1h live coding talk at Øredev conference introducing mechanisms for polymorphism (and Java interoperability) in Clojure and explaining well the different use cases for them. Included: why records, protocols & polymorphism with them (shapes, area => open, not explicit switch) (also good for Java interop.: interfaces), reify, multimethods.
  • Stuart Sierra: Thinking in Data (1h talk) – Sierra introduces data-oriented programming, i.e. programming with generic, immutable data structures (such as maps), pure functions, and isolated side-effects. Some other points: Records are an optimization, only for perforamnce (rarely) or polymorphism (ot often); the case for composable functions;  testing using simulations (generative testing) etc.; visualization of state & process

Tools & Libs

  • Netflix’ Hysterix: library to make distributed systems more resilitent by preventing a single slow/failing dependency from causing resource (thread etc.) exhaustion etc. by wrapping external calls in a separate thread with clear timeouts and support for fallbacks, with good monitoring etc. Read “Problem Definition” on the page to understand the problem it tries to solve.

Favorite Quotes

if you build something that is fundamentally broken it isn’t really interesting that you followed the plan or you followed some methodology — the thing you built is fundamentally broken.

- Bjørn Borud, Chief Architect at Comoyo.no, in an email 12/2012

The root of the Toyota Way is to be dissatisfied with the status quo; you have to ask constantly, “Why are we doing this?”

- Katsuaki Watanabe, Tyota President 2005 – 2009 (from the talk Deliberate Practice)

Posted in General, Languages, Tools, Top links of month | Tagged: , , , , , , , , , , , , | Comments Off

Most interesting links of November ’12

Posted by Jakub Holý on November 30, 2012

Recommended Readings

  • James Roper: Scaling Scala vs Java (recommended by M. Odersky) – writing scalable apps in Scala is much easier then Java because idiomatic Scala uses immutable structures and lends itself naturally to asynchronous processing while doing these things in Java is possible but very unnatural and laborious. “It [Scala] is biased towards scaling, it encourages practices that help you scale.”
  • Exception Handling Antipatterns (2006, still valuable) – Log and Throw, Throwing Exception (instead of a suitable subclass), Throwing the Kitchen Sink (declaring many exceptions in method signature), Catching Exception (instead of a particular subclass), Destructive Wrapping (not including the exception as cause), Log and Return Null, Catch and Ignore (swallowing the exception), Throw from Within Finally, Multi-Line Log Messages (via repeated log calls instead of \n), Ignoring InterruptedException (instead of breaking the loop etc.), Relying on getCause().
  • The GitHub way: Your team should work like an open source project – a provocative article about the development process in GitHub that strongly prefers asynchronous and on-line communication over face-to-face meetings and communication, which, according to the author, leads to increased productivity. That is quite the opposite of what is usually practiced. I can think of situation where direct interaction is invaluable but, on the other hand, I could certainly live with less meetings. (Comments on Hacker News)

Clojure Corner

  • Chas Emerick’s screencast Starting Clojure is a great example of Clojure development and interactive Clojure web development without restarts, with live code changes and direct access to the running app via REPL. It makes also a good job of introducing the Eclipse Clojure plugin Counterclockwise and the popular web framework Compojure with the template engine Enlive and HTTP abstraction Ring. Highly recommended! (I would however recommend to already know a little about the language.)
  • Results of the 2012 State of Clojure survey (and, for comparison, 2010 results) – some interesting facts are what people use Clojure for (math / data analysis 35%, web development 70%), 60% people evaluating ClojureScript, answers to “What have been the biggest wins for you in using Clojure?”, the fact that ~ 20% use Eclipse, around 60% Emacs, only 10% IntelliJ, 23% vim. Also interesting is “What has been most frustrating for you in your use of Clojure” (with 30% mentions of documentation, being now improved by clojure-doc.org, 23% “future stuffing concerns”)

Favorite Quotes

You can reach a point with Lisp where, between the conceptual simplicity, the large libraries, and the customization of macros, you are able to write only code that matters.

- Rich Hickey in an interview

Lisp was a piece of theory that unexpectedly got turned into a programming language.

- Paul Graham in Revenge of the Nerds

Posted in General, Languages, Top links of month | Tagged: , , , , | Comments Off

Most interesting links of October ’12

Posted by Jakub Holý on October 31, 2012

Recommended Readings

  • David Veksler: Some lesser-known truths about programming – things newcomers into the field of IT don’t know and don’t expect, true and an interesting read. Not backed by good data but anyway. F.ex.: “[..] a programmer spends about 10-20% of his time writing code [..] much of the other 90% thinking, researching, and experimenting”. “A good programmer is ten times more productive than an average programmer. A great programmer is 20-100 times more productive than the average [..]” “Bad programmers write code which lacks conceptual integrity, non-redundancy, hierarchy, and patterns, and so is very difficult to refactor.” “Continuous change leads to software rot, which erodes the conceptual integrity of the original design.” “A 2004 study found that most software projects (51%) will fail in a critical aspect, and 15% will fail totally.”
  • Brett L. Schuchert: Modern Mocking Tools and Black Magic – An example of power corrupting – interesting for two reasons: a good analysis of a poorly written piece of code and discussion of the code injection black magic (JMockIt) vs. actually breaking dependencies to enable tests.  The author presents a typical example of low-quality method (mixing multiple concerns, mixing different levels of abstractions, untestable due to a hardcoded use of an external call) and discusses ways to improve it and to make it testable. Recommended to read.
  • It’s Not About the Unit Tests – Learning from iOS Developers: iOS developers don’t do much testing yet they manage to produce high quality. How is that possible? The key isn’t testing itself, but caring for the code. (Of course, iOS is little special: small apps, no legacy, a powerful platform that does lot for the apps, very visual apps.) “It’s not about the practices. It’s about the spirit and intent behind them, and how they are applied.” (M. Fowler had a similar observation about a team that used mock-based testing exclusively and thus lacked integration tests yet all worked. [I’ve lost the link to the post and would be grateful for it])
  • Java Code Quality Tools – Overview – brief descriptions of 44 quality-related tools including some interesting tools and Eclipse plugins I didn’t know or knew but forgot. F.ex. analysis of dependencies with JBoss Tattletale or JarAnalyzer, Clirr to check libraries for source and binary backwards compatibility, JDiff generates JavaDoc-based report of removed/added/changed in an API. Spoon – read and check or transform Java code. Java PathFinder (NASA) – special JVM capable of checking all execution path to discover concurrency defects etc.

Tools

  • DirB, Directory Bookmarks for Bash (home) – moving efficiently among favourite directories (s <name> to create a bookmark for pwd, g <bookmark | relative/abs dir path> to enter a dir (=> works both for bookmarks and as a replacement for cd); also support for relative path bookmarks & more; sl lists bookmakrs in the last used order) (You might also want to check out Autojump, described in Dec 11; bashmarks is another similar project. Another similar project is rupa’s z and j2 and the fish clone z-fish)

Clojure Corner

  •  Jon Pither: Clojure at a Bank – Moving from Java –  the justification (productivity, dynamism, FP a better match for the domain) and process behind moving from Java to Clojure with a monolithic 1M LOC Spring/Hibernate app. (Random quotes: “I had used some dynamical languages before and it was quite obvious that we were essentially forcing lots of schema and type definition on to a problem domain that just didn’t want or need it.” “[..] it [dependency injection] just looks redundant in retrospect now that I’m working 95% with FP code.”) There is also a EuroClojure talk about their experiences one year later (35 min).
  • Prismatic’s “Graph” at Strange Loop – an interesting desing problem, its solution, and a resulting OSS library. The problem: How to break a large function into independently usable small ones that might depend on each other without ever needing to recompute a value once the function producing is called. The solution: Graph – “Graph is a simpledeclarative abstraction to express compositional structure.” (Enabling explicit declaration of data dependencies and pluging in different implementations.)
  • The Oblong: Blog about 2/3 D game programming in Clojure, starting from scratch (w/o an engine); interesting experiences
  • Ironclad: Steam Legions – Clojure game development battle report (the game on Github)
  • Building the Wishlisted.org webapp in Clojure – experiences from learning Clojure for real by building a webapp in Noir
  • Clojure vs. Scala smackdown (“Just kidding with the title of this post :-)”) – a short post with interesting discussion. Dmitri Sotnikov’s opinion resonates with me: “I found that for me Clojure wins on simplicity and consistency. While it looks more alien initially, once you learn the basics, you just reuse the same patterns everywhere.” Some more comments: “One major concern was maintainability, since it’s fairly easy to write very dense code. This turned out to not be a problem in practice. Because Clojure code is written as a tree, refactoring it is very easy.” REPL seems to be a big win (applies to Scala too). Scala’s type system might get tedious and learning its quirks takes time but there is lot of potential and both have they strong sides.
  • Code Fatigue – discussion of the advantages of learning, using, and combining the (many) standard Clojure functions instead of a “basic solution” using recursion etc. The argument is in favor of higher-level code with less complexity in the form of branching, recursion, nested expressions etc. and thus less mental fatigue.

Favorite Quotes

A classic test only cares about the final state – not how that state was derived. Mockist tests are thus more coupled to the implementation [emphasis mine] of a method. Changing the nature of calls to collaborators usually cause a mockist test to break.

- Martin Fowler in his classical Mocks Aren’t Stubs

I’m afraid of code. When I see a big pile of code, I get scared ;-). Some classes and method make me cry. I had troubles explaining why I prefer short pieces of code keeping the same level of abstraction, cohesive and loosely coupled. The following quote captures the essence – improved communication.

One way to improve communication is to reduce the need for it and the same can be said for code. […] Since we tend to read code more than write it, anything we can do to reduce the need to read code is time well invested in the life of a project.

- Brett L. Schuchert in Modern Mocking Tools and Black Magic – An example of power corrupting justifying extraction of code into another class or method

Posted in General, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , | Comments Off

My Scala vs. Clojure Impression In Pictures

Posted by Jakub Holý on October 13, 2012

(By kristobalite)

Clojure:
Clean
Structured
Focused

(By agiamba)

Scala:
Adorned
Overflowing
Magnificent

Clojure has a zen-like quality to it. There is extreme focus on simplicity, on defining few elementary orthogonal concepts that can be combined in powerful ways. For example it took 3 years for Clojure to get named parameters – but the result, destructuring, is something much richer and more applicable, that fits the language perfectly and has become a core part of idiomatic Clojure.

Scala feels as if trying to empower the programmer in any possible way, throwing in shortcuts, syntactic sugar, and plenty of methods so that anything can be done with the minimal amount of code. It is overwhelming.

Disclaimer: I have only a few weeks of experience with Scala and are in no position to judge it. This is just an impression I have gained so far and nothing more. I’m sure it is a great language.

Posted in General, Languages | Tagged: , , , | 1 Comment »

Most interesting links of February ’12

Posted by Jakub Holý on February 29, 2012

Recommended Readings

  • List of open source projects at Twitter including e.g. their scala_school – Lessons in the Fundamentals of Scala and effectivescala – Twitter’s Effective Scala Guide
  • M. Fowler & P. Sadalage: Introduction into NoSQL and Polyglot Persistence (pdf, 11 slides) – what RDBMS offer and why it sometimes isn’t enough, what the different NoSQL incarnations offer, how and on which projects to mix and match them
  • Two phase release planning – the best way to plan something somehow reliably is to just start doing it, i.e. just start the project with the objective of answering “Can this team produce a respectable implementation of that system by that date?” in as short time as possible (i.e. few weeks). Then: “Phase 2: At this point, there’s a commitment: a respectable product will be released on a particular date. Now those paying for the product have to accept a brute fact: they will not know, until close to that date, just what that product will look like (its feature list). What they do know is that it will be the best product this development team can produce by that date.” Final words: “My success selling this approach has been mixed. People really like the feeling of certainty, even if it’s based on nothing more than a grand collective pretending.”
  • Tumblr Architecture – 15 Billion Page Views A Month And Harder To Scale Than Twitter – what SW (Scala, Finagle, heavily partitioned MySQL, …) and HW they use, the architecture (Firehose – event bus, cell design), lessons learned (incl. “MySQL (plus sharding) scales, apps don’t.”
  • Jay Fields’ Thoughts: Compatible Opinions on Software – about teams and opinion conflicts – there are some areas where no opinion is really right (e.g. powerful language vs. powerful IDE) yet people may have very strong feeling about them. Be aware of what your opinions are and how strong they are – and compose teams so that they include more less people with compatible (not same!) opinions – because if you team people with strong opposing opinions, they’ll loose lot of productivity. Quotes: “I also believe that you can have two technically excellent people who have vastly different opinions on the most effective way to deliver software.” “I suggest that you do your best to avoid working with someone who has both an opposing view and is as inflexible as you are on the subject. The more central the subject is to the project, the more likely it is that productivity will be lost.”
  • Jay Fields’ Thoughts: Lessons Learned while Introducing a New Programming Language (namely Clojure) – introducing a new language and winning the hearts of (sufficient subset of) the people is difficult and requires lot of extra effort. This is both an experience report and a pretty good guide for doing it.
  • Jay Fields’ Thoughts: Life After Pair Programming – a proponent of pair-programming comes to the conclusion that in some contexts pairing may not be beneficial, i.e. the benefits of pair-programming don’t overweight the costs (for a small team, small software, …)
  • The Why Monitoring Sucks (and what we’re doing about it) – the #monitoringsucks initiative- what tools there are, why they suck, what to do, new tools, what metrics to collect, blogs, …
  • JBoss Byteman 2.0.0: Bytecode Manipulation, Testing, Fault Injection, Logging – a Java agent which helps testing, tracing, and monitoring code, code is injected based on simple scripts (rules) in the event-condition-action form (the conditions may use counters, timers etc.). Contrary to AOP, there is no need to create classes or compile code. “Byteman is also simpler to use and easier to change, especially for testing and ad hoc logging purposes.” “Byteman was invented primarily to support automation of tests for multi-threaded and multi-JVM Java applications using a technique called fault injection.” It was used e.g. to orchestrate the timing of activities performed by independent threads, for monitoring and statistics gathering, for application testing via fault injection. Contains a JUnit4 Runner for easily instrumenting the code under test, it can automatically load a rule before a test and unload it afterwards:
    @Test
    @BMRule(name="throw IOException at 1st call",
    targetClass = "TextLineProcessor",
    targetMethod = "processPipeline",
    action = "throw new java.io.IOException()")
    public void testErrorInPipeline() throws Exception { ... }
  • How should code search work? – a thought-provoking article about how much better code completion could be if it profited more from patterns of usage in existing source codes – and how to achieve that. Intermediate results available in the Code Recommenders Eclipse plugin.

REST

  • What Makes Jersey Interesting: Parameter Classes (by Coda Hale, 5/2009) – brief yet rich and very practical introduction into Jersey (the reference implementation of JAX-RS. i.e. REST, for Java) including error handling, parameter classes (automatic wrapping of primitive values). The following article, What Makes Jersey Interesting: Injection Providers, might be of interest too.
  • How to GET a Cup of Coffee, 10/2008 – good introduction into creating applications based on REST, explained on an example of building REST workflow for the ordering process in Starbucks – a “self-describing state machine”. The advantage of this article is that it presents the whole REST workflow with GET, OPTIONS, POST, PUT and “advanced” features such as the use of If-Unmodified-Since/If-Match, Precondition Failed, Conflict. The workflow steps are connected via the Location header and a custom <next> link tag with rel and uri. Other keywords: etag, microformats, HATEOS (-> derive the next resource to access from the links in the previous one), Atom and AtomPub, caching (web trades latency for scaleability; if 1+s latency isn’t acceptable than web isn’t the right platform), URI templates (-> more coupling than links in responses), evolution (-> links from responses, new transitions), idempotency. “The Web is a robust framework for integrating systems at local, enterprise, and Internet scale.”

Links to Keep

Tools, Libraries etc.

  • ClusterSSH – whatever commands you execute in the master SSH session are also execute in the slave sessions – useful if you often need to execute the same thing on multiple machines (requires Perl); to install on Mac: “brew install csshx”
  • HTML5 Boilerplate (H5BP) – customizable initial HTML5 project template for a website; can be combined e.g. with Bootstrap, the HTML/JS/CSS toolkit (there is even a script to set them both up). Includes server configs for optimal performance, “delivers best practices, standard elements”.
  • High performance libraries in Java – disruptor, Java Chronicle (ultra-fast in-memory db), Colt Matrix library (scientific computations), Javolution (RT Java), Trove collections for primitives, MG4J (free full-text search engine for large document collections), some serialization & other banchmarks links.
  • Twitter Finagle – “library to implement asynchronous Remote Procedure Call (RPC) clients and servers. Finagle is flexible enough to support a variety of RPC styles, including request-response, streaming, and pipelining; for example, HTTP pipelining and Redis pipelining. It also makes it easy to work with stateful RPC styles; for example, RPCs that require authentication and those that support transactions.” Supports also failover/retry, service discovery, multiple protocol (e.g. http, thrift). Build on Netty, Java NIO. See the overview and architecture.
  • Eclipse Code Recommenders – interesting plugin in incubation that tries to bring more more intelligent completion based more on context and the wisdom of the crowds (i.e. patterns of usage in existing source codes) to Eclipse

Clojure Corner

  • Clojure/huh? – Clojure’s Governance and How It Got That Way – an interesting description how the development of Clojure and inclusion of new libraries is managed. “Rich is extremely conservative about adding features to the language, and he has impressed this view on Clojure/core for the purpose of screening tickets.” E.g. it took two years to get support for named arguments – but the result is a much better and cleaner way of doing it.
  • Clojure Monads Series – comprehensive explanations of monads starting with Monads In Clojure

Quotes

A language that doesn’t affect the way you think about programming, is not worth knowing-

 - Alan Perlis

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

Eric S. Raymond, “How to Become a Hacker”

Posted in General, j2ee, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Most interesting links of November

Posted by Jakub Holý on November 30, 2011

Recommended Readings

  • Recommended Reading by Poppendiecks – an excellent selection, starting with Lean from Trenches, Management 3.0, Specification by Example, The Lean Startup etc.
  • Eric Allman says that Programming Isn’t Fun Any More  because problem solving has been replaced with learning, configuring, and integrating tons of libraries, frameworks, and tools and many people agree with that (as discussion on reddit proves). In other words we tend to go for any benefit we can have without considering the costs and for “easy” solutions without considering the true enemy: complexity. Perhaps we should always listen to the Rich Hickey’s Simple Made Easy talk before we add a lib/tool/framework?
    • Dean Wampler claims that functional programming can bring the joy back – “[..] a functional language, Scala, Clojure, Haskell, etc. will greatly reduce the amount of code you create. That won’t solve the problem of trying to integrate with too many libraries, but you’ll be less tempted. I also believe those libraries will be less bulky, etc.
    • Few quotes from a related article by M. Taylor: To put it another way, libraries make excellent servants, but terrible masters. | [..] frameworks [..] do keep their promise of making things very quick and easy … so long as you do things in exactly the way the framework author intended | On libraries: [..] we all assume (I know I do) that “plug in solutions X1 and X3″ is going to be trivial. But it never is — it’s a tedious exercise in impedance-matching, requiring lots of time spent grubbing around in poorly-written manuals [..] | On the effect of language choice: [..] different languages, with their different expressive power and especially their different culture, yield very different experiences.
    • To sum it up: Choose your tools and libraries wisely and always mind the global complexity. More usually means worse.
  • Java Magazine – Adam Bien: Stress Testing Java EE 6 Applications (page 41+) – do developer stress testing! Using: JMeter, VisualVM to find out resource consumption and behavior in the application, VisualVM’s Sampler profiling tool [cca 20% overhead], a webapp to extract metrics from GF (STM)
  • Java Magazine – Polyglot Programming on the JVM (page 50; excerpt from The Well-Grounded Java Developer) – why you should consider polyglot programming and how to decide whether to use it and what languages to pick, f.ex.: “These [Java’s] qualities make the language a great choice for implementing functionality in the stable layer [of the polyglot programming pyramid]. However, these same attributes become a burden in the middle and upper [lower, DSL, on the linked image] tiers of the pyramid; for example: Recompilation is laborious; Static typing can be inflexible and lead to long refactoring times; Deployment is a heavyweight process; Java’s syntax is not a natural fit for producing DSLs.” “There is a wide range of natural use cases for *alternative languages*. [after identifying such a UC] You *now need to evaluate* whether using an alternative language is appropriate.”
  • Intrusion Detection for Web Apps – Detection Points – If security is a concern of your web application then you should build intrusion detection into the application f.ex. leveraging the  OWASP AppSensor project. The key is to detect malicious/unexpected behavior and proactively do something such as locking the user out or alerting the admins. The page linked above lists some common suspicious behaviors such as the use of multiple usernames, unexpected HTTP command/method, additional/duplicated data in request. Worth checking out!
  • Yammer Moving From Scala to Java- Scala is a cool language but sometimes its cost is higher than the benefits. Snippets from the post: “…the friction and complexity that
    comes with using Scala instead of Java isn’t offset by enough productivity benefit or reduction of maintenance burden …”. “Scala, as a language, has some profoundly interesting ideas in it. […] But it’s also a very complex language. The number of concepts I had to explain to new members of our team for even the simplest usage of a collection was surprising: implicit parameters, builder typeclasses, ‘operator overloading’, return type inference, etc. etc.” (It’s claimed that only library authors need to know some of that but if it’s a part of library APIs, the users need to understand it too.) Notice that the author isn’t saying “Scala is bad” but only that Scala isn’t the best balance of their needs at this time, as Alex Miller put it*.
    Important note
    : The text wasn’t intended for publication and it is a private opinion of a Yammer developer, not the company itself. You should read the official Yammer’s position where Coda puts it into the right context.

Refactoring

  • Opportunistic Refactoring by Martin Fowler – refactor on the go – how & why
  • Michael Feathers: Getting Empirical about Refactoring – gather information that helps us understand the impact of our refactoring decisions using data from a SCM, namely File Churn (frequency of changes, i.e. commits) vs. Complexity – files with both high really need refactoring. Summary: “If we refactor as we make changes to our code, we end up working in progressively better code. Sometimes, however, it’s nice to take a high-level view of a code base so that we can discover where the dragons are. I’ve been finding that this churn-vs.-complexity view helps me find good refactoring candidates and also gives me a good snapshot view of the design, commit, and refactoring styles of a team.

UIs and Web Frameworks

  • Devoxx 2011 - WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks (slides) – the authors did extensive performance testing of some of the most popular web frameworks. Of course it’s always hard to guess how general their results are, if/how they apply to one’s particular situation, and if they aren’t distorted in some way but it’s worth for their approach alone (AWS with its CloudWatch monitoring, WebDriver, additional measurement of page load with HAR and a browser plugin). In their particular tests GWT scored best, followed by Spring MVC, with JSF and Wicket lagging far behind (especially the MyFaces implementation). Conclusion: A web framework may have strong impact on performance and scalability, if they are important for you then do test the performance early with as realistic code and load as possible.
  • JSF2 – Benchmark datatable by N. Labrot, 2/2011 – performance comparison of PrimeFaces 2.2.1, IceFaces 2.0, Richfaces 4.0.0M4 on a simple page with Ajax. I do not trust any benchmark that I don’t fake myself :-) (for there are always too many factors that influence the conclusions to be drawn) but it’s interesting anyway – and perhaps a good thing to do before you decide for a JSF component library.
  • Alex MacCaw: Asynchronous UIs – the future of web user interfaces and the Spine framework – users in 2011 shouldn’t anymore wait for pages to load and operations to complete, we should build asynchronous UIs where changes to the UI are performed immediately while a request to the server is sent in the background, similarly to sending e-mail in GMail, which returns at once displaying a non-intrusive “Sending…” notification. As a user I very much agree with Alex.
  • Matt Raible’s 20 criteria for evaluating web frameworks, 2010 (detailed description, here’s a brief list) – Matt’s results are disputable and as he himself says you should always do your own evaluation and spikes but the criteria are pretty useful: Developer Productivity, Developer Perception, Learning Curve, Project Health, Developer Availability, Job Trends, Templating, Components, Ajax, Plugins or Add-Ons, Scalability, Testing, i18n and l10n, Validation, Multi-language Support (Groovy / Scala), Quality of Documentation/Tutorials, Books Published, REST Support (client and server), Mobile / iPhone Support, Degree of Risk.

NoSql

  • Don’t use MongoDB via @nicolaiarocci – a (fake?!) bad experience with MongoDB – the text is not credible (the author is anonymous, s/he doesn’t explicitely state which version of MongoDB they used, the 10gen CTO can’t find a matching client and any evidence for some of the issues mentioned) but it  gives context for the read-worthy response from the 10gen CTO, and a post that nicely explains how to correctly design for MongoDB. A comment about MongoDB experience at Forsquare: “Currently we have dozens of MongoDB instances across several different data clusters storing over a TB of data and handling 10s of thousands of requests per second (mostly reads but the write load is reasonably high as well).Have we run into problems with MongoDB along the way? Yes, of course we have. It is a new technology and problems happen.Have they been problematic enough to seriously threaten our data? No they have not.
  • Martin Fowler on Polyglot Persistence – the are when will be choosing persistence solution with respect to our needs instead of mindlessly picking RDBMS is coming. Applications will combine multiple, specific solutions, f.ex. we could pick Redis (key-value) for caching, MongoDB (document DB) for product catalog, Neo4J (graph DB) for recommendations, RDBMS for financial data and reporting… (of course not all in one project!). Polyglot persistence will come at a cost (complexity, learning) – but it will come because the benefits are worth it – performance, data storage model and behavior more aligned with the business logic (NoSql databases ofer various models and tradeoffs and thus we can find a much better fit than with general-purpose RDBMs).

Talks & Video

  • Adam Bien’s JavaOne talk Java EE 6: The Cool Parts (1h) – absolutely worth the time – a very practical fly through the cool features of Java EE (eventing, ..), most of the time is spent actually coding. Don’t forget to check also the interesting discussion below the video (JEE and other frameworks, Java FX and JSF 2, …).
  • Jurgen Appelo’s keynote How to Change the World at Smidig 2011 is well done and highly useful. We all strive to change the world around us – as consultants we want to make our clients more agile, as team members we want to make our Scrum teams more self-organizing, as employees we want to help building knowledge-sharing and open culture, … . However it isn’t easy to influence or change people and culture and if we aren’t aware of all the dimensions of a change (system, individuals, interactions, environment) and how to work along each of them, we are much less likely to succeed. The knowledge and experience that Jurgen shares with us can help us a lot in having an impact. You can also download the slides and change management questions.
  • Project X: What is being a programmer like? (5min) If ever again a non-geek asks you what you as a developer are doing, just show him this short and extremely funny video (created by my ex-employer – perhaps they estimated how much time and energy developers loose trying to explain it to normal people and decided to prevent this great waste :-))
  • RSA Animate – Drive: The surprising truth about what motivates us (10 min) – entertaining and enlightening; once we’ve enough money to cover our needs, it’s autonomy (self-direction), mastery, and purpose what motivates us (money actually decrease our performance). Now this is a great evidence for lean/agile – for they’re based on making people self-directing and encourage mastery (as in continous integration and top quality to enable steady pace). Autonomy enables engagement as does a higher purpose (“make the world a better place”) – Steve Jobs with his visions was able to provide such a purpose. Atlassian’s FedEx Days are a good example of what engagement and benefits autonomy brings.
  • Simon Sinek: How great leaders inspire action (18 min, subtitles in 37 languages) – do you want to succeed, to change the world around you for the better, to start a new company? Then you must start by communicating “why” you do what you do, not “what” – like M. L. King, bro Wrights, and Apple. Very inspiring! (More in his Why book.)

Links to Keep

Favorite Quotes

Refactoring is like advertising: it doesn’t cost, it pays.
– Mary & Tom Poppendiecks, Implementing Lean Software Development, p.166

Clojure Corner

Posted in Databases, General, j2ee, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Most interesting links of February

Posted by Jakub Holý on February 28, 2011

Articles, links etc.

Git

  • The Git Parable (thx to Alexander) – a good and easy to understand explanation of the story behind Git and thus also its main goals and concepts. It really helps in understanding what makes Git different from Subversion and thus empowers you to use it to its full capabilities and in accordance with its philosophy and not (painfully) against it. Though I’d appreciate little more coverage of merging and cooperation in the Git world.

Performance

  • Some thoughts on stress testing web applications with JMeter (part 2) – what to measure, when to stop, what listeners to use, some practical tips e.g. for using remote slave JMeter instances, how to interpret the results (explanation of mean, std. deviation, confidence interval). By Nicolas Vahlas, 3/2010.
  • Scalability Factors of JMeter in Performance Testing projects (conference paper, 2008) – what factors determine what number of virtual users (VU) a load test tool can efficiently simulate and experiments to determine the impact of those factors on JMeter. Each VU has an associated cost in terms of CPU and memory (send, process, store request/response etc.) => number of VUs depends on the HW and the complexity of the messages and protocol and on the load test script complexity.
    • For example one commercial tool claims to support up to nearly 2k VUs on 1GHz PIII w/ 1GB but w/o stating the other factors.
    • JMeter scalability experiments results (Def.: Scalability limit reached when additional VUs aren’t increasing the server’s load (i.e. the test tool has become the bottleneck)):
      • Response time: “The optimal number of virtual users increases with increase in response time. It increases from around 180 virtual users to around 700 optimal virtual users when the response time changes from 500 ms to 1.5 seconds.”
      • Application response size: “Application response size has a massive effect on the load generation capabilities of JMeter. The optimal # of virtual users drops from around 500 virtual users to a measly 115 virtual users when the application response size increases form 20 kb to 100kb.”
      • Communication protocol (HTTP x HTTPS): The load generation capability decreases by 50% or more when the protocol is HTTPS
    • HW used for the test machine: P4 2.4GHz, 2GB RAM.
  • A. Bien: Can Stateful Java EE 6 Apps Scale? – Another Question Of The Week – the answer: Yes. A.B. always starts with Gateway+PDO and measures the performance/overhead with VisualVM, JMeter. Don’t forget that stateless applications usually just move the state and therefore scalability problem to the DB.

Other

  • Switching to Plan J – isn’t Scala good enough (yet)? – by Jonathan Edwards. Really an interesting read including the comments (well, the first half – esp. Martin, Stuart R., Vincent etc.) – though very impressive, is Scala too complex and do we need syntax subsets for normal people? The state of IDE support is bad but should be getting better.
  • Opinion: The unspoken truth about managing geeks – a great article on corporate culture and psychology, which turn people into stereotypical geeks – invaluable advices for managing IT professionals. And it’s fun to read! (Well, at least if you’re a geek observeing the very things he speaks about around you.)
  • ThoughtWorks Technology Radar, Januar 2011 – Scala up to Trial, DevOps, …. . Unchanged: ESB, GWT & RIA on hold, Apache Camel (integration library) on Trial. Other interesting: CSS supersets supporting variables etc. like LESS.

Quotes of the month

I’ve decided to add this occassional section to capture interesting or inspirational comments of famous as well as ordinary people.

  • if i learn, i end up going fast. if i just try to go fast, i don’t learn & only go fast briefly.” (Kent Beck’s Twitter, 2/11). Reflects nicely a discussion I recently had with a colleague :-) I also like the follow-ups:
    • Mike Hogan: any links/books you can suggest to help get more grounded in this mindset of valuing learning over trying to go fast?
    • Kent Beck: “zen in the art of archery” is one classic that i’ve found helpful. that and endlessly trying to do it wrong…
  • The computer industry is the only industry that is more fashion-driven than women’s fashion.” – by Richard Stallman in Cloud computing is a trap, 2008-09-29. I might not agree with everything he says but there certainly is a seed of truth in that!
  • Shipping is a feature. A really important feature. Your product must have it.by Joel Spolsky in The Duct Tape Programmer, 2009-09-23. This is something we should really keep in mind and explain to the product owners too :-)

Posted in General, Testing, Tools, Top links of month | Tagged: , , , , , , , | Comments Off