The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘trends’

Most interesting links of May ’13

Posted by Jakub Holý on May 31, 2013

Recommended Readings

Agile

  • Discussion: World’s Biggest ‘Agile’ Software Project Close To Failure – what is/isn’t agile, agile vs. waterfall etc; a nice collection of all the possible opinions and misunderstandings. Some of my favorites: waste: ‘[..]An “agile” project cannot fail and cost Billions because it must always deliver runnable software with a maximum of a few weeks delay[..]‘ (runnable = delivering value), separation: ‘[..] my experience has been that separating the designer/architect role from the developer role is fraught with pitfalls. The people writing the code should be the ones designing it, [..]‘, stability: ‘[..] On the successful projects that I’ve worked with in Agile, there’s strong stakeholders, good architecture keeping the vision in place and project management that keeps things well orchestrated. Without those in the mix, it’ll fail just like all software projects. [..]‘, waterfall success, people issue: ‘The problem here isn’t waterfall/agile. The problem here isn’t .Net/Linux. The problem here is the parties involved. [politicians and IT dinosaurs]‘ (learn what’s needed from drunk, bitching employees; ignore official nonsense requirements),: simplicity[..] It would probably be a lot easier if they started by making a simpler tool – instead of trying to calculate everybody’s entitlements everywhere [..]‘, agile suitability: ‘[..] You cannot use Agile to build a 100-mile canal, as the whole thing would be useless even if you completed 99 miles. [..]‘.
    Some people seem to believe that agile means no architecture and no/too little planning. Some believe that agile = hack now, fix later.

Startups etc.

Clojure Corner

  • B. Batsov’s Clojure Style Guide (based on JoC etc.) at GitHub
  • Code quaterly interview with R. Hickey (2011) – motivation behind Clojure (simplicity,..), reusability in C. (vs. Java); many valuable things
  • Clojure in the Enterprise? – about the differences between the Clojure and the enterprise java  (with heavy frameworks such as JPA, JSF, mutable state) ways and difficulties of introducing Clojure due to old-fashioned thinking, limited skills, etc. “Take away objects, mutable state, variables, loops… a lot of Java developers are immediately all at sea and have no idea how to solve even basic problems. They’ll try to bend Clojure to their OOP way of thinking and they’ll most likely fail.
    World Singles’ experience with Clojure: “We love Clojure. It’s made development a lot more fun. We’re able to solve harder problems, make changes faster, leverage multi-core concurrency more effectively, and we have a much smaller code base to maintain.
  • Replace Temp with Query in Clojure – in this post we can follow an interesting refactoring from a deeply nested if & let code to a much flatter one (featuring cond to test multiple conditions (instead of guard conditions used in imperative languages) delay to be ably to bind an expression to a local variable without evaluating it yet)
  • The Why and How of Clojure on Android (4/2013) – about the experience of using Clojure on Android, which is little slow and crazy but fun. From the author of the Nightweb app. Key components:  neko Clojure wrappers for Android and for lein-droid building w/o an IDE. Nicer than Java! (This GSoC proposal shows the limitations/future of neko.)
  • Reconstructing Clojure Macros With Speclj – a good idea to learn macros by trying to create one’s own implementation of existing Clojure macros, driven by tests that define the expected behavior of the macro (using macroexpand, macroexpand-1, macroexpand-all)
  • Advanced inspector middleware for Clojure nREPL and Emacs (nrepl-inspector on GitHub) – C-c C-i or nrepl-inspect to inspect interactively the value of a var (beware: (require ‘nrepl-inspect) fails, however calling nrepl-inspect as a function is ok). Related: Clojure Debugging ’13: Emacs, nREPL, and Ritz (May 17th) – what works, what is missing, how to set up
  • Pithering About » REPL bootstrap pimpage – neat tricks with a custom bootstrap REPL namespace – print last few commits and git branch, pre-loading common libs

Tools

Favorite Quotes

C, C#, and Java:
Applying some of the best ideas of the 1970s to the problems of today.

- Stuart Halloway in Clojure in the Field, 2013 (slide 6)

Posted in General, SW development, Tools, Top links of month | Tagged: , , , , , , | 2 Comments »

Most interesting links of May ’12

Posted by Jakub Holý on May 31, 2012

This was a rich month, bringing some hope for ORM, providing a peep-hole into the bright and awesome future with in-browser video and other cool web-stuff presented at WebRebels 2012 and IDEs providing immediate feedback and visualisation. There were valuable articles about simplicity and quality in software and good talks about the lean startup (i.e. enabling innovation) and other topics.

Recommended Readings

  • M.Fowler: ORM Hate – Why ORM is actually a good solution – a very valuable article where Fowler opposes the popular trend of criticising Object-Relational Mappers such as Hibernate. Yes, using an ORM is difficult and a leaky abstraction – but that’s because the problem of mapping from a rich in-memory object model to a relational store is inherently difficult (and you need to do it with or w/o an ORM tool) and because those last 10-20% of DB access require human intelligence. If you can avoid the need for ORM by using the relational model also in memory or by using a NoSQL database with a data model that fits your in-memory model then it’s great to do so but often you can’t and then using ORM is the best solution. You certainly don’t want to code your own “lightweight” ORM.
  • Alan Quayle on WebRTC, the HTML5 standard for in-browser video/text communication – intro & status – this is an exciting technology coming to our browsers. Some quotes: “WebRTC enables applications such as voice calls, video chat, file sharing, messaging, white-boarding, gaming, human computer interaction, etc. without any client or plug-in download to run from a browser using simple HTML and JavaScript APIs.  Real time communications becomes pervasive on the internet. … Essentially any browser becomes a SIP end point, a telephone, an ‘open’ Skype client, an end point for any real-time communication and control. … Likely by the end of this year we’ll see Chrome and Firefox running WebRTC.
  • Communicating Sequential Processes: Theory for reasoning about concurrent, interacting processes – an inspirational reading about a much better way to do concurrency than Java threads; “… [CSP] is a language for describing patterns of interaction between concurrent objects. It is supported by an elegant, mathematical theory, a set of proof tools, and an extensive literature.” The beauty is that thanks to the theory behind, you can actually reason about the interactions and verify their correctness, contrary to the feared mess of Java threads. CSP is broadly similar to the popular Actors model and is implemented in Occam while it also influenced Erlang’s concurrency model and Go. The library JSCP brings it to Java. I guess we’re better of using Actors due to their popularity and maturity though the mathematical backing of CSP with the potential of formal proofs of correctness is indeed attractive. Any of the two is better than using threads directly because:

    The monitor-threads model provided by Java, whilst easy to understand, proves very difficult to apply safely in any system above a modest level of complexity. One problem is that monitor methods are tightly interdependent, so that their semantics compose in non-trivial ways [...]

  • Rich Hickey introduces the Reducers library: simplicity in practice – a beautiful example of simplifying something by taking appart all the unrelated but mingled concerns and focus only on those really needed. Whether you’re interested in Clojure or not, you should read the beginning of the post where Hickey explains how the current collection functions based on first (returns 1st element) and rest (returns the remaining ones) mix too many things (ordering, output representation, etc.) and how this “new super-generalized and minimal abstraction for collections” avoids that and thus provides e.g. for doing things in parallel and composing transformation without producing intermediate collections. Beautiful! (PS: I’ve blogged about more examples of pursuing simplicity & gaining power.)
  • M. Fowler: Cannot Measure Productivity – a thoughful discussion of why the productivity of programmers is hard/impossible to measure (i.e. you should concentrate on measuring other, more useful metrics) “[..] false measures only make things worse.”
  • Gojko Adzic: Redefining software quality – an obligatory read that introduces a holistic view of SW quality and the quality pyramid. The key idea is that there are multiple, vertically organized facets of quality and once a more basic facet is saturated, you should move and and concentrate on the next facet and level of quality. The quality pyramid: Deployable & functionally OK > Performant & secure > Usable > Useful > Successful. Once a particular level is satisfied, it is wasteful to put more effort into it and you’ll bring much more value to the customer by focusing on the next higher level. Gojko: “Yet from what I see most software teams invest, build and test only at the lowest two levels, gold-plating things without a way to explain why that is bad.”
  • Is Pair Programming for Me? – the author, who claims to have taught pair programming to 200+500 people, points out that pair programming is a skill that must be (consciously) learned, or actually a number of inter-personal skills. He also describes the cycle people go through when learning it, including a temporary downswing in productivity and negative view of pairing (therefore people should do it at least for 3-4 weeks to overcome the problems and gain the benefits).

Videos

  • Bret Victor: Inventing on Principle (55 min, see at least the first 5 min) – very inspiring! Victor firmly believes that “creators need an immediate connection to what they create” and demonstrates how this can be achieve when coding image rendering, a game, an algorithm, when designing a circuit. After watching it for few minutes you will think: How could we have been working with such crappy tools without realizing how limited they are? Fortunately people started to apply the idea of an immediate connection between code and the result, f.ex. in LightTable and Bikeshed‘s IDE. On the other hand, there is an evidence that this may be too hard with the current programming languages.
  • Eric Ries: Evangelizing for the Lean Startup – entertaining and enriching introduction into an approach for bringing innovation to life – f.ex. in startups – withou failing unnecessary, demonstrated on the example of the author’s failed and successful startup. Many innovators fail because they don’t realize that their key challenge is that they don’t now neither the problem (who are our customers and what they need) nor the solution (the product to satisfy the need) and thus what they need to do is to experiment and learn in the shortest cycles possible. If you wander what the buzz about lean startup is or how to build innovations, this is the ultimite source you should watch. The video has 1h but the first 20-30 min will give you a sufficient overview. The key points summarized by the Iterate lean guru Anders Haugeto:
    dd

    - There is only one way to measure progress: Progress == The amount of things you have learned from your real customers
    - Hence, you need to work a continuous loop to build, measure and learn as fast as possible. Typical iterations, like sprints, are too long, hence inefficient
    - Until you have an established product, even recognized engineering practices like TDD, sprints, refactoring, and all the XP-stuff are less important than this feedback cycle
    - Even the perfect agile method is nothing, if you’re using it to build the wrong product: How can you know you are heading in the right direction?

  • WebRebels 2012 conference talks – I’d especially recommend the awakening talk by Zed Shaw pointing out that we’re building amazing things – but on top of crapy technologies without realizing anymore that the technologies are crappy and could/should be much better. Erlend Oftedal’s talk about webapp security was an (scary) eye-opener for me. If you’re considering offline webapps with HTML5′s webapp cache and/or local storage then you must listen to Jake Archibald’s painful story of various pitfalls hidden there. Christian Johansen’s Pure, functional JavaScript is a pleasure to listen to. Check out the program.

Links to Keep

  • E. King: Maximizing the Value of Your Stand-up – interesting techniques to try out at your stand-ups – Speed Scrum, Pass-the-Conch Scrum (passing a token randomly to define the order), Time-Box Scrum, Challenge Scrum (the team may ask 1 question each presenter), Impediments-Only Scrum, Award Scrum (reward for best articulation of his/her information), Business Value-Focused Scrum, No-Board Scrum, Whiteboard Scrum, Buddy Scrum (report for sb. else)

Useful Tools

  • puppet-lint – check code style of your Puppet files
  • Guard – cross-platform tool that can watch for file changes and execute actions (“guards”) when a file is changed, useful e.g. to execute tests/style checks only on the files being modified. Includes support for many testing/checking tools and multiple notification means such as Growl.
  • ThreadLogic – Thread dump analyzer that understands common patterns found in application servers and enabling the definition of custom patterns. Supports Sun, IBM, and JRockit.
  • Dumbster – mock SMTP server for unit testing (start in @Before, get sent messages in the test, stop afterwards)

Quotes

Kai Thomas Gilb, in a talk proposal for JavaZone 2012:

Accurate estimation is impossible for complex technical projects, but keeping to agreed budgets, and deadlines is achievable by using feedback and change.

Clojure Corner

  • StackOverflow: Clojure Performance Benchmarks – links to various discussions and benchmarks (beware that older results and facts are likely to be outdated). And of course you must keep in mind that 1) benchmark only measure what they measure, e.g. the outcomes cannot be generalized and that 2) benchmark results aren’t relevant for your problem unless you’re doing exactly the kind of operations being benchmarked (e.g. who cares that X is 100 ms slower if your code spends 1s waiting for a XML file download?) (Craig Andera had a pretty good experience report from webapp performance testing including what (not) to do)
  • A good wrapup of the EuroClojure conference by Deon Garrett. Such a pity I missed it!
  • Goldberg (at GitHub) – Johann Sebastian Bach’s Goldberg Variations in Overtone by @ctford; using Overtone and Clojure to build up mathematical and functional definitions of canons. Deon Garrett: “Go right now and download the code from Chris’ talk. If you don’t know Clojure, use this as an excuse to learn it – it’s that good.”

Rich Hickey interviewed by M. Fogus:

Reducing incidental complexity is a primary focus of Clojure, and you could dig into how it does that in every area.

In particular, the use of objects to represent simple informational data is almost criminal in its generation of per-piece-of-information micro-languages, i.e. the class methods, versus far more powerful, declarative, and generic methods like relational algebra. Inventing a class with its own interface to hold a piece of information is like inventing a new language to write every short story. This is anti-reuse, and, I think, results in an explosion of code in typical OO applications.

Posted in Databases, General, Testing, Tools, Top links of month | Tagged: , , , , , , , , , | Comments Off

Most interesting links of Mars ’12

Posted by Jakub Holý on March 31, 2012

Recommended Readings

  • ThoughtWorks Technology Radar 3/2012 – including apps with embedded servlet containers (assess), health check pages for webapp monitoring, testing at the appropriate level (adopt), JavaScript micro-framewors (trial, see Microjs.com), Gradle over Maven (e.g. thanks to flexibility), OpenSocial for data & content sharing between (enterprise) apps (assess), Clojure (before in asses) and CoffeeScript on trial (Scala very close to adopt), JavaScript as a 1st class language (adopt), single-threaded servers with aync I/O (Node.js, Webbit for Java [http/websocket], …; assess).
  • Jez Humble: Four Principles of Low-Risk Software Releases – how to make your releases safer by making them incremental (versioned artifacts instead of overwritting, expand & contract DB scripts, versioned APIs, releasing to a subset of customers first), separating software deployment from releasing it so that end-users can use it (=> you can do smoke tests, canary releasing, dark launching [feature in place but not visible to users, already doing something]; includes feature toggles [toggle on only for somebody, switch off new buggy feature, ...]), delivering features in smaller batches (=> more frequently, smaller risk of any individual release thanks to less stuff and easier roll-back/forward), and optimizing for resiliance (=> ability to provision a running production system to a known good state in predictable time – crucial when stuff fails).
  • The Game of Distributed Systems Programming. Which Level Are You? (via Kent Beck) – we start with a naive approach to distributed systems, treating them as just a little different local systems, then (painfully) come to understand the fallacies of distributed programming and start to program explicitely for the distributed environment leveraging asynchronous messaging and (often functional) languages with good support for concurrency and distribution. We suffer by random, subtle, non-deterministic defects and try to separate and restrict non-determinism by becoming purely functional … . Much recommended to anybody dealing with distributed systems (i.e. everybody, nowadays). The discussion is worth reading as well.
  • Shapes Don’t Draw – thought-provoking criticism of inappropriate use of OOP, which leads to bad and inflexible code. Simplification is OK as long as the domain is equally simple – but in the real world shapes do not draw themselves. (And Trades don’t decide their price and certainly shouldn’t reference services and a database.)
  • Capability Im-Maturity Model (via Markus Krüger) – everybody knows CMMI, but it’s useful to know also the negative directions an organization can develop in. Defined by Capt. Tom Schorsch in 1996, building on Anthony Finkelstein’s paper A Software Process Immaturity Model.
  • Cynefin: A Leader’s Framework for Decision Making – an introduction into the Cynefin cognitive framework – the key point is that we encounter 5 types of contexts differing by the predictability of effects and each of them requires a different management style, using the wrong one is a recipe for a disaster. Quote:

    The framework sorts the issues facing leaders into five contexts defined by the nature of the relationship between cause and effect. Four of these—simple, complicated, complex, and chaotic—require leaders to diagnose situations and to act in contextually appropriate ways. The fifth—disorder—applies when it is unclear which of the other four contexts is predominant.

  • Et spørsmål om kompleksitet (Norwegian). Key ideas mixed with my own: Command & control management in the traditional Ford way works very well – but only in stable domains with clear cause-and-effect relationships (i.e. the Simple context of Cynefin). But many tasks today have lot of uncertanity and complexity and deal with creating new, never before seen things. We try to lead projects as if they were automobile factories while often they are more like research – and researchers cannot plan when they will make a breakthrough. Most of the new development of IT systems falls into the Complex context of Cynefin – there is lot of uncertanity, no clear answers, we cannot forsee problems, and have to base our progress on empirical experience and leverage emergence (emergent design, ..).
  • The Economics of Developer Testing – a very interesting reflection on the cost and value of testing and what is enough tests. Tests cost to develop and maintain (and different tests cost differently, the more complex the more expensive). Not having tests costs too – usually quite a lot. To find the right ballance between tests and code and different types of tests we must be aware of their cost and benefits, both short & long term. Worth reading, good links. (Note: We often tend to underestimate the cost of not having good tests. Much more then you might think.)

Links to Keep

Quotes

Kent Beck answering a question about how much testing to do (highlighted by me):

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Different people will have different testing strategies based on this philosophy, but that seems reasonable to me given the immature state of understanding of how tests can best fit into the inner loop of coding. Ten or twenty years from now we’ll likely have a more universal theory of which tests to write, which tests not to write, and how to tell the difference. In the meantime, experimentation seems in order.

Posted in General, Testing, Top links of month | Tagged: , , , , , | Comments Off

Most interesting links of February

Posted by Jakub Holý on February 28, 2011

Articles, links etc.

Git

  • The Git Parable (thx to Alexander) – a good and easy to understand explanation of the story behind Git and thus also its main goals and concepts. It really helps in understanding what makes Git different from Subversion and thus empowers you to use it to its full capabilities and in accordance with its philosophy and not (painfully) against it. Though I’d appreciate little more coverage of merging and cooperation in the Git world.

Performance

  • Some thoughts on stress testing web applications with JMeter (part 2) – what to measure, when to stop, what listeners to use, some practical tips e.g. for using remote slave JMeter instances, how to interpret the results (explanation of mean, std. deviation, confidence interval). By Nicolas Vahlas, 3/2010.
  • Scalability Factors of JMeter in Performance Testing projects (conference paper, 2008) – what factors determine what number of virtual users (VU) a load test tool can efficiently simulate and experiments to determine the impact of those factors on JMeter. Each VU has an associated cost in terms of CPU and memory (send, process, store request/response etc.) => number of VUs depends on the HW and the complexity of the messages and protocol and on the load test script complexity.
    • For example one commercial tool claims to support up to nearly 2k VUs on 1GHz PIII w/ 1GB but w/o stating the other factors.
    • JMeter scalability experiments results (Def.: Scalability limit reached when additional VUs aren’t increasing the server’s load (i.e. the test tool has become the bottleneck)):
      • Response time: “The optimal number of virtual users increases with increase in response time. It increases from around 180 virtual users to around 700 optimal virtual users when the response time changes from 500 ms to 1.5 seconds.”
      • Application response size: “Application response size has a massive effect on the load generation capabilities of JMeter. The optimal # of virtual users drops from around 500 virtual users to a measly 115 virtual users when the application response size increases form 20 kb to 100kb.”
      • Communication protocol (HTTP x HTTPS): The load generation capability decreases by 50% or more when the protocol is HTTPS
    • HW used for the test machine: P4 2.4GHz, 2GB RAM.
  • A. Bien: Can Stateful Java EE 6 Apps Scale? – Another Question Of The Week – the answer: Yes. A.B. always starts with Gateway+PDO and measures the performance/overhead with VisualVM, JMeter. Don’t forget that stateless applications usually just move the state and therefore scalability problem to the DB.

Other

  • Switching to Plan J – isn’t Scala good enough (yet)? – by Jonathan Edwards. Really an interesting read including the comments (well, the first half – esp. Martin, Stuart R., Vincent etc.) – though very impressive, is Scala too complex and do we need syntax subsets for normal people? The state of IDE support is bad but should be getting better.
  • Opinion: The unspoken truth about managing geeks – a great article on corporate culture and psychology, which turn people into stereotypical geeks – invaluable advices for managing IT professionals. And it’s fun to read! (Well, at least if you’re a geek observeing the very things he speaks about around you.)
  • ThoughtWorks Technology Radar, Januar 2011 – Scala up to Trial, DevOps, …. . Unchanged: ESB, GWT & RIA on hold, Apache Camel (integration library) on Trial. Other interesting: CSS supersets supporting variables etc. like LESS.

Quotes of the month

I’ve decided to add this occassional section to capture interesting or inspirational comments of famous as well as ordinary people.

  • if i learn, i end up going fast. if i just try to go fast, i don’t learn & only go fast briefly.” (Kent Beck’s Twitter, 2/11). Reflects nicely a discussion I recently had with a colleague :-) I also like the follow-ups:
    • Mike Hogan: any links/books you can suggest to help get more grounded in this mindset of valuing learning over trying to go fast?
    • Kent Beck: “zen in the art of archery” is one classic that i’ve found helpful. that and endlessly trying to do it wrong…
  • The computer industry is the only industry that is more fashion-driven than women’s fashion.” – by Richard Stallman in Cloud computing is a trap, 2008-09-29. I might not agree with everything he says but there certainly is a seed of truth in that!
  • Shipping is a feature. A really important feature. Your product must have it.by Joel Spolsky in The Duct Tape Programmer, 2009-09-23. This is something we should really keep in mind and explain to the product owners too :-)

Posted in General, Testing, Tools, Top links of month | Tagged: , , , , , , , | Comments Off