The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘clojure’

Most interesting links of June ’14

Posted by Jakub Holý on June 30, 2014

Recommended Readings

  • The emperor’s new clothes were built with Node.js – I know sadly little about Node.js but this goes against the hype and is thus interesting. So what does Node.js give us? Performance 1-5x slower than Java [like Clojure] according to the Benchmarks Game (contrary to other benchmarks with the opposite result as mentioned in the comments), use of a single CPU/core on our multi-cpu, multi-core machines, callback hell. At the same time, there are good non-blocking servers available in other languages (Clojure’s http-kit, Vert.x, etc.) (Update: From the comments it seems that f.ex. the “callback hell” situation is geting better with 0.11, fibers and other things I do not know anything about. Also Sandro has a nice anti-comment (No. 36).)
    The Node.js Is Bad Ass Rock Star Tech 5 min video is a nice companion :)
  • The Expert (Short Comedy Sketch)  (7 min) – you’ve certainly seen this one but I had to put it here; a young engineer is hammered into being an “Of course I can do it, I am an expert” ‘expert/consultant’ during a business meeting. Maybe you too have experienced a dialog with the business where your true expert opinion was crushed by the business people’s insistence on their absurd requirements?
  • Reset The Net – Privacy Pack – privacy-enhancing apps for PC/mobile
  • The Dyslexic Programmer (via Kent Beck) – interesting read about quite a different way to percieve and think about code, the advantages of IDEs.
  • It’s Here: Docker 1.0 => more stable from now on
  • Kent Beck: Learning About TDD: The Purpose of #isTDDDead – what is the purpose and value of TDD? Where are the limits of its value? “I recognize that TDD loses value as tests take longer to run, as the number of possible faults per test failure increases, as tests become coupled to the implementation, and as tests lose fidelity with the production environment.
  • Failure & Cake: A Guide to Spotify’s Psychology of Success – want to be innovative and successfull? Learn to embrace failure, nurture the “growth mindset” (failure as opportunity to improve) rather than the “fixed mindset” (I do not learn and every failure shows I have no value). Read this if you want your org to be a better place to work!

Non-tech

  • LSD — The Problem-Solving Psychedelic – I never knew that drugs could be used to something positive, with an incredible effect. Are you stuck with a tech/design/art problem? Try LSD! :-)
  • The French are right: tear up public debt – most of it is illegitimate anyway – “Debt audits show that austerity is politically motivated to favour social elites. [..] 60% of French public debt is illegitimate” – not improving the lives of people but thos at power/rich. Time to reconsider this debt business and ways to make our system better?
  • Forbes: Why Financialization Has Run Amok – Wall Street is the kind and companies do everything to look better in its eyes – including giving up on opportunities. The might of the finance sector is destructive to our economy and distorts it, away from producing more value to making financial institutions richer, away from (value) creative activities to distributive ones. The article describes the problem and proposes a solution including limiting the size and leverage of banks, taxing financial transactions etc. Example of the effects: “[..] a cabal of senior IBM executives and the managers of some big investment firms got together and devised a five-year scheme—IBM’s Roadmap 2015—for increasing IBM’s earnings per share—and their own compensation—through measures that are not only increasing earnings per share but also steadily crippling IBM’s ability to innovate and compete [..]
  • Why Can’t We All Just Get Along? The Uncertain Biological Basis of Morality – very interesting criticism of “morality” that is mostly based on emotions and thus contradictory, a good argument for utilitarian morality [not that it hasn't its own challenges]. According to the author, many conflicts are nor primarily due to divergent values but due to different interpretation of the reality and history (such as “who has right to this land?”). People suffer “[..] from a deep bias—a tendency to overestimate their team’s virtue, magnify their grievances, and do the reverse with their rivals.” “This is the way the brain works: you forget your sins (or never recognize them in the first place) and remember your grievances. [..] As a result, the antagonisms confronting you may seem mysterious, and you may be tempted to attribute them to an alien value system.” This leads to partial judgements that play very badly with another psychological feature – “Namely: the sense of justice—the intuition that good deeds should be rewarded and bad deeds should be punished.” “When you combine judgment that’s naturally biased with the belief that wrongdoers deserve to suffer, you wind up with situations like two people sharing the conviction that the other one deserves to suffer. Or two groups sharing that conviction. And the rest is history.” And “The most common explosive additive is the perception that relations between the groups are zero-sum—that one group’s win is the other group’s loss.” => “So maybe the first step toward salvation is to become more self-aware.
    When you’re in zero-sum mode and derogating your rival group, any of its values that seem different from yours may share in the derogation. Meanwhile, you’ll point to your own tribe’s distinctive, and clearly superior, values as a way of shoring up its solidarity. So outsiders may assume there’s a big argument over values. But that doesn’t mean values are the root of the problem.
    Those who choose not to act in the trolley dilemma[..] are just choosing to cause five deaths they won’t be blamed for rather than one death they would be blamed for. Not a profile in moral courage!

Clojure Corner

  • The Case for Clojure (video, 5 min) – a short video arguing for Clojure as a good solution language based on its simplicity, power, and fun factor. There are many claims and few facts (as dictated by the short length) but it might be interesting for somebody.
  • CrossClj.info – cross-reference of many OSS Clojure projects – find all uses of a fn across the projects, all fns with a given name, all projects using ring, … . Search by fn, macro, var, ns, prj.
  • The Weird and Wonderful Characters of Clojure – ‘A reference collection of characters used in Clojure that are difficult to “google”.’

Tools/Libs

Posted in General, Languages, Tools, Top links of month | Tagged: , , , , , , , , | Leave a Comment »

Review: Clojure for Machine Learning (Ch 1-3)

Posted by Jakub Holý on June 26, 2014

Book coverPack Publishing has asked me to review their new book, Clojure for Machine Learning (4/2014) by Akhil Wali. Interested both in Clojure and M.L., I have taken the challenge and want to share my impressions from the first chapters. Regarding my qualification, I am a medium-experienced Clojure developer and have briefly encountered some M.L. (regression etc. for quantitive sociological research and neural networks) at the university a decade ago, together with the related, now mostly forgotten, math such as matrices and derivation.

In short, the book provides a good bird-eye view of the intersection of Clojure and Machine Learning, useful for people coming from both sides. It introduces a number of important methods and shows how to implement/use them in Clojure but does not – and cannot – provide deep understanding. If you are new to M.L. and really like to understand things like me, you want to get a proper textbook(s) to learn more about the methods and the math behind them and read it in parallel. If you know M.L. but are relatively new to Clojure, you want to skip all the M.L. parts you know and study the code examples and the tools used in them. To read it, you need only elementary knowledge of Clojure and need to be comfortable with math (if you haven’t worked with matrices, statistics, or derivation and equations scare you, you will have a hard time with some of the methods). You will learn how to implement some M.L. methods using Clojure – but without deep understanding and without knowledge of their limitations and issues and without a good overview of alternatives and the ability to pick the best one for a particular case.

Read the rest of this entry »

Posted in Languages | Tagged: , , , | Comments Off

Most interesting links of May ’14

Posted by Jakub Holý on May 31, 2014

Recommended Readings

  • Monolith – from The Codeless Code – fables and koans for the SW engineer – the Monad monolth #Haskell #fun
  • http2 explained (pdf, 27 pages) – cons of http 1 (huge spec / no full impl., wasteful use of TCP <=> latency [x spriting, inlining, concatenation, sharding]) => make it less latency sensitive, fix pipelining (issue a req before previous one finished), stop the need for ever increasing # connections, remove/reduce optional parts of http. Http2 is binary; multiple “streams” over 1 connection => much less conns, faster data delivery; header/data compression; [predictive] resource pushing; . Inspired by SPDY. Chrome and Mozilla will only support it over TLS, yay! (see also Is TLS Fast Yet? [yes, it is]) Promise: faster, more responsive web pages & deprecation of http/1 workarounds => simplified web dev.

Special

  • exercism.io – crowd-sourced good code mentorship – get an exercise, implement it in any of the supported language(s), submit and get feedback, repeat; when finished, you too can comment the same excercise submitted by others while working on your next assignment. Languages include Clojure, JS, Scala, Python, Haskell, Go, Elixir, Java, and more.

Podcasts (FP & related)

  • Cognicast (also @ iTunes) – Clojure, FP, etc.
  • Functional Geekery (@ iTunes) – A podcast on Functional Programming, covering topics across multiple languages.
  • Mostly λazy…a Clojure podcast by Chas Emerick
  • Giant Robots Smashing into other Giant Robots – “a weekly technical podcast discussing development, design, and the business of software development”
  • Software Engineering Radio (@ iTunes) – “The goal is to be a lasting educational resource, not a newscast. Every two to four weeks, a new episode is published that covers all topics software engineering. Episodes are either tutorials on a specific topic, or an interview with a well-known expert from the software engineering world.”
  • EngineerVsDesigner – design insight (@ iTunes) – product design podcast – the latest digital design news, tips & tricks, Q&A, and an industry special guest

Other

Clojure Corner

Tools/Libs

  • ownCloud – your own Dropbox/Google Drive, run on your server – sharing files between devices / PCs / web, syncing calendar and contacts, collaborative editing of documents (ODF)
  • Mailpile – “A modern, fast web-mail client with user-friendly encryption and privacy features.”, to be self-hosted on a PC, RaspberryPI, USB stick
  • Blackhole – role-based ssh proxy – an app that enables you to manage what users can ssh to what server as a particular user, from users’ point of view this is a ssh proxy; useful if many people need access to many servers but you do not want to add them all as users on those servers.
  • Wuala – Secure Cloud Storage – Backup. Sync. Share. Access Everywhere. – Dropbox alternative, secure by default
  • fb-flo – Facebook’s live-coding tool
  • owncloud.org – self-hosted Dropbox-like service with calendar and contact sync and more

Favourite Quotes

Posted in General, Languages, Tools, Top links of month | Tagged: , , , , , , , , , , | Comments Off

Clojure/Java: Prevent Exceptions With “trace missing”

Posted by Jakub Holý on May 19, 2014

The other day I got this little helpful exception from Clojure:

(cond (>= nil 1) :unreachable)
;=> NullPointerException [trace missing]

- no line number or anything to troubleshoot it.

It turns out it is not Clojure’s failure but a HotSpot optimization that can apply to NullPointerException, ArithmeticException, ArrayIndexOutOfBoundsException, ArrayStoreException, and ClassCastException. The remedy is to run the JVM with

-XX:-OmitStackTraceInFastThrow

From Oralce JDK release notes:

The compiler in the server VM now provides correct stack backtraces for all “cold” built-in exceptions. For performance purposes, when such an exception is thrown a few times, the method may be recompiled. After recompilation, the compiler may choose a faster tactic using preallocated exceptions that do not provide a stack trace. To disable completely the use of preallocated exceptions, use this new flag: -XX:-OmitStackTraceInFastThrow.

Many thanks to Ivan Kozik for the info!

Posted in Languages | Tagged: , | Comments Off

Most interesting links of April ’14

Posted by Jakub Holý on April 30, 2014

Recommended Readings

  • The economics of reuse – developing code for reuse costs much more than for one need – it might cost 300% more to develop and save you 75% of work when (re)using it instead of developing from scratch (if one of the factors goes down, the other one typically goes down too). Summary: “That means that to get any value from your reused component, you better have five or more reusers or you have to find a way to substantially improve the [reuse value factor] or [reusability cost factor]. Very smart people have failed to do this.
  • Book in making: Reactive Design Patterns (1st ch free)

Sharing data on the web

Clojure Corner

  • 8th Light: Combining Clojure and ClojureScript Libraries (3/2014) – really good and detailed article / tutorial using CLJX and platform-specific platform.clj[s] files to share code between Clojure and ClojureScript. It also recommends a file structure (src/(clj|cljs)/), demonstrates testing, discusses macro development, shows how to pack both into one jar.

Tools/Libs

Favourite Quotes

Posted in General, Languages, Top links of month | Tagged: , , , , | Comments Off

Clojure: How To Prevent “Expected Map, Got Vector” And Similar Errors

Posted by Jakub Holý on April 30, 2014

What my Clojure code is doing most of the time is transforming data. Yet I cannot see the shape of data being transformed – I have to know what the data looks like on the input and hold a mental model of how they change at each step. But I make mistakes. I make mistakes in my code so that the data does not correspond anymore to the model it should follow. And I make mistakes in my mental model of what the data currently looks like, leading likely to a code error later on. The end result is the same – a little helpful exception at some later step regarding wrong shape of data. There are two problems here: The error typically provides too little useful information and it usually manifests later than where the code/model mistake actually is. I therefore easily spend an hour or more troubleshooting these mistakes. In addition to that, it is also hard to read such code because a reader lacks the writer’s mental model of the data and has to derive it herself – which is quite hard especially if the shape of the input data is not clear in the first place.

I should mention that I of course write tests and experiment in the REPL but I still hit these problems so it is not enough for me. Tests cannot protect me from having a wrong model of the input data (since I write the [unit] tests based on the same assumptions as the code and only discover the error when I integrate all the bits) and even if they help to discover an error, it is still time-consuming the root cause.

Can I do better? I believe I can.

Read the rest of this entry »

Posted in Languages | Tagged: | 8 Comments »

Most interesting links of March ’14

Posted by Jakub Holý on March 31, 2014

Recommended Readings

Clojure Corner

  • Timo Mihaljov’s Pimp My REPL (3/2014)- really great tips – user.clj, :dev profile, user-wide config in .lein/profiles.clj, tools.namespace, making funs available everywhere & more via Vinyasa, form println with Spyscope, debug-repl, difform, clj-ns-browser

Tools/Libs

  • clj-ds – Clojure immutable datastructures extracted from Clojure and made easier for use directly in Java

Favourite Quotes

Posted in General, Top links of month | Tagged: | Comments Off

Most interesting links of February ’14

Posted by Jakub Holý on February 28, 2014

Recommended Readings

Development

  • Nathan Marz: Principles of Software Engineering, Part 1 – Nathan has worked with Big Data at Twitter and other places and really knows the perils or large, distributed, real-time systems and this post contains plenty of valuable advice for making robust, reliable SW. Main message: “there’s a lot of uncertainty in software engineering“; every SW operates correctly only for a certain range of inputs (including volume, HW it runs on, …) and you never control all of them so there always is an opportunity for failure; you can’t predict what inputs you will encounter in the wild. “[..] while software is deterministic, you can’t treat it as deterministic in any sort of practical sense if you want to build robust software.” “Making software robust is an iterative process: you build and test it as best you can, but inevitably in production you’ll discover new areas of the input space that lead to failure. Like rockets, it’s crucial to have excellent monitoring in place so that these issues can be diagnosed.“. From the content: Sources of uncertainty (bugs, humans, requirements, inputs, ..), Engineering for uncertainty (minimize dependencies, lessen % of cascading failure [JH: -> Hystrix], measure and monitor)
    • Suffering-oriented programming is certainly also worth reading (summary: do not start with great designs; only start generalizing and creating libs when you have suffered enough from doing things more manually and thus learned the domain; make it possible > make it beautiful > make it fast, repeat)
  • ThoughtWorks open-sources Go, continuous delivery platform – good bye, Jenkins! – better support for pipelines etc., see features and elementary concepts
  • Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications (recommended by @markusbk so it must be good); Patterns: Cache-aside, Circuit Breaker, Compensating Transaction, Competing Consumers, Compute Resource Consolidation, Command and Query Responsibility Segregation (CQRS), Event Sourcing, External Configuration Store, Federated Identity, Gatekeeper, Health Endpoint Monitoring, Index Table, Leader Election, Materialized View, Pipes and Filters, Priority Queue, Queue-based Load Leveling, Retry, Runtime Reconfiguration, Scheduler Agent Supervisor, (data) Sharding, Static Content Hosting (-> CDN), Throttling, Valet Key.
    Guidance topics: Asynchronous Messaging Primer, Autoscaling, Caching, Compute Partitioning, Data Consistency Primer, Data Partitioning, Data Replication and Synchronization, Instrumentation and Telemetry, Multiple Datacenter Deployment, Service Metering
  • MOOC course Functional programming with Clojure at Uni of Helsinki – to get started you need, I suppose, follow the “Material and course content” – essentially read the text for each chapter, clone its repo, submit pull requests to get your work graded
  • Jez Humble: The Case for Continuous Delivery – read to persuade manager about CD: “Still, many managers and executives remain unconvinced as to the benefits [of CD], and would like to know more about the economic drivers behind CD.” CD reduces waste: “[..]online controlled experiments (A/B tests) at Amazon. This approach created hundreds of millions of dollars of value[..],” reduces risks: “[..] Etsy, has a great presentation which describes how deploying more frequently improves the stability of web services.” CD makes development cheaper by reducing the cost of non-value-adding activities such as integration and testing. F.ex. HP got dev. costs down by 40%, dev cost/program by 78%

Web

  • Client-side messaging in JavaScript – Part 3 (anti-patterns) (via @ruudud so it must be worth reading)
  • Request Quest (via @jraregris) – entertaining and educational intractive quiz regarding what does (not) trigger a request in browsers and differences between them (and deviances from the standard) – img, script, css, etc.
  • The REST Statelessness Constraint – a nice post about statelessness in REST if you, like me, don’t know REST so much in depth; highlights: Statelesness (and thus the requirement for clients to send their state with every request) is a trade-off crucial for web-scale and partially balanced by caching – while typical enterprise apps have different needs (more state, less scale) so REST isn’t a perfect match. Distinguish application (client-side) and server (resources) state. Using a DB to hold the state still violates the requirement. Use links to transfer some state (e.g. contain a link to fetch the next page of records in the response).
  • Functional Programming in Javascript – an interactive tutorial teaching map, filter, mergeAll, reduce, zip

Other

  • CodeMesh 2013 presentations – good stuff! F.ex. Refactoring Functional Programs: Past and Future, Distribution, Scale and Flexibility with ZeroMQ, Deepak Giridharagopal on Puppet, Immutable Deployments, Analyzing Systems with PuppetDB, Francesco Cesarini and Viktor Klang on the Reactive Manifesto and more
  • Cognitive Biases in Times of Uncertainty – people under pressure/stress start to focus on risks over gains and (very) short-term rather than long-term and thus also adopt 0-some mindset (i.e. if sb. else wins, I loose) => polarization into we x them and focus on getting as big piece of the cake possible at any price, now, dismissal of collaboration. With accelerating rate of change in the society due to technology, this is exactly what is happening. How to counter it? Create more positive narratives than the threat-based ones (views of the world), support them via short-term gains. Bottom line: each of us must work on spreading a more positive attitude to save us from bleak future.
  • Book – Nathan Marz: Big Data – I dislike the big data hype (and, with passion, Hadoop) but would love to read this book; it presents a fresh look at big data processing, heavily inspired by functional programming. Nathan has plenty of experiences from Twitter and creating Storm and Cascalog (both in Clojure, btw.). Read ch 1:  A new paradigm for big data.
  • Facebook Engineering: The Mature Optimization Handbook (or go directly to the pdf,   ePubMobi). If you get bored, jump directly to ch 5. Instrumentation.

Clojure Corner

  • Schmetterling – Debug running clojure processes from the browser! – upon an exception, the process will pause and S. will show the stack, which you can navigate and see locals and run code in the context of any stack frame; you can also trigger it from your code
  • Gorilla REPL (screenshot, 11min video)- interactive web-based notebook where you can mix text (with Markdown formatting), mathematical formulas via LaTeX, graphs, tables, Clojure code. Great for exploring and, at the same time, describing data. <3
  • Local state is harmful – how can we answer the questions about when/why did state X change, how did output Y get where it is? Make state explicit, f.ex. one global map holding all of it, and perhaps not just the current state but also history – thus we can easily query it. Prismatic’ Graph can be used to make the state map, watches to keep history. Inspired by databases (Datomic is an excellent example of SW where answering such questions is trivial)
  • S. Corfield: Insanely Useful Leiningen Plugins – lein-ancient (find updated deps), lein-exec (execute Clj from cmd.line / scripts in Clj), lein-try (try a lib in REPL), Eastwood – a lint tool for Clojure
  • Sente – Clojure(Script) + core.async + WebSockets/Ajax – a tiny 600 LoC library for websockets (with fall-back to long-polling) communication between ClojureScript frontend and clojure backend, using EDN, support for request-reply and multiple user windows/tabs (comparison with Chord (no non-WS fallback or req/resp))
  • Nicholas Kariniemi: Why is Clojure bootstrapping so slow? – don’t blame the JVM, most time spent in clojure.core according to this analyzes on JVM and Android (create and set vars, load other namespaces); some proposals for improving it – lazy loading, excluding functionality not used, …
  • Cheat your way to running CLJS on Node – (ab)use D. Nolen’s mies template intended for client-side cljs development to create a Node project; the trick: compile everything into 1 file so that Node does not fail to find dependencies, disable source maps etc. Update: the nodecljs template now does this
  • lt-clojure-tutorial – A Clojure tutorial optimized for Light Table, ported from Nolen’s cljs tutorial

Tools/Libs

Posted in General, Languages, Tools, Top links of month | Tagged: , , , , , , , , | Comments Off

Most interesting links of December ’13

Posted by Jakub Holý on December 31, 2013

Recommended Readings

Society

  • HBR: Want to Build Resilience? Kill the Complexity – a highly interesting, thought provoking article relevant both to technology in particular and the society in general; f.ex.: more security features are bad for they make us behave less safely (risk compensation) and are more fragile w.r.t. unexpected events. “Complexity is a clear and present danger to both firms and the global financial system: it makes both much harder to manage, govern, audit, regulate and support effectively in times of crisis. [..] Combine complex, Robust-Yet-Fragile systems, risk-compensating human psyches, and risk-homeostatic organizational cultures, and you inevitably get catastrophes of all kinds: meltdowns, economic crises, and the like.” The solution to future financial crisis is primarily not more regulation but simplification of the system – to make it easier to police, tougher to game. We also need to decrease interconnectednes (of banks etc.), one of the primary sources of complexity. Also a great example of US Army combatting complex, high-risk situations by employing “devil’s advocates / professional skeptics” trained to help “avoid the perils of overconfidence, strategic brittleness, and groupthink. The goal is to respectfully help leaders in complex situations unearth untested assumptions, consider alternative interpretations and “think like the other”“.
  • The Dark Side of Technology – technologies provide great opportunities – but also risks we should be aware of – they create a world of mounting performance pressure for all of us (individuals, companies, states), accelerate the rate of change, increasing uncertanity (=> risk of Taleb’s black swans). “All of this mounting pressure has an understandable but very dangerous consequence. It draws out and intensifies certain cognitive biases [..]” – magnify our perception of risk, shrink our time horizons, foster a more and more reactive approach to the world, the “if you win, I will lose” view, erode our ability to trust anyone – and “combined effect of these cognitive biases increases the temptation to use these new digital infrastructures in a dysfunctional way: surveillance and control in all aspects of our economic, social and political life.” => “significantly increase[d] the likelihood of an economic, social and political backlash, driven by an unholy alliance between those who have power today and those who have achieved some modest degree of income and success.
    Complexity theory: the more connected a system is, the more vulnerable it becomes to cascades of disruptive information/action.
  • What Do Government Agencies Have Against 23andMe, Uber, and Airbnb? – innovative startups do not fit into established rules and thus bureaucrats do not know how to handle them and resort to their favourite weapon: saying no, i.e. enforcing rules that harm them (f.ex. France recently passed a law that requires Uber etc. drivers to wait 15 min before picking up a customer so that established taxi services have it easier; wot?!)
  • Nonviolent communication in action – wonderful stories about NVC being applied in difficult situations with a great success

Tech

  • D. Nolen: The Future of JavaScript MVC Frameworks – highly recommended thought food – about React.js, disadvantages of event-based UI, benefits of immutability, performance, the ClojureScript React wrapper Om  – “I simply don’t believe in event oriented MVC systems – the flame graph above says it all. [...] Hopefully this gives fans of the current crop of JS MVCs and even people who believe in just using plain JavaScript and jQuery some food for thought. I’ve shown that a compile to JavaScript language that uses slower data structures ends up faster than a reasonably fast competitor for rich user interfaces. To top it off Om TodoMVC with the same bells and whistles as everyone else weighs in at ~260 lines of code
  • Quora: Michael Wolfe’s answer to Engineering Management: Why are software development task estimations regularly off by a factor of 2-3? – a wonderful story explaining to a layman why estimation is hard, on the example of a hike from SF to LA
  • Style Guide for JavaScript/Node.js by Felix Geisendörfer, recommended by a respectable web dev; nothing groudn breaking I suppose but great start for a team’s standards
  • Johannes Brodwall: Why I stopped using Spring [IoC] – worth to read criticism of Spring by a respected and experienced architect and developer; summary – dependency injection is good bug “magical” frameworks decrease understandability and encourage unnecessarily complex code => smaller code, , easier to navigate and understand and easier to test
  • Misunderstanding technical debt – a brief discussion of the various forms of tech. debt (crappy code x misaligned design and problem domain x competence debt)
  • Tension and Flaws Before Health Website Crash – surprising lack of understanding and tensions between the government and contractors on HealthCare.gov – “a huge gap between the administration’s grand hopes and the practicalities of building a website that could function on opening day” – also terribly decision making, shifting requirements (what news!), management’s lack of decision power, CGI’s blame-shifting. A nice horror story. The former head knew that they should “greatly simplify the site’s functions” – but the current head wasn’t able to “talk them out of it”.
  • The Log: What every software engineer should know about real-time data’s unifying abstraction – logs are everywhere and especially important in distributed apps – DB logs, append-only logs, transaction logs – “You can’t fully understand databases, NoSQL stores, key value stores, replication, paxos, hadoop, version control, or almost any software system without understanding logs” – I have only read a small part but it looks useful
  • What I Wish I Knew When Learning Haskell tl;dr
  • Better Than Unit Tests – a good overview of testing approaches beyond unit tests – including “Automated Contract Testing” (ability to define a contract for a web service, use it to test it and to simulate it; see Internet of Strings for more info), Property-based Testing (test generic properties using random data/calls as with Quickcheck), Fault Injection (run on multiple VMs, simulate network failures), Simulation Testing as with Simulant.
  • Use #NoEstimates to create options and deliver value reliably – a brief post with an example of an estimation-based vs. no-estimates project (i.e. more focus on delivering early, discovery)
  • How Google Sold Its Engineers on Management – managers may be useful after all :-); a report about Google’s research into management and subsequent (sometimes radical) improvements in management style/skills and people satisfaction; I love that Google hasn’t HR but “people ops”
  • Roy Osherove: Technical Disobedience – take nothing for granted, don’t let the system/process stop you, be creative about finding ways to improve your team’s productivity; there always is a way. Nice examples.
  • Uncle Bob: Extreme Programming, a Reflection – a reflection on changes in the past ~ 14 years since XP that have seen many of the “extreme” practices becoming mainstream
  • The Anti-Meeting Manifesto – essentially a checklist and tips for limitting meetings to minimum

Other

Talks

  • Pete Hunt: React: Rethinking best practices (JSConf 2013, 30 min) – one of the most interesting talks about frontend development, design, and performance I have heard this year, highly recommended. Facebook’s React JavaScript framework  is a fresh and innovative challenger in the MVC field. It is worthwile to learn why they parted ways with the popular approach of templates (spoiler: concern separation, cohesion x coupling, performance). Their approach with virtual DOM enables some cool things (run in Node, provide HTML5-like events in any browser with consistent behavior, …). Key: templates are actually tightly coupled to display logic (controllers) via the model view tailored for them (i.e. Controller must know what specific data & in what form View needs) => follow cohesion and keep them together componets, separate from other components and back-end code. Also, state changing over time at many places is hard => re-render the whole app rather than in-place updates. Also, the ClojureScript Om wrapper enables even more performance optimizations thanks to immutable data structures etc.
  • David Pollak: Some musings on Scala and Clojure by a long time Scala dude (46 min) – a subjective but balanced comparison of Scala and Clojure and their strengths/weaknesses by the author of the Scala Lift framework (doing Scala since 2006, Clojure since 2013)

Clojure Corner

Tools/Libs

  • Apache Sirona – a new monitoring tool in the Apache incubator – “a simple but extensible monitoring solution for Java applications” with support for HTTP, JDBC, JAX-RS, CDI, ehcache, with data published e.g. to Graphite or Square Cube. It is still very new.
  • GenieJS – Ctrl-Space to popup a command-prompt for your web page, inspired by Alfred (type ‘ to see all possible commands)

Favourite Quotes

A good #agile team considers their backlog inaccurate. It is merely a list of assumptions that must be tested & refined by shipping product
- @mick maguire 12/10

Ada Lovelace (1st program), Grace Hopper (1st compiler), Adele Goldberg (1st OO language), why would anyone think women aren’t in computing?
@Dan North 12/11

There will always be a shortage of talented, self-motivated creative professionals who will unquestioningly follow orders.
@Thomas K Nilsson 12/7

Estimation paradox = If something unpredictable happens, predict how long it will take to fix it
me 12/7

IT systems can be inspired by AK-47 a.k.a. Kalashnikov. The rifle was purposefully designed to be simple and to be tolerant to imperfections in most parts; as a result, it required essentially no maintenance and was extremely reliable.
– summarized from Roman Pichlík’s Odkaz Michaila Kalašnikova softwarovému vývoji

Posted in General, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , , , , | Comments Off

Most interesting links of November ’13

Posted by Jakub Holý on November 30, 2013

Recommended Readings

Some interesting topics this time despite me spending lot of time on the Principles of Reactive Programming course: Java x Node.js, REST x other future-proof architectures, scary legacy code. Of course, also plenty of Clojure.

People, organizations, teams, development:

  • Chris Argyris (1923-2013): An Appreciation – Thinkers 50 – recently departed Ch. Argyris is a person whose work you should know, if a bit interested in learning and organizations and how they (dis)function; and since we all work in organizations and want our work to be pleasant, this means all of us. We all want to work in orgs that do double-loop learning, i.e. they actually evovle as they learn. “Argyris argued that organizations depend fundamentally on people and that personal development is and can be related to work.” Now stop and go read it!
  • Bob Marshall: The Antimatter Principle – “the only principle we need for becoming wildly effective at collaborative knowledge work” – can be summarized as “attend to folks’ needs” (importantly, including your own) => find out what people actually need, interpret their behavior (including anger, seemingly irrational or stupid requests etc.) in terms of needs; mastering this will make you excell in communication and effective work. Read the post to find out more.
  • It’s a state of mind: some practical indicators on doing agile vs. being agile – are you agile or are just “doing agile”? Read on ti find out, if you dare! F.ex. “Non Agile teams have a process that slows the review of the changes.” Cocnlusion: “An Agile mindset is just that – a state of mind, a set of values. Its a constant questioning, and an opening up to possibilities. Its a predisposition to produce great things.
  • Johannes Brodwall:  Humble architects – how to avoid being an architect that does more harm than good, as so many out there? Some tips: Don’t assume stupidity, Be aware that you might be wrong, Be careful with technology (i.e. simplicity beats everything; applies so much to us developers too!), Consistency isn’t as important as you think (or beware context), Tactical reuse in across systems is suboptimization (i.e. reuse has a cost), separate between (coding) rules and dogma (i.e. is that way unsafe, incomprehensible, or just a heresy w.r.t. a dogma?) Very valuable insights into creating good technical solutions and teams that work.
  • Liz Keogh’s The Dream Team Nightmare: a Review – a very good review of this adventure-style book about coaching “agile” teams through (around?) common pitfalls, provides a good base for deciding whether you shall read the book (Liz essentially says yes)
  • Fibonacci Kittens: One Idea One Commit – a short story of coming from biannual releases to frequent release of individual features; I link to this primarily to spread optimism, if this company managed it then, perhaps, we other can too?
  • The Eternal Struggle Between Business and Programmers – “Business: More features! Now! Programmers: More refactoring! Now!” How can we resolve this eternal conflict of needs? This post reveals how the two parties can find a common ground and mutual understanding (beware, everybody must give up on something) and thus work together rather than against each other.

Coding, architecture, legacy

  • Why the future is NOT RESTful – always refreshing to read something against the mainstream; “REST is not fit for the next generation of smart client applications because it has not been designed for smart clients.” According to the author, a smart client app stack needs: “1. persistence (storage and query), 2. documents/orm (conversion to tree-like datastructures), 3. data authorization (once authenticated), 4. pub/sub (websocket communications), 5. client db (client-side caching and querying), 6. templating (presentation level)” Meteor.js has nearly all but #3 thanks to mongodb (1+2), dpp (4), mongo on the client (5), spark (6). The author considers a similar but Clojure-based stack (with Datomic, Angular etc.) and looks at authorization possibilities. “Secured, personalised, CRUD operations are the future to a more simplified web.” We may agree or not but it certainly is worth reading.
  • Michael Feathers (of Working Effectively With Legacy fame): Unconditional Programming – “Over and over again, I find that better code has fewer if-statements, fewer switches, and fewer loops. Often this happens because developers are using languages with better abstractions. [..] The problem with control structures is that they often make it easy to modify code in bad ways.” Includes a nice example of replacing if-else with much more readable and safer code.
  • The Quality of Embedded Software, or the Mess Has Happened – an interesting and scary read about terrible spaghetti code (and hardware) that makes some Toyotas to accelerate when the driver tries to break; 11,000 global variables, the key function showing so high cyclomatix complexity that “makes it impossible not only to test but even maintain this program in any way.” Then 80k violations of the car industry coding standard cannot surprise. And a safety control that does not work. Interesting that a great manufacturer may have so terrible IT (and Toyota isn’t the only one).
  • The string type is broken – the String type is M. Feathers’ favourite example of a leaky abstraction – most languages fail to process/split/count less common Unicode characters properly, the fact that String is implemented as a series of bytes leaks through (UTF-16 langs like Java); worth reading to be aware of the limitations

Languages

  • Why I’m Productive In Clojure – some interesting points about simplicity, consciousness, interactive development, power without overwhelming fatures, etc. “With it [Clojure] I can always easily derive a solution to a particular problem from a small set of general patterns. [..] However, the number of ways that these concepts can be combined to solve all manner of problems appears to be inexhaustible.
  • Node.js at PayPal – PayPal is switching from Java to Node.js (among others to promote language consistency) and, as a part of that, has implemented the same app in Node and Java; results: Node was done earlier, had less code, performed better (though, as Daniel Kvasnička pointed out, “Comparing Node.js and servlet-based archs is not fair… compare Node with @vertx_project and you’ll get a whole different story ;)”; also, as Charles Nutter said, “The @PayPal numbers for their Java to Node move are absurd. A JVM app doing 1.8 pages/s isn’t slow…it’s broken.“)
  • IBM: Developing mobile apps with Node.js and MongoDB, Part 1: A team’s methods and results – also IBM has implemented the same (REST) app once with Java and DB2, once with Node and Mongo where Node+Mongo required less work and performed better; one of the great wins was having JSON as a native structure everywhere instead of transforming from/to it so Mongo is, in my opinion, an important factor in this particular case
  • Dynamics of Programming: Benefits of Scala in CS1 – reasons for and experiences with using Scala in an introductory computer science course, worth reading; some of the advantages over Java are consistency (.asInt on String and Double vs. casting/parsing, no “primitive” types), REPL with time inference good for learning, functional style enables focus on what rather than how; quite persuasive arguments

Security

  • The New Threat: Targeted Internet Traffic Misdirection – did you know that internet traffic to any site can be made to go through a particular server without anybody noticing? This has been observed repeatedly in the wild, for banks and other sites. Rather make sure you use strong encryption (NSA-approved, of course ;)).
  • A (relatively easy to understand) primer on elliptic curve cryptography – Everything you wanted to know about the next generation of public key crypto – you cannot just read this, you have to study it, which I did not; but it looks good and I guess the time will come when I will come back to it

Other

Clojure Corner

  • Stuart Sierra’s Component – a library for making it easier to implement Stuart’s reloadable workflow; a component is something that can be started, stopped, and depend on other components so that it is easier to do interactive REPL development
  • Logan Linn: Clojure/conj 2013 – a pretty good overview of the conference
  • Caribou – “the kernel of usefulness that has emerged from years of this basic practice“- a new Clojure web framework – seems to be interesting
  • Results of the 2013 State of Clojure & ClojureScript survey and drill-down into what features people want – the most interesting fact is how many more participants use Clojure in production than last year and perhaps also the relatively wide adoption of Datomic among the respondents. Light Table has become the 3rd most popular dev. env., after Emacs and Vim. Some of the most mentioned language features requested were types (=> core.typed, Prismatic’s Schema), better error reporting (=> slingshotdireclj-stacktraceio.aviso:pretty, etc.), debuger (though progress is being made)
  • Book: Clojure High Performance Programming
  • Improving Clojure Feedback : Stack Traces – making Clojure stacktraces more usable by filtering out noise and linking to relevant content – io.aviso:pretty,  io.aviso:twixt
  • Clojure Dev discussion: Hashing strategies – Executive summary – “In Clojure, however, it is far more common than in Java to use longs, vectors, sets, maps and compound objects comprised of those components (e.g., a map from vectors of longs to sets) as keys in other hash maps.  It appears that Java’s hash strategy is not well-tuned for this kind of usage.  Clojure’s hashing for longs, vectors, sets, and maps each suffer from some weaknesses that can multiply together to create a crippling number of collisions.” Ex.: An implementation of N-Queens took forever, spending most time on hash collisions. But be calm, smart Clojurians are working on a solution.
  • Datomic Pro Starter Edition – Datomic with all storages, Datomic Console, a year of updates, full Datomic programming model; limitations: no HA transactor, no integrated memcached, max 2 peers and 1 transactor

Tools/Libs

  • AirPair.com – a new site that enables developers to get help from other devs via remote pairing, code review, code mentoring etc. – a good opportunity to get help / help others (and earn something); I haven’t tried it but it sounds pretty interesting

MongoDB web stacks

  • Meteor: JS frontend + MongoDB backend with changes in the DB pushed live to the clients, i.e. MongoDB is used both as the “application server” and storage. It seems great for apps where users need to collaborate in real-time with each other, certainly great for quick proof of concepts etc.; worth checking out; it also comes with free (at least for start?) hosting so really good for prototyping – “an open-source platform for building top-quality web apps in a fraction of the time.” The intro screencast will give you a good overview (10 min).
  • Mean.io – MEAN (Mongo, Express, Angular, Node) stack Boilerplate – frontend, backend and storage using the same language and some of the most popular technologies (not that popular = best fit for you :)); it seems to be very new but since it just glues together 4 popular and documented technologies, that should not be an obstacle. There is an intro on the MongoDB blog.

Other

  • BusyBox (get latest win binary from Tigress.co.uk) – reportedly a better POSIX env for Windows than gow, Cygwin, et al.

Favourite Quotes

[..] no organization should exist unless it is “of service” to its employees, its customers, its community.
- @Tom_Peters 28/11

I hope you’ll agree that there is a certain amount of irony involved in having to write repetitive code
Dmitri Sotnikov in Why I’m Productive In Clojure

Happy teams are productive teams but:

Morale is 95% a function of the prevailing system (the way the work works). Which in turn is a function of the prevailing collective mindset
@flowchainsensei Nov 10th

Posted in General, Languages, Top links of month | Tagged: , , , , , , , , , , | Comments Off