Experience: Awesome productivity with ClojureScript’s REPL

Re-posted from Telia’s tech blog.

What’s the deal with ClojureScript? How can you justify picking such a “niche” language? I have recently experienced a “wow” session, demonstrating the productivity gains of ClojureScript and the interactive development it enables thanks to its REPL. I would like to share the experience with you. (If you have never heard about it before – it is a modern, very well designed Lisp that compiles to JavaScript for frontend and backend development. It comes with a REPL that makes it possible to reload code changes and run code in the context of your live application, developing it while it is running.)

Advertisements

Most interesting links of April ’14

Recommended Readings

• The economics of reuse – developing code for reuse costs much more than for one need – it might cost 300% more to develop and save you 75% of work when (re)using it instead of developing from scratch (if one of the factors goes down, the other one typically goes down too). Summary: “That means that to get any value from your reused component, you better have five or more reusers or you have to find a way to substantially improve the [reuse value factor] or [reusability cost factor]. Very smart people have failed to do this.“
• Book in making: Reactive Design Patterns (1st ch free)

Sharing data on the web

• GoodRelations: The Web Vocabulary for E-Commerce – make data about your company, products, opening hours etc. more SEO-friendly and understandable to a computer by adding some semantic markup. If more sites did this, the web would be much easier to navigate. (GoodRelations has been added almost entirely to the schema.org vocabulary so you can use just schema.org with microdata or schema.org where possible and G.R. to extend it.)

Clojure Corner

• 8th Light: Combining Clojure and ClojureScript Libraries (3/2014) – really good and detailed article / tutorial using CLJX and platform-specific platform.clj[s] files to share code between Clojure and ClojureScript. It also recommends a file structure (src/(clj|cljs)/), demonstrates testing, discusses macro development, shows how to pack both into one jar.

Tools/Libs

Favourite Quotes

HttpServletRequest: requestURI/requestURL/contextPath/servletPath/pathInfo/queryString

I never remember what some of these HttpServletRequest methods return so here is an example:

URL METHODS BREAKDOWN (JavaEE 6):
- requestURL  = http://localhost:8080/myapp/users/profile.xhtml
- requestURI  = /myapp/users/profile.xhtml = contextPath + servletPath + pathInfo
- contextPath = /myapp = the first segment after hostname (unless the app runs and the root app with context /)
- servletPath = /users/profile.xhtml (part after contextPath to the servlet that handled the request)
- pathInfo    = null (what remains after servletPath up to the queryString; would return '/dummy' if the url was '.../profile.xhtml/dummy?id=007')
- queryString = id=007

Most interesting links of February ’14

Recommended Readings

Development

• Falsehoods Programmers Believe About Names – summary: there are no rules that apply to names, do not assume anything (my favourite: 12 + 13)

• Nathan Marz: Principles of Software Engineering, Part 1 – Nathan has worked with Big Data at Twitter and other places and really knows the perils or large, distributed, real-time systems and this post contains plenty of valuable advice for making robust, reliable SW. Main message: “there’s a lot of uncertainty in software engineering“; every SW operates correctly only for a certain range of inputs (including volume, HW it runs on, …) and you never control all of them so there always is an opportunity for failure; you can’t predict what inputs you will encounter in the wild. “[..] while software is deterministic, you can’t treat it as deterministic in any sort of practical sense if you want to build robust software.” “Making software robust is an iterative process: you build and test it as best you can, but inevitably in production you’ll discover new areas of the input space that lead to failure. Like rockets, it’s crucial to have excellent monitoring in place so that these issues can be diagnosed.“. From the content: Sources of uncertainty (bugs, humans, requirements, inputs, ..), Engineering for uncertainty (minimize dependencies, lessen % of cascading failure [JH: -> Hystrix], measure and monitor)
  • Suffering-oriented programming is certainly also worth reading (summary: do not start with great designs; only start generalizing and creating libs when you have suffered enough from doing things more manually and thus learned the domain; make it possible > make it beautiful > make it fast, repeat)
• ThoughtWorks open-sources Go, continuous delivery platform – good bye, Jenkins! – better support for pipelines etc., see features and elementary concepts
• Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications (recommended by @markusbk so it must be good); Patterns: Cache-aside, Circuit Breaker, Compensating Transaction, Competing Consumers, Compute Resource Consolidation, Command and Query Responsibility Segregation (CQRS), Event Sourcing, External Configuration Store, Federated Identity, Gatekeeper, Health Endpoint Monitoring, Index Table, Leader Election, Materialized View, Pipes and Filters, Priority Queue, Queue-based Load Leveling, Retry, Runtime Reconfiguration, Scheduler Agent Supervisor, (data) Sharding, Static Content Hosting (-> CDN), Throttling, Valet Key.
  Guidance topics: Asynchronous Messaging Primer, Autoscaling, Caching, Compute Partitioning, Data Consistency Primer, Data Partitioning, Data Replication and Synchronization, Instrumentation and Telemetry, Multiple Datacenter Deployment, Service Metering
• MOOC course Functional programming with Clojure at Uni of Helsinki – to get started you need, I suppose, follow the “Material and course content” – essentially read the text for each chapter, clone its repo, submit pull requests to get your work graded
• Jez Humble: The Case for Continuous Delivery – read to persuade manager about CD: “Still, many managers and executives remain unconvinced as to the benefits [of CD], and would like to know more about the economic drivers behind CD.” CD reduces waste: “[..]online controlled experiments (A/B tests) at Amazon. This approach created hundreds of millions of dollars of value[..],” reduces risks: “[..] Etsy, has a great presentation which describes how deploying more frequently improves the stability of web services.” CD makes development cheaper by reducing the cost of non-value-adding activities such as integration and testing. F.ex. HP got dev. costs down by 40%, dev cost/program by 78%

Web

• Client-side messaging in JavaScript – Part 3 (anti-patterns) (via @ruudud so it must be worth reading)
• Request Quest (via @jraregris) – entertaining and educational intractive quiz regarding what does (not) trigger a request in browsers and differences between them (and deviances from the standard) – img, script, css, etc.
• The REST Statelessness Constraint – a nice post about statelessness in REST if you, like me, don’t know REST so much in depth; highlights: Statelesness (and thus the requirement for clients to send their state with every request) is a trade-off crucial for web-scale and partially balanced by caching – while typical enterprise apps have different needs (more state, less scale) so REST isn’t a perfect match. Distinguish application (client-side) and server (resources) state. Using a DB to hold the state still violates the requirement. Use links to transfer some state (e.g. contain a link to fetch the next page of records in the response).
• Functional Programming in Javascript – an interactive tutorial teaching map, filter, mergeAll, reduce, zip

Other

• CodeMesh 2013 presentations – good stuff! F.ex. Refactoring Functional Programs: Past and Future, Distribution, Scale and Flexibility with ZeroMQ, Deepak Giridharagopal on Puppet, Immutable Deployments, Analyzing Systems with PuppetDB, Francesco Cesarini and Viktor Klang on the Reactive Manifesto and more
• Cognitive Biases in Times of Uncertainty – people under pressure/stress start to focus on risks over gains and (very) short-term rather than long-term and thus also adopt 0-some mindset (i.e. if sb. else wins, I loose) => polarization into we x them and focus on getting as big piece of the cake possible at any price, now, dismissal of collaboration. With accelerating rate of change in the society due to technology, this is exactly what is happening. How to counter it? Create more positive narratives than the threat-based ones (views of the world), support them via short-term gains. Bottom line: each of us must work on spreading a more positive attitude to save us from bleak future.
• Book – Nathan Marz: Big Data – I dislike the big data hype (and, with passion, Hadoop) but would love to read this book; it presents a fresh look at big data processing, heavily inspired by functional programming. Nathan has plenty of experiences from Twitter and creating Storm and Cascalog (both in Clojure, btw.). Read ch 1:  A new paradigm for big data.
• Facebook Engineering: The Mature Optimization Handbook (or go directly to the pdf,   ePub, Mobi). If you get bored, jump directly to ch 5. Instrumentation.

Clojure Corner

• Schmetterling – Debug running clojure processes from the browser! – upon an exception, the process will pause and S. will show the stack, which you can navigate and see locals and run code in the context of any stack frame; you can also trigger it from your code
• Gorilla REPL (screenshot, 11min video)- interactive web-based notebook where you can mix text (with Markdown formatting), mathematical formulas via LaTeX, graphs, tables, Clojure code. Great for exploring and, at the same time, describing data. ❤
• Local state is harmful – how can we answer the questions about when/why did state X change, how did output Y get where it is? Make state explicit, f.ex. one global map holding all of it, and perhaps not just the current state but also history – thus we can easily query it. Prismatic’ Graph can be used to make the state map, watches to keep history. Inspired by databases (Datomic is an excellent example of SW where answering such questions is trivial)
• S. Corfield: Insanely Useful Leiningen Plugins – lein-ancient (find updated deps), lein-exec (execute Clj from cmd.line / scripts in Clj), lein-try (try a lib in REPL), Eastwood – a lint tool for Clojure
• Sente – Clojure(Script) + core.async + WebSockets/Ajax – a tiny 600 LoC library for websockets (with fall-back to long-polling) communication between ClojureScript frontend and clojure backend, using EDN, support for request-reply and multiple user windows/tabs (comparison with Chord (no non-WS fallback or req/resp))
• Nicholas Kariniemi: Why is Clojure bootstrapping so slow? – don’t blame the JVM, most time spent in clojure.core according to this analyzes on JVM and Android (create and set vars, load other namespaces); some proposals for improving it – lazy loading, excluding functionality not used, …
• Cheat your way to running CLJS on Node – (ab)use D. Nolen’s mies template intended for client-side cljs development to create a Node project; the trick: compile everything into 1 file so that Node does not fail to find dependencies, disable source maps etc. Update: the nodecljs template now does this

• lt-clojure-tutorial – A Clojure tutorial optimized for Light Table, ported from Nolen’s cljs tutorial

Tools/Libs

• Charles – web debugging proxy application (via @ruudud) – to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet (shareware)

Most interesting links of December ’13

Recommended Readings

Society

• HBR: Want to Build Resilience? Kill the Complexity – a highly interesting, thought provoking article relevant both to technology in particular and the society in general; f.ex.: more security features are bad for they make us behave less safely (risk compensation) and are more fragile w.r.t. unexpected events. “Complexity is a clear and present danger to both firms and the global financial system: it makes both much harder to manage, govern, audit, regulate and support effectively in times of crisis. [..] Combine complex, Robust-Yet-Fragile systems, risk-compensating human psyches, and risk-homeostatic organizational cultures, and you inevitably get catastrophes of all kinds: meltdowns, economic crises, and the like.” The solution to future financial crisis is primarily not more regulation but simplification of the system – to make it easier to police, tougher to game. We also need to decrease interconnectednes (of banks etc.), one of the primary sources of complexity. Also a great example of US Army combatting complex, high-risk situations by employing “devil’s advocates / professional skeptics” trained to help “avoid the perils of overconfidence, strategic brittleness, and groupthink. The goal is to respectfully help leaders in complex situations unearth untested assumptions, consider alternative interpretations and “think like the other”“.
• The Dark Side of Technology – technologies provide great opportunities – but also risks we should be aware of – they create a world of mounting performance pressure for all of us (individuals, companies, states), accelerate the rate of change, increasing uncertanity (=> risk of Taleb’s black swans). “All of this mounting pressure has an understandable but very dangerous consequence. It draws out and intensifies certain cognitive biases [..]” – magnify our perception of risk, shrink our time horizons, foster a more and more reactive approach to the world, the “if you win, I will lose” view, erode our ability to trust anyone – and “combined effect of these cognitive biases increases the temptation to use these new digital infrastructures in a dysfunctional way: surveillance and control in all aspects of our economic, social and political life.” => “significantly increase[d] the likelihood of an economic, social and political backlash, driven by an unholy alliance between those who have power today and those who have achieved some modest degree of income and success.”
  Complexity theory: the more connected a system is, the more vulnerable it becomes to cascades of disruptive information/action.
• What Do Government Agencies Have Against 23andMe, Uber, and Airbnb? – innovative startups do not fit into established rules and thus bureaucrats do not know how to handle them and resort to their favourite weapon: saying no, i.e. enforcing rules that harm them (f.ex. France recently passed a law that requires Uber etc. drivers to wait 15 min before picking up a customer so that established taxi services have it easier; wot?!)
• Nonviolent communication in action – wonderful stories about NVC being applied in difficult situations with a great success

Tech

• D. Nolen: The Future of JavaScript MVC Frameworks – highly recommended thought food – about React.js, disadvantages of event-based UI, benefits of immutability, performance, the ClojureScript React wrapper Om  – “I simply don’t believe in event oriented MVC systems – the flame graph above says it all. […] Hopefully this gives fans of the current crop of JS MVCs and even people who believe in just using plain JavaScript and jQuery some food for thought. I’ve shown that a compile to JavaScript language that uses slower data structures ends up faster than a reasonably fast competitor for rich user interfaces. To top it off Om TodoMVC with the same bells and whistles as everyone else weighs in at ~260 lines of code“
• Quora: Michael Wolfe’s answer to Engineering Management: Why are software development task estimations regularly off by a factor of 2-3? – a wonderful story explaining to a layman why estimation is hard, on the example of a hike from SF to LA
• Style Guide for JavaScript/Node.js by Felix Geisendörfer, recommended by a respectable web dev; nothing groudn breaking I suppose but great start for a team’s standards
• Johannes Brodwall: Why I stopped using Spring [IoC] – worth to read criticism of Spring by a respected and experienced architect and developer; summary – dependency injection is good bug “magical” frameworks decrease understandability and encourage unnecessarily complex code => smaller code, , easier to navigate and understand and easier to test
• Misunderstanding technical debt – a brief discussion of the various forms of tech. debt (crappy code x misaligned design and problem domain x competence debt)
• Tension and Flaws Before Health Website Crash – surprising lack of understanding and tensions between the government and contractors on HealthCare.gov – “a huge gap between the administration’s grand hopes and the practicalities of building a website that could function on opening day” – also terribly decision making, shifting requirements (what news!), management’s lack of decision power, CGI’s blame-shifting. A nice horror story. The former head knew that they should “greatly simplify the site’s functions” – but the current head wasn’t able to “talk them out of it”.
• The Log: What every software engineer should know about real-time data’s unifying abstraction – logs are everywhere and especially important in distributed apps – DB logs, append-only logs, transaction logs – “You can’t fully understand databases, NoSQL stores, key value stores, replication, paxos, hadoop, version control, or almost any software system without understanding logs” – I have only read a small part but it looks useful
• What I Wish I Knew When Learning Haskell tl;dr
• Better Than Unit Tests – a good overview of testing approaches beyond unit tests – including “Automated Contract Testing” (ability to define a contract for a web service, use it to test it and to simulate it; see Internet of Strings for more info), Property-based Testing (test generic properties using random data/calls as with Quickcheck), Fault Injection (run on multiple VMs, simulate network failures), Simulation Testing as with Simulant.
• Use #NoEstimates to create options and deliver value reliably – a brief post with an example of an estimation-based vs. no-estimates project (i.e. more focus on delivering early, discovery)
• How Google Sold Its Engineers on Management – managers may be useful after all :-); a report about Google’s research into management and subsequent (sometimes radical) improvements in management style/skills and people satisfaction; I love that Google hasn’t HR but “people ops”
• Roy Osherove: Technical Disobedience – take nothing for granted, don’t let the system/process stop you, be creative about finding ways to improve your team’s productivity; there always is a way. Nice examples.
• Uncle Bob: Extreme Programming, a Reflection – a reflection on changes in the past ~ 14 years since XP that have seen many of the “extreme” practices becoming mainstream
• The Anti-Meeting Manifesto – essentially a checklist and tips for limitting meetings to minimum

Other

• Ethics Daily – a freuently published collection of ethics-related links, articles, talks etc.
• British Library uploads one million public domain images to the net for remix and reuse – from 17th-19th century books, available at Flicker; it asks for help categorizing and documenting them, metadata of the images at GitHub (with links to Flicker); great project, interesting pics
• The Software Engineers’ Oath – lets bring ethics back to our daily (work) lives – respect the knowledge of others, use technology for good (!!!), keep learning, writing code for people, not ashamed to admit lack of knowledge, respect for privacy, obligation to make lifes of humans better, …

Talks

• Pete Hunt: React: Rethinking best practices (JSConf 2013, 30 min) – one of the most interesting talks about frontend development, design, and performance I have heard this year, highly recommended. Facebook’s React JavaScript framework  is a fresh and innovative challenger in the MVC field. It is worthwile to learn why they parted ways with the popular approach of templates (spoiler: concern separation, cohesion x coupling, performance). Their approach with virtual DOM enables some cool things (run in Node, provide HTML5-like events in any browser with consistent behavior, …). Key: templates are actually tightly coupled to display logic (controllers) via the model view tailored for them (i.e. Controller must know what specific data & in what form View needs) => follow cohesion and keep them together componets, separate from other components and back-end code. Also, state changing over time at many places is hard => re-render the whole app rather than in-place updates. Also, the ClojureScript Om wrapper enables even more performance optimizations thanks to immutable data structures etc.
• David Pollak: Some musings on Scala and Clojure by a long time Scala dude (46 min) – a subjective but balanced comparison of Scala and Clojure and their strengths/weaknesses by the author of the Scala Lift framework (doing Scala since 2006, Clojure since 2013)

Clojure Corner

• Hoplon – a reportedly interesting web framework using ClojureScript and targetting single-page apps, with “a spreadsheet-like dataflow programming environment”
• Peek at the new, simpler Pedestal API using core.async
• Review: Clojure high performance programming – “If you’re looking for a refresher or a primer on the topics discussed, then it’s not a bad place to start. However, if you’re looking for a comprehensive discussion on doing high performance programming with Clojure, you’ll likely be left wanting.“
• Clojure On Emacs – A CIDER Workflow Hack – using Cider’s cider-interactive-eval to execute Clojure code on a keystroke in the REPL, f.ex. to run tests, refresh your env., or print an atom you are actively working with; neat!
• Clojure core.async Channels by Hickey, StrangeLoop 2013 – introduction into the design and rationale for core.async vs. alternative solutions such as actors and RxJava
• Discussion: Good resources on dataflow based programming [w.r.t. Pedestal] – I wasn’t even aware that Pedestal is based on dataflows; the recommended (though highly opinionated) articles were 1. http://my.opera.com/Vorlath/blog/2008/01/06/simple-example-of-the-difference-between-imperative-functional-and-data-flow and 2. http://my.opera.com/Vorlath/blog/2008/07/25/hierarchies-and-equivalence. Also the book Dataflow and Reactive Programming Systems that should ship 4/2014.

Tools/Libs

• Apache Sirona – a new monitoring tool in the Apache incubator – “a simple but extensible monitoring solution for Java applications” with support for HTTP, JDBC, JAX-RS, CDI, ehcache, with data published e.g. to Graphite or Square Cube. It is still very new.
• GenieJS – Ctrl-Space to popup a command-prompt for your web page, inspired by Alfred (type ‘ to see all possible commands)

Favourite Quotes

A good #agile team considers their backlog inaccurate. It is merely a list of assumptions that must be tested & refined by shipping product
– @mick maguire 12/10

Ada Lovelace (1st program), Grace Hopper (1st compiler), Adele Goldberg (1st OO language), why would anyone think women aren’t in computing?
– @Dan North 12/11

There will always be a shortage of talented, self-motivated creative professionals who will unquestioningly follow orders.
– @Thomas K Nilsson 12/7

Estimation paradox = If something unpredictable happens, predict how long it will take to fix it
– me 12/7

IT systems can be inspired by AK-47 a.k.a. Kalashnikov. The rifle was purposefully designed to be simple and to be tolerant to imperfections in most parts; as a result, it required essentially no maintenance and was extremely reliable.
– summarized from Roman Pichlík’s Odkaz Michaila Kalašnikova softwarovému vývoji

Installing & Troubleshooting Google Analytics 2013 (ga / analytics.js)

Setting up the new Google Universal Analytics (still in beta) is not completely obvious. You normally won’t be able to send events from localhost and it will claim that “Tracking Not Installed.” Here are some tips how to use Analytics from localhost and test it.

Tools for Editor – Browser Integration for Interactive JS/HTML Development

Chrome Development Tools and similar ones are great for interactive, exploratory coding of JavaScript, HTML, and CSS – but the changes aren’t persistent and the tools haven’t the power of a programmer’s editor. I’d like to be able to use a powerful editor yet be able to see changes to JS/HTML/CSS without having to save-[compile]-reload and I want to be able to execute pieces of JS in the context of the browser. Fortunately, there are ways to get at least some of this and it is getting continually better. Let’s see what tools we have now.

These tools usually use either remoting capabilities of the browser or a long-polling connection to the web site, sending and executing JavaScript.