The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘JavaScript’

Most interesting links of February ’14

Posted by Jakub Holý on February 28, 2014

Recommended Readings

Development

  • Nathan Marz: Principles of Software Engineering, Part 1 – Nathan has worked with Big Data at Twitter and other places and really knows the perils or large, distributed, real-time systems and this post contains plenty of valuable advice for making robust, reliable SW. Main message: “there’s a lot of uncertainty in software engineering“; every SW operates correctly only for a certain range of inputs (including volume, HW it runs on, …) and you never control all of them so there always is an opportunity for failure; you can’t predict what inputs you will encounter in the wild. “[..] while software is deterministic, you can’t treat it as deterministic in any sort of practical sense if you want to build robust software.” “Making software robust is an iterative process: you build and test it as best you can, but inevitably in production you’ll discover new areas of the input space that lead to failure. Like rockets, it’s crucial to have excellent monitoring in place so that these issues can be diagnosed.“. From the content: Sources of uncertainty (bugs, humans, requirements, inputs, ..), Engineering for uncertainty (minimize dependencies, lessen % of cascading failure [JH: -> Hystrix], measure and monitor)
    • Suffering-oriented programming is certainly also worth reading (summary: do not start with great designs; only start generalizing and creating libs when you have suffered enough from doing things more manually and thus learned the domain; make it possible > make it beautiful > make it fast, repeat)
  • ThoughtWorks open-sources Go, continuous delivery platform – good bye, Jenkins! – better support for pipelines etc., see features and elementary concepts
  • Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications (recommended by @markusbk so it must be good); Patterns: Cache-aside, Circuit Breaker, Compensating Transaction, Competing Consumers, Compute Resource Consolidation, Command and Query Responsibility Segregation (CQRS), Event Sourcing, External Configuration Store, Federated Identity, Gatekeeper, Health Endpoint Monitoring, Index Table, Leader Election, Materialized View, Pipes and Filters, Priority Queue, Queue-based Load Leveling, Retry, Runtime Reconfiguration, Scheduler Agent Supervisor, (data) Sharding, Static Content Hosting (-> CDN), Throttling, Valet Key.
    Guidance topics: Asynchronous Messaging Primer, Autoscaling, Caching, Compute Partitioning, Data Consistency Primer, Data Partitioning, Data Replication and Synchronization, Instrumentation and Telemetry, Multiple Datacenter Deployment, Service Metering
  • MOOC course Functional programming with Clojure at Uni of Helsinki – to get started you need, I suppose, follow the “Material and course content” – essentially read the text for each chapter, clone its repo, submit pull requests to get your work graded
  • Jez Humble: The Case for Continuous Delivery – read to persuade manager about CD: “Still, many managers and executives remain unconvinced as to the benefits [of CD], and would like to know more about the economic drivers behind CD.” CD reduces waste: “[..]online controlled experiments (A/B tests) at Amazon. This approach created hundreds of millions of dollars of value[..],” reduces risks: “[..] Etsy, has a great presentation which describes how deploying more frequently improves the stability of web services.” CD makes development cheaper by reducing the cost of non-value-adding activities such as integration and testing. F.ex. HP got dev. costs down by 40%, dev cost/program by 78%

Web

  • Client-side messaging in JavaScript – Part 3 (anti-patterns) (via @ruudud so it must be worth reading)
  • Request Quest (via @jraregris) – entertaining and educational intractive quiz regarding what does (not) trigger a request in browsers and differences between them (and deviances from the standard) – img, script, css, etc.
  • The REST Statelessness Constraint – a nice post about statelessness in REST if you, like me, don’t know REST so much in depth; highlights: Statelesness (and thus the requirement for clients to send their state with every request) is a trade-off crucial for web-scale and partially balanced by caching – while typical enterprise apps have different needs (more state, less scale) so REST isn’t a perfect match. Distinguish application (client-side) and server (resources) state. Using a DB to hold the state still violates the requirement. Use links to transfer some state (e.g. contain a link to fetch the next page of records in the response).
  • Functional Programming in Javascript – an interactive tutorial teaching map, filter, mergeAll, reduce, zip

Other

  • CodeMesh 2013 presentations – good stuff! F.ex. Refactoring Functional Programs: Past and Future, Distribution, Scale and Flexibility with ZeroMQ, Deepak Giridharagopal on Puppet, Immutable Deployments, Analyzing Systems with PuppetDB, Francesco Cesarini and Viktor Klang on the Reactive Manifesto and more
  • Cognitive Biases in Times of Uncertainty – people under pressure/stress start to focus on risks over gains and (very) short-term rather than long-term and thus also adopt 0-some mindset (i.e. if sb. else wins, I loose) => polarization into we x them and focus on getting as big piece of the cake possible at any price, now, dismissal of collaboration. With accelerating rate of change in the society due to technology, this is exactly what is happening. How to counter it? Create more positive narratives than the threat-based ones (views of the world), support them via short-term gains. Bottom line: each of us must work on spreading a more positive attitude to save us from bleak future.
  • Book – Nathan Marz: Big Data – I dislike the big data hype (and, with passion, Hadoop) but would love to read this book; it presents a fresh look at big data processing, heavily inspired by functional programming. Nathan has plenty of experiences from Twitter and creating Storm and Cascalog (both in Clojure, btw.). Read ch 1:  A new paradigm for big data.
  • Facebook Engineering: The Mature Optimization Handbook (or go directly to the pdf,   ePubMobi). If you get bored, jump directly to ch 5. Instrumentation.

Clojure Corner

  • Schmetterling – Debug running clojure processes from the browser! – upon an exception, the process will pause and S. will show the stack, which you can navigate and see locals and run code in the context of any stack frame; you can also trigger it from your code
  • Gorilla REPL (screenshot, 11min video)- interactive web-based notebook where you can mix text (with Markdown formatting), mathematical formulas via LaTeX, graphs, tables, Clojure code. Great for exploring and, at the same time, describing data. <3
  • Local state is harmful – how can we answer the questions about when/why did state X change, how did output Y get where it is? Make state explicit, f.ex. one global map holding all of it, and perhaps not just the current state but also history – thus we can easily query it. Prismatic’ Graph can be used to make the state map, watches to keep history. Inspired by databases (Datomic is an excellent example of SW where answering such questions is trivial)
  • S. Corfield: Insanely Useful Leiningen Plugins – lein-ancient (find updated deps), lein-exec (execute Clj from cmd.line / scripts in Clj), lein-try (try a lib in REPL), Eastwood – a lint tool for Clojure
  • Sente – Clojure(Script) + core.async + WebSockets/Ajax – a tiny 600 LoC library for websockets (with fall-back to long-polling) communication between ClojureScript frontend and clojure backend, using EDN, support for request-reply and multiple user windows/tabs (comparison with Chord (no non-WS fallback or req/resp))
  • Nicholas Kariniemi: Why is Clojure bootstrapping so slow? – don’t blame the JVM, most time spent in clojure.core according to this analyzes on JVM and Android (create and set vars, load other namespaces); some proposals for improving it – lazy loading, excluding functionality not used, …
  • Cheat your way to running CLJS on Node – (ab)use D. Nolen’s mies template intended for client-side cljs development to create a Node project; the trick: compile everything into 1 file so that Node does not fail to find dependencies, disable source maps etc. Update: the nodecljs template now does this
  • lt-clojure-tutorial – A Clojure tutorial optimized for Light Table, ported from Nolen’s cljs tutorial

Tools/Libs

Posted in General, Languages, Tools, Top links of month | Tagged: , , , , , , , , | Comments Off

Most interesting links of December ’13

Posted by Jakub Holý on December 31, 2013

Recommended Readings

Society

  • HBR: Want to Build Resilience? Kill the Complexity – a highly interesting, thought provoking article relevant both to technology in particular and the society in general; f.ex.: more security features are bad for they make us behave less safely (risk compensation) and are more fragile w.r.t. unexpected events. “Complexity is a clear and present danger to both firms and the global financial system: it makes both much harder to manage, govern, audit, regulate and support effectively in times of crisis. [..] Combine complex, Robust-Yet-Fragile systems, risk-compensating human psyches, and risk-homeostatic organizational cultures, and you inevitably get catastrophes of all kinds: meltdowns, economic crises, and the like.” The solution to future financial crisis is primarily not more regulation but simplification of the system – to make it easier to police, tougher to game. We also need to decrease interconnectednes (of banks etc.), one of the primary sources of complexity. Also a great example of US Army combatting complex, high-risk situations by employing “devil’s advocates / professional skeptics” trained to help “avoid the perils of overconfidence, strategic brittleness, and groupthink. The goal is to respectfully help leaders in complex situations unearth untested assumptions, consider alternative interpretations and “think like the other”“.
  • The Dark Side of Technology – technologies provide great opportunities – but also risks we should be aware of – they create a world of mounting performance pressure for all of us (individuals, companies, states), accelerate the rate of change, increasing uncertanity (=> risk of Taleb’s black swans). “All of this mounting pressure has an understandable but very dangerous consequence. It draws out and intensifies certain cognitive biases [..]” – magnify our perception of risk, shrink our time horizons, foster a more and more reactive approach to the world, the “if you win, I will lose” view, erode our ability to trust anyone – and “combined effect of these cognitive biases increases the temptation to use these new digital infrastructures in a dysfunctional way: surveillance and control in all aspects of our economic, social and political life.” => “significantly increase[d] the likelihood of an economic, social and political backlash, driven by an unholy alliance between those who have power today and those who have achieved some modest degree of income and success.
    Complexity theory: the more connected a system is, the more vulnerable it becomes to cascades of disruptive information/action.
  • What Do Government Agencies Have Against 23andMe, Uber, and Airbnb? – innovative startups do not fit into established rules and thus bureaucrats do not know how to handle them and resort to their favourite weapon: saying no, i.e. enforcing rules that harm them (f.ex. France recently passed a law that requires Uber etc. drivers to wait 15 min before picking up a customer so that established taxi services have it easier; wot?!)
  • Nonviolent communication in action – wonderful stories about NVC being applied in difficult situations with a great success

Tech

  • D. Nolen: The Future of JavaScript MVC Frameworks – highly recommended thought food – about React.js, disadvantages of event-based UI, benefits of immutability, performance, the ClojureScript React wrapper Om  – “I simply don’t believe in event oriented MVC systems – the flame graph above says it all. [...] Hopefully this gives fans of the current crop of JS MVCs and even people who believe in just using plain JavaScript and jQuery some food for thought. I’ve shown that a compile to JavaScript language that uses slower data structures ends up faster than a reasonably fast competitor for rich user interfaces. To top it off Om TodoMVC with the same bells and whistles as everyone else weighs in at ~260 lines of code
  • Quora: Michael Wolfe’s answer to Engineering Management: Why are software development task estimations regularly off by a factor of 2-3? – a wonderful story explaining to a layman why estimation is hard, on the example of a hike from SF to LA
  • Style Guide for JavaScript/Node.js by Felix Geisendörfer, recommended by a respectable web dev; nothing groudn breaking I suppose but great start for a team’s standards
  • Johannes Brodwall: Why I stopped using Spring [IoC] – worth to read criticism of Spring by a respected and experienced architect and developer; summary – dependency injection is good bug “magical” frameworks decrease understandability and encourage unnecessarily complex code => smaller code, , easier to navigate and understand and easier to test
  • Misunderstanding technical debt – a brief discussion of the various forms of tech. debt (crappy code x misaligned design and problem domain x competence debt)
  • Tension and Flaws Before Health Website Crash – surprising lack of understanding and tensions between the government and contractors on HealthCare.gov – “a huge gap between the administration’s grand hopes and the practicalities of building a website that could function on opening day” – also terribly decision making, shifting requirements (what news!), management’s lack of decision power, CGI’s blame-shifting. A nice horror story. The former head knew that they should “greatly simplify the site’s functions” – but the current head wasn’t able to “talk them out of it”.
  • The Log: What every software engineer should know about real-time data’s unifying abstraction – logs are everywhere and especially important in distributed apps – DB logs, append-only logs, transaction logs – “You can’t fully understand databases, NoSQL stores, key value stores, replication, paxos, hadoop, version control, or almost any software system without understanding logs” – I have only read a small part but it looks useful
  • What I Wish I Knew When Learning Haskell tl;dr
  • Better Than Unit Tests – a good overview of testing approaches beyond unit tests – including “Automated Contract Testing” (ability to define a contract for a web service, use it to test it and to simulate it; see Internet of Strings for more info), Property-based Testing (test generic properties using random data/calls as with Quickcheck), Fault Injection (run on multiple VMs, simulate network failures), Simulation Testing as with Simulant.
  • Use #NoEstimates to create options and deliver value reliably – a brief post with an example of an estimation-based vs. no-estimates project (i.e. more focus on delivering early, discovery)
  • How Google Sold Its Engineers on Management – managers may be useful after all :-); a report about Google’s research into management and subsequent (sometimes radical) improvements in management style/skills and people satisfaction; I love that Google hasn’t HR but “people ops”
  • Roy Osherove: Technical Disobedience – take nothing for granted, don’t let the system/process stop you, be creative about finding ways to improve your team’s productivity; there always is a way. Nice examples.
  • Uncle Bob: Extreme Programming, a Reflection – a reflection on changes in the past ~ 14 years since XP that have seen many of the “extreme” practices becoming mainstream
  • The Anti-Meeting Manifesto – essentially a checklist and tips for limitting meetings to minimum

Other

Talks

  • Pete Hunt: React: Rethinking best practices (JSConf 2013, 30 min) – one of the most interesting talks about frontend development, design, and performance I have heard this year, highly recommended. Facebook’s React JavaScript framework  is a fresh and innovative challenger in the MVC field. It is worthwile to learn why they parted ways with the popular approach of templates (spoiler: concern separation, cohesion x coupling, performance). Their approach with virtual DOM enables some cool things (run in Node, provide HTML5-like events in any browser with consistent behavior, …). Key: templates are actually tightly coupled to display logic (controllers) via the model view tailored for them (i.e. Controller must know what specific data & in what form View needs) => follow cohesion and keep them together componets, separate from other components and back-end code. Also, state changing over time at many places is hard => re-render the whole app rather than in-place updates. Also, the ClojureScript Om wrapper enables even more performance optimizations thanks to immutable data structures etc.
  • David Pollak: Some musings on Scala and Clojure by a long time Scala dude (46 min) – a subjective but balanced comparison of Scala and Clojure and their strengths/weaknesses by the author of the Scala Lift framework (doing Scala since 2006, Clojure since 2013)

Clojure Corner

Tools/Libs

  • Apache Sirona – a new monitoring tool in the Apache incubator – “a simple but extensible monitoring solution for Java applications” with support for HTTP, JDBC, JAX-RS, CDI, ehcache, with data published e.g. to Graphite or Square Cube. It is still very new.
  • GenieJS – Ctrl-Space to popup a command-prompt for your web page, inspired by Alfred (type ‘ to see all possible commands)

Favourite Quotes

A good #agile team considers their backlog inaccurate. It is merely a list of assumptions that must be tested & refined by shipping product
- @mick maguire 12/10

Ada Lovelace (1st program), Grace Hopper (1st compiler), Adele Goldberg (1st OO language), why would anyone think women aren’t in computing?
@Dan North 12/11

There will always be a shortage of talented, self-motivated creative professionals who will unquestioningly follow orders.
@Thomas K Nilsson 12/7

Estimation paradox = If something unpredictable happens, predict how long it will take to fix it
me 12/7

IT systems can be inspired by AK-47 a.k.a. Kalashnikov. The rifle was purposefully designed to be simple and to be tolerant to imperfections in most parts; as a result, it required essentially no maintenance and was extremely reliable.
– summarized from Roman Pichlík’s Odkaz Michaila Kalašnikova softwarovému vývoji

Posted in General, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , , , , | Comments Off

Most interesting links of September ’13

Posted by Jakub Holý on September 30, 2013

Recommended Readings

  • Stuff The Internet Says On Scalability For September 13, 2013 – a collection of interesting performance related articles with summaries (via @_dagi)
  • Can you copy a culture? The NUMMI story (audio/transcript) – how the GM factory with the worst workforce has been turned around via a good application of Toyota Production System – “a truly inspiring story of human potential and how systems can be designed to bring the best or worst of of people.” And how GM failed to learn from it and to copy Toyota’s culture.
  • The Reactive Manifesto – why to write reactive SW – “Reactive applications represent a balanced approach to addressing a wide range of contemporary challenges in software development. Building on an event-driven, message-based foundation, they provide the tools needed to ensure scalability and resilience. On top of this they support rich, responsive user interactions. We expect that a rapidly increasing number of systems will follow this blueprint in the years ahead.
  • Frameworkless JavaScript – Why Angular, Ember, or Backbone don’t work for us [Moot discussion platform] (via JavaScriptWeekly) Me: Frameworks are not always evil, but are likely overused and there are good cases when rolling your own solution is the best way. Why in Moot? Because the want a minimal API (no framework methods), small code size, small and familiar code base, no dependency hell and external package updates, no lock-in to technology that will be gone in few years, need WebSockets not REST. “Moot uses native pushState for managing URLs, John Resig’s “micro templating” for views, and internal communication between model and views happens with a custom event library. There is no router or automatic data-binding.” The looked at Angular, Ember, Backbone. “As a result of our combined perfectionism and minimalism, Moot is an extremely lightweight, manageable, and independent web application [..]
  • NYT: Eiji Toyoda, Promoter of the Toyota Way and Engineer of Its Growth, Dies at 100 – learn about the life of one of the founders of lean thinking
  • Gojko Adzic: How we solved our #1 product management problem – valuable experience of false assumptions, learning from users, and a much helpful UI remake: even if you build a product to scratch your itch, you have to test it with real users

Big data

  • Don’t use Hadoop – your data isn’t that big – a great post about the downside of Hadoop and that there are much better options (large disks, large RAM, Pandas/R/Postgres) for data up to few TBs. “In addition to being more difficult to code for, Hadoop will also nearly always be slower than the simpler alternatives.”
  • Gartner On Big Data: Everyone’s Doing It, No One Knows Why – golf talk / hype -driven initiatives FTW! “According to a recent Gartner report, 64% of enterprises surveyed indicate that they’re deploying or planning Big Data projects. Yet even more acknowledge that they still don’t know what to do with Big Data.”
  • What makes Spark exciting – why it might be a good replacement for Hive/Hadoop, based on experiences with H/H: “Hive has served us well for quite a while now. [...]  That said, it has gotten to the point where Hive is more frequently invoked in negative contexts (“damn it, Hive”) than positive. (Primarily due to being hard to test, hard to extend.)” “We had heard about Spark, but did not start trying it until being so impressed by the Spark presentation at AWS re:Invent [..] that we wanted to learn more. [..] Spark, either intentionally or serendipitously, addresses both of Hive’s primary shortcomings, and turns them into huge strengths. (Easy to test, extend.) [..] I find the codebase small and very easy to read, [..] –which is a nice consolation compared to the daunting codebases of Hadoop and Hive.” Cons.: Spark is only pre-1.0, the author hasn’t yet tested it heavily.
  • 10 Ways to Make Your Office Fun To Work In – because we spend there plenty of our time so why not have a pleasant/cosy, inspiring environment? Some tips: plants, not-your-boring-enteprprise-look-and-feel, open it to the nature (I want this!), design it as home, not office, provide play space (I am too into work to want to play but having a resting place for a nap is st. I’d love).

Books

Other

  • Stanford engineers build computer using carbon nanotube technology (via @RiczWest)
  • NYT: The Banality of Systemic Evil – a good article about human tendency to “obey the system” thus potentially causing evil – and thus the need to resist the system, as heroic individuals such as Snowden, Hammond, Schwartz, Manning. See the famous Eichmann in Jerusalem for how “doing your job” can create evil – “[..] what happens when people play their “proper” roles within a system, following prescribed conduct with respect to that system, while remaining blind to the moral consequences of what the system was doing — or at least compartmentalizing and ignoring those consequences.” (Tip: The book Moral Mazes explores the ethics of decision making within several corporate bureaucracies => mid-managers rules of life: (1) never go around your boss, (2) tell the boss what she wants to hear, (3) drop what she wants dropped, (4) anticipate what the boss wants so that she doesn’t need to act as a boss to get it, (5) do not report something the boss does not want reported, cover it up; the the job & keep your mouth shut.) “The bureaucracy was telling him [Snowden] to shut up and move on (in accord with the five rules in “Moral Mazes”), but Snowden felt that doing so was morally wrong.” “[..] there can be no expectation that the system will act morally of its own accord. Systems are optimized for their own survival and preventing the system from doing evil may well require breaking with organizational niceties, protocols or laws.
  • Fairphone – “A seriously cool smartphone that puts social values first” (likely the only one not built by poorly paid workers and creating too much ecological burden), for just €325. You can see detailed cost breakdown, list of suppliers, specs, and essentially everything. This is, in my opinion, super cool! Go and read the story!

Clojure Corner

  • Amazonica – “A comprehensive Clojure client for the entire Amazon AWS api.”
  • Talk Ritz, The Missing Clojure Tooling (40min, 9/2013) – thanks to this I finally understood how to use Ritz but it still seems not to work well, f.ex. setting a breakpoint always reported “Set 0 breakpoints” (lein ritz/middleware 0.7.0, nrepl-ritz.el 0.7.1); according to callen, debug-repl is simpler and nicer if you only care about local vars and evaluation. To try ritz: use M-x nrepl-ritz-jack-in, then M-x nrepl-ritz-break-on-exception, exec. f.ex. “(/ 1 0)”. In the poped-up buffer, t or enter to show frame locals, e to eval a code in the context of the frame etc. If you managed to trigger the debug buffer through a breakpoint, the actions lists would contain STEP etc. (See fun. nrepl-ritz-line-breakpoint)
  • C. Grand’s spreadmap – “library to turn Excel spreadsheets in persistent reactive associative structures” => access content via map functions; changing a value updates formula cells using it
  • Alembic Reloads your Leiningen project.clj Dependencies – add a dependency to your project.clj w/o needing to restart your REPL (just call (alembic.still/load-project), provided you have it in your lein dependencies). Limitations: cannot remove deps or change versions.
  • Defeating stack overflows – techniques for transforming mutually recursive calls etc. into something that won’t blow the stack – “Priming the pump” (memoize subresults first), core.async
  • Google Groups: Clean Architecture for Functional Programming – How do the Clean Architecture and the Clean Code best practices apply to FP (Clojure/Haskell)? Some points: OOP isn’t worse than FP, only people do class-oriented programming instead; OO better e.g. for UIs, combining them (func. core, imperative shell) can be sometimes best. Some clean arch. patterns are actually more like functions – “Interactors and Presenters, for example, do not maintain any state of their own.  Even those objects that do imply some kind of state, such as entities and gateways, keep that state hidden behind boundaries and present a functional interface instead.
  • night-vision: Handy, super light weight debugging utility – add it to your lein profile and then call (night-vision.goggles/introspect-ns! '<name of ns>) and it will print each entry/exit of a function within the scope of the namespace with the argument/return values
  • Nil Punning (Or Null Pointers Considered Not So Bad) – a great post about why nil in Clojure is not bad contrary to Java’s null (because it is actually an object, you can call functions on it, treat it as false/empty list/map/set, most core functions work on it)

Tools/Libs

Posted in General, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , , , | Comments Off

Most interesting links of August ’13

Posted by Jakub Holý on August 31, 2013

Sorry folks, this month it will be very brief. I have many more great stuff in the queue but haven’t managed to write it down yet. Next month will be heavy :-)

Recommended Readings

  • Interested in native vs. webapp? Check out Why mobile web apps are slow (mobile browser much slower, not much real improvements, weak CPUs,…; seems to be really high-quality, plenty of data) and Sencha’s 5 Myths About Mobile Web Performance (Mobile web performance is mostly driven by JavaScript performance on the CPU, CPU-Bound JavaScript has only become faster because of HW improvements, Mobile browsers are already fully optimized, Future hardware improvements are unlikely to help, JavaScript garbage collection is a performance killer).
  • Why Software Projects are Terrible and How Not To Fix Them – many teams are not ready to embrace new/better software practices, primarly for two reasons: 1) most of them are nonintuitive (f.ex. adding more people will slow dev down) and need to be sold through a high hierarchy of managament – but people/managers/organizations don’t really care, it takes years for good/bad practices to have an impact, which is not relevant “now.” 2) Businss objectives change too quickly and SW is blamed for not delivering. Based on evaluating many failed projects. Conclusion: Choose carefully people/organizations your work with. Avoid blame-driven ones. Quote on middle managers: “He has to put more developers on the project, call a meeting and yell at people, and other arbitrary bad ideas.  Not because he thinks those will solve the problem.  In fact, managers often do this in spite of the fact that they know it’s bad. Because that’s what will convince upper management that they’re doing their best.” “In the vast majority of failed projects I’ve been called to looked at, the managers have not read one book on software engineering.

Data & Analytics

  • Big Data: Kafka for uSwitch’s Event Pipeline – a better alternative to log files – use LinkedIn’s Kafka for messaging, have MR jobs to import latest messages into Hadoop/HDFS. The advantage of Kafka is that it persists the messages for a period of time so it is easy to batch-import and even re-import them. The uSwitch’s talk Users As Data explains the downsides of log files. LinkedIn’s Camus is a tool for importing messages from Kafka to HDFS.
  • Realtime Analytics with Storm and Hadoop (at Twitter; presentation deck) – pre-aggregate some data into a read-only, random read DB such as ElephantDB, Voldemort, or Manhattan. For newer data use Storm and aggregated data in a read-write, big-data DB such as HBase, Riak, or Cassandra. For stuff that cannot be pre-aggregated you might use Storm’s Distributed RPC.
  • The Unified Logging Infrastructure for Data Analytics at Twitter – a paper from late 2012 that presents “Twitter’s production logging infrastructure and its evolution from application-specific logging to a uni- fied “client events” log format, where messages are captured in common, well-formatted, flexible Thrift messages” – with the benefit of “s streamlined log collection and data analysis”.

Other

  • Development and Deployment at Facebook (Kent Beck et. al., 8/2013, 13p paper) – “More than one billion users log in to Facebook at least once a month to connect and share content with each other. Among other activities, these users upload over 2.5 billion content items every day. In this article we describe the development and deployment of the software that supports all this activity, focusing on the site’s primary codebase for the Web front-end.

Talks

  • One of the most valuable talks I’ve seen, in just 18 min: The Progress Principle – about the disengagement crisis and motivation at work by Teresa Amabile at TEDx Atlanta (via @thovden). Disengagement from work is increasing, at all age and salary levels, and leads to unhappy people, low productivity, huge financial losses. Based on analysing diaries of 12k participants, the single most important engaging and motivating factor is making progress in a meaningful work (including small wins). A culture of management by fear and punishment for failure creates disengagement and can crush even an innovative, profitable, praised company in a few years. Everybody, though especially the management, creates the culture through their everyday, small actions. If everybody focuses on catalysing progress and supporting their fellow humans through good and bad times, engagement and success will follow. Remove progress inhibitors, nourish the human spirit (acknowledge what we humans value, encourage people). Yet of the managers asked, very few knew of the significance of making progress (or, I can assume, of supporting people and making them happy(er) and the impact of our inner work life (perceptions, emotions, etc.) on our productivity and creativity). The study included two seemingly similar, successfull companies, one with great engagement, another with a new management that managed to destroy the engagement and thus eventually the company. Actions to take: catalyse progress, celebrate wins, encourage and support your colleagues.

Clojure Corner

  • Wonderful Clojure Cheatsheet 1.5 with tooltips showing the doc and summary of information available at clojuredocs.org (other Clj versions), by Andy Fingerhut
  • Chas Emerick’s Clojure type selection flowchart to help you decide whether to use a map, a record, reify, proxy, gen-class, or deftype.  (Reify and proxy don’t produce a class but just an instance of an anonymous class; proxy can extend a base class, reify cannot. gen-class produces a class visible from Java and can extend Java classes.  …)

Tools/Libs

  • Docker.io – pack, ship and run any application (and its dependencies) as a lightweight container, i.e. essentially “a VM without the overhead of a VM,” using linux containers (chroot on steroids with resource limits via control groups) see reports of some uses such as Java app deployment, desktop virtualization, automatic app deployment in GitHub commit. Docker also supports evolving the containers over time, i.e. deploying new version, by pushing just diffs so it’s low-overhead. You can build a container (include files, SW, forward ports, …) using a Dockerfile. See dotScale 2013 – Solomon Hykes – Why we built Docker for an intro (20 min).
  • Packer.io – tool for building pre-configured VM images for different platforms (EC2, VirtualBox, …), remotely similar to Netflix’s Aminator. See Immutable Servers With Packer and Puppet for an example use case.
  • Ubuntu-build Vagrant boxes at cloud-images.ubuntu.com/vagrant/
  • SlimerJS – PhantomJS-compatible headless browser engine based on Firefox/Gecko (well, it is not fully headless yet but that is planned; the main focus now is full compatibility with PhantomJS’ API) (Both work with CasperJS for navigational steps/testing.)
  • localtunnel – instantly show locally running webapp/server to the rest of the world (gem install localtunnel,  localtunnel <port to share>, => share the url returned, e.g. http://xyz.localtunnel.com) – I haven’t tried it but it looks simple and very convenient
  • Logstash + Kibana (via @mortenberg): take control of your logs – while Logstash can collect (from multiple servers/services), parse (over 100 built-in patterns), store, index, search your logs, Kibana is a web interface to seach them, view them in realtime (based on a query) etc. See this Logstash slides (9/2012) and an overview of Kibana’s powers. PS: Logstash can also compute metrics and send them to graphite etc. It is typically used with ElasticSearch.
  • ncdu is an interactive, command-like disk usage browser that shows a list of directories sorted by size shown in human-friendly units, you can navigate with arrows and enter and i to show the current dir/file info, d to delete it, q to quit; check out this article about ncdu with screenshots and ncdu man page. Install via Apt etc., run f.ex. with ncdu -x / .
  • vagrant-cachier – Vagrant plugin for caching apt/yum/.. packages locally, thus speeding up destroy+up

Posted in General, Tools, Top links of month | Tagged: , , , , , , , , | 2 Comments »

Most interesting links of July ’13

Posted by Jakub Holý on July 31, 2013

This month focuses on languages and approaches (reactive programming, F#, Erlang, FP talks etc.), agile (need for speed, recommended books), Clojure/Linux/cloud tools and libs.

Recommended Readings

Development, agile

  • The Need For Speed – the top 10 reasons for fast development flow (with time to market being one of the less important) – more learning, focus on the MVP, focus on the puprose/goal, happier customers/leadership, better quality (sic!), higher morale (I concur!), push for cotninuous improvement, “one of the only sustainable differentiators”; => “sense of urgency and motivation”; “[..] I continue to meet people and teams that not only move very slow, they don’t understand the relationship between speed and innovation, or speed and quality.”
  • agile42 Summer Reading List 2013 – books recommanded by experienced people/agile experts – lot of interesting stuff! Topics: Communication and Coaching (f.ex. Practicing Nonviolent Communication), Business (How Great Companies Get Their Mojo from Maslow ,..),  Learning From the Military, Agile and Technology (e.g. The Art of Action: How Leaders Close the Gaps between Plans, Actions and Results), Agile and Technology (f.ex. The people’s Scrum)
  • Dan North: Are we nearly there yet? – optimize for time to business impact; SW dev as mountaineering (impossible to estimate correctly, many unknown details, dead ends, …); go fast – but sustainable; the tyranny of backglog (there are multiple paths to the top yet backlog defines only one; have you ever considerably changed it?) “Instead we could embrace the fact that today we always know more than we did yesterday, and that tomorrow we will know even more. We can take a fresh look up the mountain every time we pause to regroup, to plan.” => we ask 1) what gives us the shortest lead time to business impact? 2) what can help us to learn/invalidate more? 3) how to assure our stakeholders we are approaching the goal?
  • Joel on Software: Software Inventory (7/2012) – a classical article about the evilness of software inventory (backlogs, issue trackers, undeployed features, …) ‘When I hear about product teams that regularly have “backlog grooming” sessions, in which they carefully waste a tiny amount of time and mental energy every day or every week thinking about every single feature which will never be implemented, I want to poke my eyes out.
  • Job satisfaction self-test: Twelve questions that define a great place to work – check yourself how satisfied you are with your job (example questions: How well do I know what is expected of me? How often in the past seven days have I received recognition or feedback on my work? How much does the mission/purpose of the company make me feel like my work is important?)
  • Coaching Anti-Patterns: Prescriptive Agile – a prescriptive coach “knows” what is “right” and forces it onto the client, without listening to her; instead, we should “Meet them where they are and leave them in a better place” => “[..] my first responsibility is to understand how and why they came to this practice. How did they come to this decision? What challenges does this approach address? What benefits are they optimizing for?” Worth remembering AND practicing :)

Languages, paradigms, approaches

  • Bacon.js Makes Functional Reactive Programming Sizzle – a nice introduction into Bacon.js that brings Functional Reactive Programming (FRP) to JavaScript and helps thus escape the callback hell. Reactive programming has been made popular by Microsoft’s Rx and recently ported to Java as RxJava by Netflix. FRP is a subtype of RP with functional concepts (map, filter, immutability, …). It provides a much cleaner way to handle multiple independent sources of events and reaction to those events, the main concepts are composable Streams of events and Properties, whose values are automatically updated based on a stream. Bacon.js Tutorial Part I : Hacking With jQuery provides a nice example of the complexity and ugly code you can run into without (F)RP even for a simple interactive web form, the Tutorial Part II: Get Started then shows the nicer Bacon.js solution.
  • The Trouble with Erlang (or Erlang is a ghetto) – an objective criticism of Erlang by somebody who seems to be quite experienced with it; as I know very little about Erlang, it was interesting to learn about its weaknesses (no map/dict data structure, slow memory management, poor “JIT,” not usable for shared-state concurrency (contrary to e.g. Clojure), immutable state is not necessary and makes some things bad, inconsistent and ugly standard lib, …)
  • Adventures in Multi Paradigm Programming – different programming paradigms/approaches re-implemented in Emacs Lisp – interesting 1) to see and compare these different approaches and 2) the flexibility of Lisp. Including iteration – Ruby’s map, Python’s list comprehension ([an_expression for x in list]), Scala’s default argument (_); search – Java’s for; arguments: direct, variadic (i.e. any number of args), named args; destructuring and pattern matching in CoffeScript/OCaml style; Haskell-like monads; objects with mixins;  namespaces.
  • Why bugs don’t like F# – no nulls, immutable data, strong type system, composition of small functions, asynchronous programming abstractions, higher-order functions over collections (no off-by-one), units of measure

Other

  • IBM high-fives Netflix open-source tools – it is interesting to see the spreading of Netflix’s open source tools for better cloud infrastructures; f.ex. “Karyon, is what Netflix calls the base container for applications and services built using the NetflixOSS ecosystem; Eureka is mid-tier load balancing; Hystrix controls interactions between myriad distributed services to nip cascading failures in the bud;  and Ribbon is a Remote Procedure Call library.”
  • ZeroMQ instead of HTTP, for internal services (with implementation in Clojure) – an interesting idea of using ZeroMQ – the sockets on steroids library – instead of HTTP in a way compatible with existing HTTP routing libs; advantages of ZeroMQ: automatic retrial (=> can restart the target service withou noticing), speed, reuse of a connection. The trick is to send a http-like structure (i.e. with method, uri, body) and pass that to Compojure or similar (update: there are now Clojure/core.async bindings for ZeroMQ)
  • Joel on Software: Victory Lap for Ask Patents – killing a bad Microsoft patent request in 15 minutes – Ask Patents is a new StackExchange site that enables experts to look at SW patent requests and point to previous existing works that invalidate them; as Joel describes in his successful patent kill story, it is not difficult at all. Hopefully this will manage to really help the patent office and hit woul-be patent trolls hard! #victory
  • The Dangers Of “Gamification” In Education by Kathy Sierra (a former game designer, a trainer of trainers at Sun, author of the Head First book series) – gamification is often regarded as something very desirable that will improve our lives; however, as Kathy discusses, it has also dark sides and, applied unappropriately, can actually decrease our intrinsic motivation (therefore it should be nearly never used in e.g. education)
  • Choosing an OSS license doesn’t need to be scary (by GitHub) – a human-readable overview of OSS licenses; you should always assign a license to your GitHub account (Add A License can help with that; otherwise it is considered to be “all rights reserved” and you are not giving back to the community (I use the same as Clojure, Eclipse Public License)

Talks

  • FunctionalTalks.org – “Brilliant people giving brilliant talks on functional programming” – f.ex. Wilkes Joiner: Functional Reactive Programming, Alexander Gounares: All your cores are belong to us, Katie Miller: Superhero monads, Bryan O’Sullivan: Running A Startup On Haskell, Rich Hickey: Introduction To Clojure, John A. De Goes: Building a Data Science Platform in Scala and many more.
  • Types vs. Tests: An Epic Battle? – “Amanda Laucher and Paul Snively debate solving problems through types and tests using different approaches.” – can type system replace tests or vice versa? Interesting intro into the discussion for me. Using F#, Scala & more. Same claims: types don’t pay out so much for “small” codebases but scale better than tests. Types – Tests is a spectrum, not two single extremes. When a property should hold “for all,” a type would be a good match. Inductive types (Scala, Haskell?) can become quite complex, dependant types (as in Coq) would be much nicer [if I got that right].
  • Paul Irish on Web Application Development Workflow (via M. Noddeland) – if you need to do some web development but are not up to date on the state of art, this might be useful – an overview of tools, utilities, services by a Googler and the person behind Modernizr, HTML5 Boilerplate, Yeoman etc. Including effective shell & dotfiles.GH, better ssh via .ssh/config and authorized_hosts, the all-in-one dev/build tool Yeoman with live reload, BrowserStack for testing, LocalTunnel to easily share anything running locally, Chrome Dev Tools support for SASS and testing devices (emulate touch events, screen sizes), JetBrains’ WebStorm, sharing tools via setapp.me. A genous idea to use GoogleAnalytics to track usage of features in a CLI app!

Other Interesting Stuff

Azul Systems’ high-performance JVM on the Vega architecture (from The Trouble with Erlang (or Erlang is a ghetto)) looks very interesting:

The other night I tweeted “If you’re looking for a language that gets multicore concurrency right, look at how Azul implemented Java on their Vega architecture” and I definitely stand by that. Azul is a company that got a lot of smart hardware and software people together and had them work on designing a custom system which would scale to hundreds of CPU cores (up to 768 of them), heaps that topped 500 GB (up to 768GB), and had the GC pause only 10-20ms at a time. The realtime performance characteristics Azul managed to eek out of their system lead them to often describe their GC as “pauseless”.

Articles:

Clojure Corner

  • Discussion: How core.async compares to agents, future and promise? – future/promise: 1 producer, 1 value, multiple consumers; agent: an unbounded queue of functions mutating a single value, with multiple producers and consumers (reading the latest value produced); channel: multiple 1:1 producers/consumers, i.e. a value can only be taken once from the channel, using a bounded queue (=> slow consumers can block fast producers). As mentioned elsewhere, channels is a relatively low-level abstraction and other things can be built on the top of it.
  • Clojure Tradeoffs (design implications and why you should care) – perhaps not very unbiased but interesting anyway :) (shared-memory over other computing paradigms, i.e. message-passing, dynamic over static, speed over convenience, composition over IoC, …)
  • Rich Hickey’s post introducing core.async with its Go-like channels as a better alternative to a collback hell (I know everybody has already read it but it is still an important link :))
  • Tools etc.
    • Faster Clojure Startup with Class Data Sharing – use JVM’s capability to include any classes in its boot image and include clojure in it
    • lein-ancient – checks for outdated dependencies and plugins => run “lein ancient :all”
    • lein-try – a Leiningen plugin that enables you to try a library in a REPL in the context of your project without having to add it to project.clj; simply run “lein try clj-time 0.5.1″ and then in the REPL “(require ‘[clj-time.core :refer :all])” and e.g. “(date-time 1986 10 14)”
    • Lemur: tool to launch a Hadoop job locally/on EMR from a job definition file + actions before/after
    • Emacs: sexp fold/expand is very useful for exploring source code (hide all but the first lines of all top-level forms with hs-hide-all) – the built-in hs-minor-mode can hide/show all, or hide/show/toggle one but the keys for it are cumbersome; hideshow-org makes it possible to toggle hide/show with TAB, while preserving the original TAB behavior (it does the normal TAB first only only if nothing changes does it expand/fold); very useful!

Tools/Libs

  • devdocs.io (via @palruud): “an all-in-one API documentation reader for [web] developers,” navigable via keyboard – JS, HTML, CSS, DOM, DOM events, jQuery, Underscore.js
  • Kilim – a message-passing framework for Java that provides ultra-lightweight threads and facilities for fast, safe, zero-copy messaging between these threads.
  • AssertJ – a library of assertions similar to fest-assert but providing a richer set of assertions (nicer API then fest-assert, according to a friend)
  • NetflixOSS – Netflix, the online streaming gigant, has open-sourced many fascinating components of its cloud infrastructure such as Karyon, a blueprint for web-ready components with many features (monitoring,…), Genie/Hadoop as a Service, Servo for monitoring, Archaius for configuration management – too many to list. Check out Chris Fregly’s fluxcapacitor, a demo distributed application that uses many of the components
  • Tools to keep a daemon running:

Posted in General, Tools, Top links of month | Tagged: , , , , , , , , , , , , | Comments Off

Tools for Editor – Browser Integration for Interactive JS/HTML Development

Posted by Jakub Holý on March 25, 2013

Chrome Development Tools and similar ones are great for interactive, exploratory coding of JavaScript, HTML, and CSS – but the changes aren’t persistent and the tools haven’t the power of a programmer’s editor. I’d like to be able to use a powerful editor yet be able to see changes to JS/HTML/CSS without having to save-[compile]-reload and I want to be able to execute pieces of JS in the context of the browser. Fortunately, there are ways to get at least some of this and it is getting continually better. Let’s see what tools we have now.

These tools usually use either remoting capabilities of the browser or a long-polling connection to the web site, sending and executing JavaScript.

Read the rest of this entry »

Posted in Tools | Tagged: , , , | 3 Comments »

Most interesting links of November

Posted by Jakub Holý on November 30, 2011

Recommended Readings

  • Recommended Reading by Poppendiecks – an excellent selection, starting with Lean from Trenches, Management 3.0, Specification by Example, The Lean Startup etc.
  • Eric Allman says that Programming Isn’t Fun Any More  because problem solving has been replaced with learning, configuring, and integrating tons of libraries, frameworks, and tools and many people agree with that (as discussion on reddit proves). In other words we tend to go for any benefit we can have without considering the costs and for “easy” solutions without considering the true enemy: complexity. Perhaps we should always listen to the Rich Hickey’s Simple Made Easy talk before we add a lib/tool/framework?
    • Dean Wampler claims that functional programming can bring the joy back – “[..] a functional language, Scala, Clojure, Haskell, etc. will greatly reduce the amount of code you create. That won’t solve the problem of trying to integrate with too many libraries, but you’ll be less tempted. I also believe those libraries will be less bulky, etc.
    • Few quotes from a related article by M. Taylor: To put it another way, libraries make excellent servants, but terrible masters. | [..] frameworks [..] do keep their promise of making things very quick and easy … so long as you do things in exactly the way the framework author intended | On libraries: [..] we all assume (I know I do) that “plug in solutions X1 and X3″ is going to be trivial. But it never is — it’s a tedious exercise in impedance-matching, requiring lots of time spent grubbing around in poorly-written manuals [..] | On the effect of language choice: [..] different languages, with their different expressive power and especially their different culture, yield very different experiences.
    • To sum it up: Choose your tools and libraries wisely and always mind the global complexity. More usually means worse.
  • Java Magazine – Adam Bien: Stress Testing Java EE 6 Applications (page 41+) – do developer stress testing! Using: JMeter, VisualVM to find out resource consumption and behavior in the application, VisualVM’s Sampler profiling tool [cca 20% overhead], a webapp to extract metrics from GF (STM)
  • Java Magazine – Polyglot Programming on the JVM (page 50; excerpt from The Well-Grounded Java Developer) – why you should consider polyglot programming and how to decide whether to use it and what languages to pick, f.ex.: “These [Java's] qualities make the language a great choice for implementing functionality in the stable layer [of the polyglot programming pyramid]. However, these same attributes become a burden in the middle and upper [lower, DSL, on the linked image] tiers of the pyramid; for example: Recompilation is laborious; Static typing can be inflexible and lead to long refactoring times; Deployment is a heavyweight process; Java’s syntax is not a natural fit for producing DSLs.” “There is a wide range of natural use cases for *alternative languages*. [after identifying such a UC] You *now need to evaluate* whether using an alternative language is appropriate.”
  • Intrusion Detection for Web Apps – Detection Points – If security is a concern of your web application then you should build intrusion detection into the application f.ex. leveraging the  OWASP AppSensor project. The key is to detect malicious/unexpected behavior and proactively do something such as locking the user out or alerting the admins. The page linked above lists some common suspicious behaviors such as the use of multiple usernames, unexpected HTTP command/method, additional/duplicated data in request. Worth checking out!
  • Yammer Moving From Scala to Java- Scala is a cool language but sometimes its cost is higher than the benefits. Snippets from the post: “…the friction and complexity that
    comes with using Scala instead of Java isn’t offset by enough productivity benefit or reduction of maintenance burden …”. “Scala, as a language, has some profoundly interesting ideas in it. [...] But it’s also a very complex language. The number of concepts I had to explain to new members of our team for even the simplest usage of a collection was surprising: implicit parameters, builder typeclasses, ‘operator overloading’, return type inference, etc. etc.” (It’s claimed that only library authors need to know some of that but if it’s a part of library APIs, the users need to understand it too.) Notice that the author isn’t saying “Scala is bad” but only that Scala isn’t the best balance of their needs at this time, as Alex Miller put it*.
    Important note
    : The text wasn’t intended for publication and it is a private opinion of a Yammer developer, not the company itself. You should read the official Yammer’s position where Coda puts it into the right context.

Refactoring

  • Opportunistic Refactoring by Martin Fowler – refactor on the go – how & why
  • Michael Feathers: Getting Empirical about Refactoring – gather information that helps us understand the impact of our refactoring decisions using data from a SCM, namely File Churn (frequency of changes, i.e. commits) vs. Complexity – files with both high really need refactoring. Summary: “If we refactor as we make changes to our code, we end up working in progressively better code. Sometimes, however, it’s nice to take a high-level view of a code base so that we can discover where the dragons are. I’ve been finding that this churn-vs.-complexity view helps me find good refactoring candidates and also gives me a good snapshot view of the design, commit, and refactoring styles of a team.

UIs and Web Frameworks

  • Devoxx 2011 - WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks (slides) – the authors did extensive performance testing of some of the most popular web frameworks. Of course it’s always hard to guess how general their results are, if/how they apply to one’s particular situation, and if they aren’t distorted in some way but it’s worth for their approach alone (AWS with its CloudWatch monitoring, WebDriver, additional measurement of page load with HAR and a browser plugin). In their particular tests GWT scored best, followed by Spring MVC, with JSF and Wicket lagging far behind (especially the MyFaces implementation). Conclusion: A web framework may have strong impact on performance and scalability, if they are important for you then do test the performance early with as realistic code and load as possible.
  • JSF2 – Benchmark datatable by N. Labrot, 2/2011 – performance comparison of PrimeFaces 2.2.1, IceFaces 2.0, Richfaces 4.0.0M4 on a simple page with Ajax. I do not trust any benchmark that I don’t fake myself :-) (for there are always too many factors that influence the conclusions to be drawn) but it’s interesting anyway – and perhaps a good thing to do before you decide for a JSF component library.
  • Alex MacCaw: Asynchronous UIs – the future of web user interfaces and the Spine framework – users in 2011 shouldn’t anymore wait for pages to load and operations to complete, we should build asynchronous UIs where changes to the UI are performed immediately while a request to the server is sent in the background, similarly to sending e-mail in GMail, which returns at once displaying a non-intrusive “Sending…” notification. As a user I very much agree with Alex.
  • Matt Raible’s 20 criteria for evaluating web frameworks, 2010 (detailed description, here’s a brief list) – Matt’s results are disputable and as he himself says you should always do your own evaluation and spikes but the criteria are pretty useful: Developer Productivity, Developer Perception, Learning Curve, Project Health, Developer Availability, Job Trends, Templating, Components, Ajax, Plugins or Add-Ons, Scalability, Testing, i18n and l10n, Validation, Multi-language Support (Groovy / Scala), Quality of Documentation/Tutorials, Books Published, REST Support (client and server), Mobile / iPhone Support, Degree of Risk.

NoSql

  • Don’t use MongoDB via @nicolaiarocci – a (fake?!) bad experience with MongoDB – the text is not credible (the author is anonymous, s/he doesn’t explicitely state which version of MongoDB they used, the 10gen CTO can’t find a matching client and any evidence for some of the issues mentioned) but it  gives context for the read-worthy response from the 10gen CTO, and a post that nicely explains how to correctly design for MongoDB. A comment about MongoDB experience at Forsquare: “Currently we have dozens of MongoDB instances across several different data clusters storing over a TB of data and handling 10s of thousands of requests per second (mostly reads but the write load is reasonably high as well).Have we run into problems with MongoDB along the way? Yes, of course we have. It is a new technology and problems happen.Have they been problematic enough to seriously threaten our data? No they have not.
  • Martin Fowler on Polyglot Persistence – the are when will be choosing persistence solution with respect to our needs instead of mindlessly picking RDBMS is coming. Applications will combine multiple, specific solutions, f.ex. we could pick Redis (key-value) for caching, MongoDB (document DB) for product catalog, Neo4J (graph DB) for recommendations, RDBMS for financial data and reporting… (of course not all in one project!). Polyglot persistence will come at a cost (complexity, learning) – but it will come because the benefits are worth it – performance, data storage model and behavior more aligned with the business logic (NoSql databases ofer various models and tradeoffs and thus we can find a much better fit than with general-purpose RDBMs).

Talks & Video

  • Adam Bien’s JavaOne talk Java EE 6: The Cool Parts (1h) – absolutely worth the time – a very practical fly through the cool features of Java EE (eventing, ..), most of the time is spent actually coding. Don’t forget to check also the interesting discussion below the video (JEE and other frameworks, Java FX and JSF 2, …).
  • Jurgen Appelo’s keynote How to Change the World at Smidig 2011 is well done and highly useful. We all strive to change the world around us – as consultants we want to make our clients more agile, as team members we want to make our Scrum teams more self-organizing, as employees we want to help building knowledge-sharing and open culture, … . However it isn’t easy to influence or change people and culture and if we aren’t aware of all the dimensions of a change (system, individuals, interactions, environment) and how to work along each of them, we are much less likely to succeed. The knowledge and experience that Jurgen shares with us can help us a lot in having an impact. You can also download the slides and change management questions.
  • Project X: What is being a programmer like? (5min) If ever again a non-geek asks you what you as a developer are doing, just show him this short and extremely funny video (created by my ex-employer – perhaps they estimated how much time and energy developers loose trying to explain it to normal people and decided to prevent this great waste :-))
  • RSA Animate – Drive: The surprising truth about what motivates us (10 min) – entertaining and enlightening; once we’ve enough money to cover our needs, it’s autonomy (self-direction), mastery, and purpose what motivates us (money actually decrease our performance). Now this is a great evidence for lean/agile – for they’re based on making people self-directing and encourage mastery (as in continous integration and top quality to enable steady pace). Autonomy enables engagement as does a higher purpose (“make the world a better place”) – Steve Jobs with his visions was able to provide such a purpose. Atlassian’s FedEx Days are a good example of what engagement and benefits autonomy brings.
  • Simon Sinek: How great leaders inspire action (18 min, subtitles in 37 languages) – do you want to succeed, to change the world around you for the better, to start a new company? Then you must start by communicating “why” you do what you do, not “what” – like M. L. King, bro Wrights, and Apple. Very inspiring! (More in his Why book.)

Links to Keep

Favorite Quotes

Refactoring is like advertising: it doesn’t cost, it pays.
– Mary & Tom Poppendiecks, Implementing Lean Software Development, p.166

Clojure Corner

Posted in Databases, General, j2ee, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Most interesting links of October

Posted by Jakub Holý on October 31, 2011

Recommended Readings

  • Steve Yegge’s Execution in the Kingdom of Nouns – I guess you’ve already read this one but if not – it is a well-written and amusing post about why not having functions as first class citizens in Java causes developers to suffer. Highly recommended.
  • Reply to Comparing Java Web Frameworks – a very nice and objective response to a recent blog summarizing a JavaOne presentation about the “top 4″ web frameworks. The author argues that based on number of resources such as job trends, StackOverflow questions etc. (however data from each of them on its own is biased in a way) JSF is a very popular framework – and rightly so for even though JSF 1 sucked, JSF 2 is really good (and still improving). Interesting links too (such as What’s new in JSF 2.2?). Corresponds to my belief that GWT and JSF are some of the best frameworks available.
  • Using @Nullable – use javax.annotation.Nullable with Guava’s checkNotNull to fail fast when an unexpected null appeares in method arguments
  • JavaOne 2011: Migrating Spring Applications to Java EE 6 (slides) – nice (and visually attractive) comparison of JavaEE and Spring and proposal of a migration path. It’s fun and worthy to see.
  • xUnitPatterns – one of the elementary sources that anybody interested in testing should read through. Not only it explains all the basic concepts (mocks, stubs, fakes,…) but also many pitfalls to avoid (various test smells such as fragile tests due to Data Sensitivity, Behavior Sensitivity, Overspecified Software [due to mocks] etc.), various strategies (such as for fixture setup), and general testing principles. The materials on the site were turned into the book xUnit Test Patterns: Refactoring Test Code (2007), which is more up-to-date and thus a better source.
  • Eclipse tip: Automatically insert at correct position: Semicolon, Braces – in “while(|)” type “true {” to get “while(true) {|” i.e. the ‘{‘ is moved to the end where it belongs, the same works for ‘;’
  • Google Test Analytics – Now in Open Source – introduces Google’s Attributes-Components-Capabilities (ACC) application intended to replace laborous and write&forget test plans with something much more usable and quicker to set up, it’s both a methodology for determining what needs to be tested and a tool for doing so and tracking the progress and high-risk areas (based not just on estimates but also actual data such as test coverage and bug count). The article is a good and brief introduction, you may also want to check a live hosted version and a little more detailed explanation on the project’s wiki.
  • JSF and Facelets: build-time vs. render-time (component) tags (2007) – avoid mixing them incorrectly
  • StackOverflow: What are the main disadvantages of Java Server Faces 2.0? Answer: The negative image of JSF comes from 1.x, JSF 2 is very good (and 2.2 is expected to be just perfect :-)). Nice summary and JSF history review.
  • Ola Bini: JavaScript in the small – best practices for projects using partly JavaScript – the module pattern (code in the body of an immediately executed function not to polute the global var namespace), handling module dependencies with st. like RequireJS, keeping JS out of HTML, functions generating functions for more readable code, use of many anonymous functions e.g. as a kind of named parameters, testing, open questions.

Talks

  • Kent Beck’s JavaZone talk Software G Forces: The Effects of Acceleration is absolutely worth the 1h time. Kent describes how the development process, practices and partly the whole organization have to change as you go from annual to monthly to weekly, daily, hourly deployments. What is a best practice for one of these speeds becomes an impediment for another one – so know where you are. You can get an older version of the slides and there is also a detailed summary of the talk from another event.
  • Rich Hickey: Simple Made Easy - Rich, the author of Clojure, argues very well that we should primarily care for our tools, constructs and artifacts to be “simple”, i.e. with minimal complexity, rather than “easy” i.e. not far from our current understanding and skill set. Simple means minimal interleaving – one concept, one task, one role, minimal mixing of who, what, how, when, where, why. While easy tools may make us start faster, only simplicity will make it possible to keep going fast because (growing) comlexity is the main cause of slowness.  And simplicity is a choice – we can create the same programs we do today with the tools of complexity with drastically simpler tools. Rich of course explains what, according to him, are complex tools and their simple(r) alternatives – see below. The start of the 1h talk is little slow but it is worth the time. I agree with him that we should much more thing about the simplicity/complexity of the things we use and create rather than easiness (think ORM).
    Read also Uncle Bob’s affirmative reaction (“All too often we do what’s easy, at the expense of what’s simple. And so we make a mess. [...] doing what is simple as opposed to what is easy is one of the defining characteristics of a software craftsman.”).

Random Notes from Rich’s Simple Made Easy Talk:

There are also better notes by Alex Baranosky and you may want to check a follow-up discussion with some Rich’s answers.

The complex vs. simple toolkit (around 0:31):

COMPLEXITY                             SIMPLICITY
State, objects                            Values
Methods                                   Functions, namespaces
vars                                          Managed refs
Inheritance, switch, matching  Polymorphism a la carte
Syntax                                      Data
Imperative loops, fold              Set functions
Actors                                      Queues
ORM                                         Declarative data manipulation
Conditionals                             Rules
Inconsistency                            Consistency

What each of the complexity constructs mixes (complects) together

CONSTRUCT                        COMPLECTS (MIXES)
State, objects – everything that touches it (for state complects time and value)
Methods – function and state, namespaces (2 classes, same m. name)
Syntax – Meaning, order
Inheritance – Types (ancestors, child)
Switch/matching – Multiple who/what pairs (1.decide who, 2.do what ?)
var(iable)s – Value, time
Imperative loops, fold – what/how (fold – order)
Actors – what/who
ORM – OMG :-)
Conditionals – Why, rest of program (rules what program does are intertw. with the structure and order of the program, distributed all over it)

HE SIMPLICITY TOOLKIT (around 0:44)
CONSTRUCT            GET IT IVA…
Values – Final, persistent collections
Functions – a.k.a. stateless methods
Namespaces – Language support
Data – Maps, arrays, sets, XML, JSON etc.
Polymorphism a la carte – Protocols, Haskell type classes
Managed refs – Clojure/Haskell refs (compose time and value , not mix)
Set functions – Libraries
Queues – Libraries
Declarative data manipulation – SQL/LINQ/Datalog
Rules – Libraries, Prolog
Consistency – Transactions, values

True abstraction isn’t hiding complexity but drawing things away – along one of the dimensions of who, what, when, where, why [policy&rules of the app.], how.
Abstraction => there are things I don’t need – and don’t want – to know.
Why – do explore rules and declarative logic systems.
When, where – when obj. A communicates with obj. B. => put a queue in between them so that A doesn’t need to know where B is; you should use Qs extensively.

Links to Keep

  • Incredibly Useful CSS Snippets –  “a list of CSS snippets that will help you minimize headaches, frustration and save your time while writing css” – few float resets, targetting specific browsers & browser hacks, cross-rowser transparency/min height/drop shadow, Google Font API, link styled by file type,

DevOps: Tools and libraries for system monitoring and (time series) data plotting

  • Hyperic SIGAR API – open-source library that unifies collection of system-related metrics such as memory, CPU load, processes, file system metrics across most common operating systems
  • rrd4j – Java clone of the famous RRDTool, which stores, aggregates and plots time-series data (RRD = round-robin database, i.e. keeps only a given number of samples and thus has a fixed size)
  • JRDS “is performance collector, much like cacti or munins”, uses rrd4j. The documentation could be better and it seems to be just a one man project but it might be interesting to look at it.

Clojure Corner

  • Alex Miller: Real world Clojure – a summary of experiences with using Clojure in enterprise data integration and analytics products at Revelytix, since early 2011 with a team of 5-10 devs. Some observations: Clojure code is 1-2 order of magnitude smaller than Java. It might take more time to learn than Java but not much. Clojure tooling is acceptable, Emacs is still the best. Debugging tools are unsurprisingly quite inferior to those for Java. Java profiling tools work but it may be hard to interpret the results. “[..]  I’ve come to appreciate the data-centric approach to building software.” Performance has been generally good so far.
  • Article series Real World Clojure at World Singles – the series focuses on various aspects of using Clojure and how it was used to solve particular problems at a large dating site that starting to migrate to it in 2010. Very interesting. F. ex. XML generation, multi-environment configuration, tooling (“If Eclipse is your drug of choice, CCW [Counter ClockWise] will be a good way to work with Clojure.”, “Clojure tooling is still pretty young [..]  – but given how much simpler Clojure is than most languages, you may not miss various features as much as you might expect!”)
  • StackOverflow: Comparing Clojure books – Programming Clojure, Clojure in Action, The Joy of Clojure, Practical Clojure – which one to pick? A pretty good comparison.
  • Clojure is a Get Stuff Done Language – experience report – “For all that people think of Clojure as a “hard” “propeller-head” language, it’s actually designed right from the start not for intellectual purity, but developer productivity.”

Posted in eclipse, General, j2ee, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , | 3 Comments »

Most interesting links of May

Posted by Jakub Holý on May 31, 2011

Recommanded Readings

Acceptance testing / Specification by example:

  • Gojko Adzic: Anatomy of a good acceptance test – an example of refactoring a bad acceptance test into a good one – good for learning about pitfalls and how a good one should look like
  • Gojko: Top 10 reasons why teams fail with Acceptance Testing – acceptance testing is great and brings lot of value but must not be underestimated; some of the problems are bad collaboration, focusing on “how” instead of “what,” confusing AT with full regression tests. Brief, worth reading.
  • Specification by Example: a love story (go directly to the PDF with the story): A nice, made-up story of going from low-level, workflow-based Selenium tests through similar Cucumber ones to true BDD tests describidng clearly what, not how – very well shows the point of specification by example and how it should (and should not) look like

(Enterprise) Java best practices:

  •  Clean code, clean logs: 10 brief posts on logging best-practices – nothing really new here for me but in total it is a very good overview that every developer should know
  • Make Large Scale Changes Incrementally with Branch By Abstraction – Continuous integration doesn’t work well with branches but as this article shows, you can manage even large-scale refactorings without branches using “branch by abstraction,” an approach reminding me of Fowler’s “strangler application” (an incremental replacement of a legacy system). The idea is: 1. Create an abstraction over the part of code to be changed;  2. Refactor the code to use it; 3. Implement new functionality using the new way / step by step move old functionality too, the abstraction layer delegating either to the new or old implementation … . It may be more work but: 1) your software is always working and deliverable; 2) (side-effect) in the end it will be more decoupled

Git:

  • John Wiegley’s Git from the bottom upp (31p, Git 1.5.4.5, PDF) – a useful explanation of the fundamentals of Git, i.e. how it is constructed and how it works, which makes it much easier to understand how to  use it properly (recommended by Pål R.). Reading the The Git Parable first may be a good idea for an easy introduction into the fundamentals, though absolutely not necessary. This document introduces very well the important Git concepts (blob, index, commit, commit names such as branches, reflog) and how they cooperate to provide the rich set of functionality it has. It also explains well the value and usage of rebase. Among others I’ve appreciated the tip to use checkout, branch -m <new-branch> master, branch -D instead of the much more dangerous reset –hard and the tip to use stash / stash apply to create daily backups of your working tree in the reflog (with clearing it with ‘git reflog expire –expire=30.days refs/stash‘ instead of stash clear). Also git diff/log master..[HEAD] for reviewing work done in the current branch and and git diff/log ..master for checking the changes since the last merge/rebase after a fetch are interesting.

Tools:

  • The secret power of bookmarklets – bookmarklets are an indispensable tool for every developer who works with web applications (to fill in test data, speed up log in, …), yet I’m sometimes surprised by meeting people who don’t know or use them; this blog explains them nicely, links to some useful ones and some useful tools for building them

Recommended Books

  • (*****) Implementing Lean Software Development: From Concept to Cash by Mary Poppendieck, Tom Poppendieck – A great introduction into lean thinking (the values and principles it is build upon), clearly communicated with the help of “war stories”. I absolutely recommend it to anybody interested in lean/agile.
  • (**** ) Agile Project Management with Scrum (Microsoft Professional) by Ken Schwaber – Even though you can’t understand Scrum without experiencing it, this book full of war stories will help you to avoid many Scrum implementation pitfalls and to understand its mantra of “the art of the possible” and will show you how to adapt Scrum to various situations. It’s very easy to read thanks to its format of brief case studies organized by topics (team, product owner, …).

Favourite Quotes of the Month

@unclebobmartin: Cleaning code does NOT take time. NOT cleaning code does take time.

Posted in General, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , | Comments Off

CKEditor: Collapsing only 2nd+ toolbar rows – howto

Posted by Jakub Holý on April 1, 2011

Normally CKEditor (v3.5.2) hides/shows all the toolbar buttons when you press the collapse/expand button but I needed to always show the first row with “basic tools” and only collapse the second and following rows with advanced functionality tool buttons. CKEditor doesn’t have proper support for that but there is a simple workaround.

Update: Example solution (CKEditor 3.6.1) published, see the changes done or download the full source and open _samples/replacebyclass.html.

Read the rest of this entry »

Posted in General | Tagged: , , | 5 Comments »