The Holy Java

Building the right thing, building it right, fast

Refactoring & Type Errors in Clojure: Experience and Prevention

Posted by Jakub Holý on October 6, 2015

While refactoring a relatively simple Clojure code to use a map instead of a vector, I have wasted perhaps a few hours due to essentially type errors. I want to share the experience and my thoughts about possible solutions since I encounter this problem quite often. I should mention that it is quite likely that it is more a problem (an opportunity? :-)) with me rather than the language, namely with the way I write and (not) test it.

The core of the problem is that I write chains of transformations based on my sometimes flawed idea of what data I have at each stage. The challenge is that I cannot see what the data is and have to maintain a mental model while writing the code, and I suck at it. Evaluating the code in the REPL as I develop it helps somewhat but only when writing it – not when I decide to refactor it.

Read the rest of this entry »

Posted in Languages | Tagged: , , | Comments Off on Refactoring & Type Errors in Clojure: Experience and Prevention

Nginx: Protecting upstream from overload on cache miss

Posted by Jakub Holý on October 1, 2015

These 2 magical lines will protect your upstream server from possible overload of many users try to access the same in cached or expired content: 

proxy_cache_use_stale updating timeout; # Serve the cached version even when outdated while refreshing it
proxy_cache_lock on; # Only one req is allowed to load/refresh the item, others wait / get the stale one 

You can verify this using Shopify’s Toxiproxy. 

❤ Nginx

Posted in [Dev]Ops | Comments Off on Nginx: Protecting upstream from overload on cache miss

Shipping a Refactoring & Feature One Tiny Slice at a Time, to Reduce Risk

Posted by Jakub Holý on September 1, 2015

You don’t need to finish a feature and your users don’t need to see it to be able to release and start battle-testing it. Slice it as much as possible and release the chunks ASAP to shorten the feedback loop and decrease risk.

My colleagues have been working on a crucial change in our webshop – replacing our legacy shopping cart and checkout process with a new one and implementing some new, highly desired functionality that this change enables. We have decided to decrease the risk of the change by doing it first only for product accessories. However the business wanted the new feature included and that required changes to the UI. But the UI has to be consistent across all sections so we would need to implement it also for the main products before going live – which would necessitate implementing also the more complex process used by the main products (and not yet supported by the new backend). And suddenly we had a a load of work that would take weeks to complete and would be released in a big bang deployment.

Such a large-scale and time-consuming change without any feedback from reality whatsoever and then releasing it all at once, having impact on all our sales – I find that really scary (and have fought it before). It is essentially weeks of building risk and then releasing it in a big kaboom. How could we break it down, to release it in small slices, without making the business people unhappy?

Read the rest of this entry »

Posted in SW development | Tagged: , | Comments Off on Shipping a Refactoring & Feature One Tiny Slice at a Time, to Reduce Risk

Running Gor, the HTTP traffic replayer, as a service on AWS Elastic Beanstalk

Posted by Jakub Holý on July 30, 2015

Gor is a great utility for replicating (a subset of) production traffic to a staging/test environment. Running it on AWS Elastic Beanstalk (EB) has some challenges, mainly that it doesn’t support running as a daemon and that there isn’t any documentation/examples for doing this. Well, here is a solution:

Read the rest of this entry »

Posted in [Dev]Ops | Tagged: | Comments Off on Running Gor, the HTTP traffic replayer, as a service on AWS Elastic Beanstalk

AWS ebextensions: Avoiding “Could not enable service” (or .. disable ..)

Posted by Jakub Holý on July 30, 2015

If you are adding a service entry to your .ebextensions/ config to run a service in AWS Elastic Beanstalk and it fails with either “Could not enable service [..]” or “Could not disable service [..]” (based on the value of ensureRunning), make sure that the service init.d file supports chkconfig, i.e. contains the comments it looks for.

Posted in [Dev]Ops | Tagged: | 1 Comment »

Fixing a mysterious .ebextensions command time out (AWS Elastic Beanstalk)

Posted by Jakub Holý on July 29, 2015

Our webshop, nettbutikk.netcom.no, runs on AWS Elastic Beanstalk and we use .ebextensions/ to customize the environment. I have been just trying to get Gor running on our leader production instance to replay some traffic to our staging environment so that we get a much richer feedback from it. However the container_command I used caused the instance to time out and trash the environment, against all reason. The documentation doesn’t help and troubleshooting this is hard due to lack of feedback and time-consuming. Luckily I have arrived to a solution.

Read the rest of this entry »

Posted in [Dev]Ops | Tagged: | Comments Off on Fixing a mysterious .ebextensions command time out (AWS Elastic Beanstalk)

AWS: Passing private configuration to a Docker container (via S3)

Posted by Jakub Holý on July 29, 2015

Philipp Garbe describes how to pass environment variables that you want to keep private to a public Docker instance run on Amazon Web Services (beanstalk or ECS) in his post How to Run HuBot in Docker on AWS EC2 Container Services – Part 3. The trick is:

  1. Put them into an env.sh file that you can source on S3 (and allow the appropriate EC2 IAM role to access it)
  2. As a part of your startup CMD, run aws s3 cp to fetch and then source it

Here is his example of the CMD from a Dockerfile:

CMD ["/bin/sh", "-c", "aws s3 cp --region eu-west-1 s3://your-bucket/env.sh .; . ./env.sh; bin/hubot --adapter slack"]

See the full source code in his GitHub repo. Thanks for sharing, Phillipp!

Posted in [Dev]Ops | Tagged: , | Comments Off on AWS: Passing private configuration to a Docker container (via S3)

Book Review & Digest: Release It! Design and Deploy Production-Ready Software

Posted by Jakub Holý on July 22, 2015

By Michael T. Nygard, 2007, ISBN: 978-0-9787-3921-8

My digest and review of the book.

Review

Of the books I have read, Release It! is the one I would require all “senior” developers to read (together with something like Architecting Enterprise Solutions: Patterns for High-Capability Internet-based Systems). Especially the first part on stability with its patterns and anti-patterns is a must read. Without knowing and applying them, we create systems that react to problems like a dry savannah to a burning match. I found also to next to last chapter, #17 Transparency, very valuable, especially the metrics and design of the OpsDB and observation practices.

One thing I have left out of the digest which is really worth reading are the war stories that introduce each section, they are really interesting, inspiring, and educational.

Extra Links

Stability

Stability x longevity bugs

ReleaseIt-patterns

Selected (anti)patterns

Stability antipatterns

Integration points

Integration point = call to a DB, WS, … . Stability risk #1.

Read the rest of this entry »

Posted in [Dev]Ops | Tagged: , , , | Comments Off on Book Review & Digest: Release It! Design and Deploy Production-Ready Software

NDC Oslo 2015: Talk notes, recommended talks (security, FP, etc.)

Posted by Jakub Holý on June 19, 2015

A great conference. A good deal of good talks.

To (perhaps) check later

Wednesday

Thursday

Friday

Keynote Data and Goliath ☆☆☆☆

Read the rest of this entry »

Posted in General | Comments Off on NDC Oslo 2015: Talk notes, recommended talks (security, FP, etc.)

Notes from Troy Hunt’s Hack Yourself First workshop

Posted by Jakub Holý on June 17, 2015

Troy Hunt (@troyhunt, blog) had a great, very hands-on 2-day workshop about webapp security at NDC Oslo. Here are my notes.

Highlights – resources

Personal security and privacy

Site security

Breaches etc.

To follow

Read the rest of this entry »

Posted in General | Tagged: , | 1 Comment »