Posts Tagged ‘DevOps’
Posted by Jakub Holý on May 7, 2015
I have derived the Docker container docker-grafana-influxdb-cloudwatch that bundles Grafana dashboards, InfluxDB for metrics storage, and runs cloudwatch-to-graphite as a cron job to fetch selected metrics from AWS CloudWatch and feed them into the InfluxDB using its Graphite input plugin. It is configured so that you can run it in AWS Elastic Beanstalk (the main problem being that only a single port can be exposed – I therefore use Nginx to expose the InfluxDB API needed by Grafana at :80/db/).
Read the rest of this entry »
Posted in General | Tagged: cloud, DevOps, Docker, ops | Comments Off on All-in-one Docker with Grafana, InfluxDB, and cloudwatch-to-graphite for AWS/Beanstalk monitoring
Posted by Jakub Holý on April 30, 2015
The first Continuous Delivery and DevOps Conference in Oslo is over. It was nice to see so many people interested in the topic. I would have preferred more practical talks of the “how we did it” type over the “why” type but it was OK, though next year I would prefer flatMap. Here are my highlights:
- Atmel is using a physical robot to plug and connect a particular configuration of circuit boards to test; your automated testing challenges cannot be greater than theirs!
- Continuous Delivery decreases the risk of outage and time-to-recovery while enabling faster innovation, correlates with higher profits; No efficiency improvement will outperform cycle time reduction
- Estimation pathologies; focus on value rather than costs
- Stop talking about requirements, they are fake; they’re just beliefs about what may add value to customers. Use hypothesis instead!
- Cisco: Most of the tools increasing productivity (and some innovation) were produced by engineers in their “spare” time; slack time is thus crucial
- How does Cisco grow professionalism : optimise for the 10% best, not the 10% weakest developers; slack time; make everything visible; encourage code reviews but avoid making them mandatory; see the slide
- CALMS: Culture, Automation, Lean, Measurement, Sharing. The pillars of devOps
- Cisco invested a lot in crafting their build system, tailored test frameworks, and emulators to be able to get quick and quality feedback – because it pays off
- “Make you own build system” says @olvemaudal at @CoDeOSL. IME this is inevitable for non-trivial projects, and a good investment.
- Unleash: Feature Toggles server and Java/Node client by FINN.no
- “They asked for a report while they actually need just a list of data, the result of a simple SQL query; have we listened to them, we would have wasted hours creating a report in the report framework with logos and all the crap.”
Posted in General | Tagged: continuous_deployment, DevOps | Comments Off on My Highlights from Continuous Delivery and DevOps Conference 2015
Posted by Jakub Holý on May 31, 2014
- Monolith – from The Codeless Code – fables and koans for the SW engineer – the Monad monolth #Haskell #fun
- http2 explained (pdf, 27 pages) – cons of http 1 (huge spec / no full impl., wasteful use of TCP <=> latency [x spriting, inlining, concatenation, sharding]) => make it less latency sensitive, fix pipelining (issue a req before previous one finished), stop the need for ever increasing # connections, remove/reduce optional parts of http. Http2 is binary; multiple “streams” over 1 connection => much less conns, faster data delivery; header/data compression; [predictive] resource pushing; . Inspired by SPDY. Chrome and Mozilla will only support it over TLS, yay! (see also Is TLS Fast Yet? [yes, it is]) Promise: faster, more responsive web pages & deprecation of http/1 workarounds => simplified web dev.
- exercism.io – crowd-sourced good code mentorship – get an exercise, implement it in any of the supported language(s), submit and get feedback, repeat; when finished, you too can comment the same excercise submitted by others while working on your next assignment. Languages include Clojure, JS, Scala, Python, Haskell, Go, Elixir, Java, and more.
Podcasts (FP & related)
- Cognicast (also @ iTunes) – Clojure, FP, etc.
- Functional Geekery (@ iTunes) – A podcast on Functional Programming, covering topics across multiple languages.
- Mostly λazy…a Clojure podcast by Chas Emerick
- Giant Robots Smashing into other Giant Robots – “a weekly technical podcast discussing development, design, and the business of software development”
- Software Engineering Radio (@ iTunes) – “The goal is to be a lasting educational resource, not a newscast. Every two to four weeks, a new episode is published that covers all topics software engineering. Episodes are either tutorials on a specific topic, or an interview with a well-known expert from the software engineering world.”
- EngineerVsDesigner – design insight (@ iTunes) – product design podcast – the latest digital design news, tips & tricks, Q&A, and an industry special guest
- core.async walkthrough for practicioners (12/2013; original title “Core.Async”) – Tim Baldrige 40m talk from Conj is a must-watch for anybody that wants to understand and use core.async. Examples from and beyond the talk are at GitHub. Highlights: While take! takes a callback, <!! wraps that into a promise so that it blocks and returns the value. Thread is similar to future, running the code asynchronously, but returns a channel instead of a promise. So does go, only it does not use real threads but simulated light-weight ones via a state machine and parking. Tim shows the different buffers (normal, dropping = earliest values kept, sliding = latest valus kept). Other topics: alts!!, …
Related vids by Tim: Core.Async Channel Internals, Core Async Go Macro Internals – Part I, Core Async Go Macros – Part II
- Clojure, Leiningen and Functional Programming for Java developers– implementing a queue in a functional way
- ownCloud – your own Dropbox/Google Drive, run on your server – sharing files between devices / PCs / web, syncing calendar and contacts, collaborative editing of documents (ODF)
- Mailpile – “A modern, fast web-mail client with user-friendly encryption and privacy features.”, to be self-hosted on a PC, RaspberryPI, USB stick
- Blackhole – role-based ssh proxy – an app that enables you to manage what users can ssh to what server as a particular user, from users’ point of view this is a ssh proxy; useful if many people need access to many servers but you do not want to add them all as users on those servers.
- Wuala – Secure Cloud Storage – Backup. Sync. Share. Access Everywhere. – Dropbox alternative, secure by default
- fb-flo – Facebook’s live-coding tool
- owncloud.org – self-hosted Dropbox-like service with calendar and contact sync and more
Posted in General, Languages, Tools, Top links of month | Tagged: clojure, cloud, DevOps, e-mail, haskell, mentoring, podcast, privacy, security, sharing, ssh | Comments Off on Most interesting links of May ’14
Posted by Jakub Holý on March 5, 2014
Ansible 1.5 has partial support for recursive copy of files:
- the synchronize module, using rsync
- cons: does not support group=, owner=
- -C and –diff – it does not print diff of the files changed; when running ansible with -v, it will print output of rsync’s –itemize-changes, i.e., for each changed file/dir, something like “<f.st…… conf/httpd.conf\n” (< = file uploaded, s = size changed, t = timestamp changed, . = this attribute has not been changed)
- the copy module
- -C –diff – it only reports “changed” without naming the changed files or showing diffs (unless there is only one changed file)
- the local_action module, used to run rsync manually (essentially the same as synchronize but more control)
- So the only way to do a kind of recursive copy with working diff is to use
copy with with_fileglob for each directory and subdirectory 😦
To check differences (without diff) manually:
rsync -e ssh -vrnc --itemize-changes source/dir myuser@myserver:/opt/dest/
where v= verbose, r = recursive, n = dry-run, c = check based on checksum, not timestamp+size; a typical output for a changed file is
<fcsT...... httpd.conf (< = to be uploaded, f = it is a file, c = checksum differ, s = size differ, T = timestamp would be updated).
Posted in Tools | Tagged: ansible, DevOps | Comments Off on Recursive Copy In Ansible 1.5 And –diff
Posted by Jakub Holý on March 4, 2014
Few tips for troubleshooting Ansible, based on my brief experiences with Ansible 1.4 (read: do not rely on this info too much).
Run ansible-playbook in the verbose mode
ansible-playbook -vvvv ... will provide you with plenty of details of what is going on. (Notice that additional v:s, starting from none, add more detail.)
Check out Ansible sources and use the ./hacking/test-module script – see Developing Modules.
Read the rest of this entry »
Posted in Tools | Tagged: ansible, DevOps | Comments Off on Ansible Troubleshooting Tips
Posted by Jakub Holý on February 24, 2014
It is tricky to manage JBoss with a provisioner such as Puppet or Ansible because its
domain.xml contains not only rather static configuration but also sections that change quite often such as
deployments. So how can we manage the static parts of
domain.xml with f.ex. Ansible while still enabling developers to deploy at will via
jboss-cli (and thus changing the
<deployments> sections of the file)? Here is one possible solution, based on extracting the sections from the current file and merging them into the template.
Read the rest of this entry »
Posted in General | Tagged: ansible, DevOps, jboss | Comments Off on Handling Deployments When Provisioning JBoss domain.xml (With Ansible)
Posted by Jakub Holý on February 16, 2014
I have created a small project to demonstrate some features of Ansible, the new DevOps hotness, including Vagrant VMs for running Ansible and for testing the configuration. Either go straight to
or continue reading the copy & paste here.
This project has three things of interest:
- A non-trivial Ansible configuration that demonstrates couple of useful features and tricks
- A Vagrant/VirtualBox virtual machine with Ansible & co. to make it easy to run it (even on Windows)
- Another VM that can be used to test the configuration
And of course all the plumbing that makes them work together. It might be therefore a good base for Ansible projects of your own.
Read the rest of this entry »
Posted in Tools | Tagged: ansible, DevOps | Comments Off on Demonstration of Ansible Features With Control & Test VMs
Posted by Jakub Holý on December 31, 2013
- HBR: Want to Build Resilience? Kill the Complexity – a highly interesting, thought provoking article relevant both to technology in particular and the society in general; f.ex.: more security features are bad for they make us behave less safely (risk compensation) and are more fragile w.r.t. unexpected events. “Complexity is a clear and present danger to both firms and the global financial system: it makes both much harder to manage, govern, audit, regulate and support effectively in times of crisis. [..] Combine complex, Robust-Yet-Fragile systems, risk-compensating human psyches, and risk-homeostatic organizational cultures, and you inevitably get catastrophes of all kinds: meltdowns, economic crises, and the like.” The solution to future financial crisis is primarily not more regulation but simplification of the system – to make it easier to police, tougher to game. We also need to decrease interconnectednes (of banks etc.), one of the primary sources of complexity. Also a great example of US Army combatting complex, high-risk situations by employing “devil’s advocates / professional skeptics” trained to help “avoid the perils of overconfidence, strategic brittleness, and groupthink. The goal is to respectfully help leaders in complex situations unearth untested assumptions, consider alternative interpretations and “think like the other”“.
- The Dark Side of Technology – technologies provide great opportunities – but also risks we should be aware of – they create a world of mounting performance pressure for all of us (individuals, companies, states), accelerate the rate of change, increasing uncertanity (=> risk of Taleb’s black swans). “All of this mounting pressure has an understandable but very dangerous consequence. It draws out and intensifies certain cognitive biases [..]” – magnify our perception of risk, shrink our time horizons, foster a more and more reactive approach to the world, the “if you win, I will lose” view, erode our ability to trust anyone – and “combined effect of these cognitive biases increases the temptation to use these new digital infrastructures in a dysfunctional way: surveillance and control in all aspects of our economic, social and political life.” => “significantly increase[d] the likelihood of an economic, social and political backlash, driven by an unholy alliance between those who have power today and those who have achieved some modest degree of income and success.”
Complexity theory: the more connected a system is, the more vulnerable it becomes to cascades of disruptive information/action.
- What Do Government Agencies Have Against 23andMe, Uber, and Airbnb? – innovative startups do not fit into established rules and thus bureaucrats do not know how to handle them and resort to their favourite weapon: saying no, i.e. enforcing rules that harm them (f.ex. France recently passed a law that requires Uber etc. drivers to wait 15 min before picking up a customer so that established taxi services have it easier; wot?!)
- Nonviolent communication in action – wonderful stories about NVC being applied in difficult situations with a great success
- Quora: Michael Wolfe’s answer to Engineering Management: Why are software development task estimations regularly off by a factor of 2-3? – a wonderful story explaining to a layman why estimation is hard, on the example of a hike from SF to LA
- Johannes Brodwall: Why I stopped using Spring [IoC] – worth to read criticism of Spring by a respected and experienced architect and developer; summary – dependency injection is good bug “magical” frameworks decrease understandability and encourage unnecessarily complex code => smaller code, , easier to navigate and understand and easier to test
- Misunderstanding technical debt – a brief discussion of the various forms of tech. debt (crappy code x misaligned design and problem domain x competence debt)
- Tension and Flaws Before Health Website Crash – surprising lack of understanding and tensions between the government and contractors on HealthCare.gov – “a huge gap between the administration’s grand hopes and the practicalities of building a website that could function on opening day” – also terribly decision making, shifting requirements (what news!), management’s lack of decision power, CGI’s blame-shifting. A nice horror story. The former head knew that they should “greatly simplify the site’s functions” – but the current head wasn’t able to “talk them out of it”.
- The Log: What every software engineer should know about real-time data’s unifying abstraction – logs are everywhere and especially important in distributed apps – DB logs, append-only logs, transaction logs – “You can’t fully understand databases, NoSQL stores, key value stores, replication, paxos, hadoop, version control, or almost any software system without understanding logs” – I have only read a small part but it looks useful
- What I Wish I Knew When Learning Haskell tl;dr
- Better Than Unit Tests – a good overview of testing approaches beyond unit tests – including “Automated Contract Testing” (ability to define a contract for a web service, use it to test it and to simulate it; see Internet of Strings for more info), Property-based Testing (test generic properties using random data/calls as with Quickcheck), Fault Injection (run on multiple VMs, simulate network failures), Simulation Testing as with Simulant.
- Use #NoEstimates to create options and deliver value reliably – a brief post with an example of an estimation-based vs. no-estimates project (i.e. more focus on delivering early, discovery)
- How Google Sold Its Engineers on Management – managers may be useful after all :-); a report about Google’s research into management and subsequent (sometimes radical) improvements in management style/skills and people satisfaction; I love that Google hasn’t HR but “people ops”
- Roy Osherove: Technical Disobedience – take nothing for granted, don’t let the system/process stop you, be creative about finding ways to improve your team’s productivity; there always is a way. Nice examples.
- Uncle Bob: Extreme Programming, a Reflection – a reflection on changes in the past ~ 14 years since XP that have seen many of the “extreme” practices becoming mainstream
- The Anti-Meeting Manifesto – essentially a checklist and tips for limitting meetings to minimum
- Ethics Daily – a freuently published collection of ethics-related links, articles, talks etc.
- British Library uploads one million public domain images to the net for remix and reuse – from 17th-19th century books, available at Flicker; it asks for help categorizing and documenting them, metadata of the images at GitHub (with links to Flicker); great project, interesting pics
- The Software Engineers’ Oath – lets bring ethics back to our daily (work) lives – respect the knowledge of others, use technology for good (!!!), keep learning, writing code for people, not ashamed to admit lack of knowledge, respect for privacy, obligation to make lifes of humans better, …
- David Pollak: Some musings on Scala and Clojure by a long time Scala dude (46 min) – a subjective but balanced comparison of Scala and Clojure and their strengths/weaknesses by the author of the Scala Lift framework (doing Scala since 2006, Clojure since 2013)
- Apache Sirona – a new monitoring tool in the Apache incubator – “a simple but extensible monitoring solution for Java applications” with support for HTTP, JDBC, JAX-RS, CDI, ehcache, with data published e.g. to Graphite or Square Cube. It is still very new.
- GenieJS – Ctrl-Space to popup a command-prompt for your web page, inspired by Alfred (type ‘ to see all possible commands)
A good #agile team considers their backlog inaccurate. It is merely a list of assumptions that must be tested & refined by shipping product
– @mick maguire 12/10
Ada Lovelace (1st program), Grace Hopper (1st compiler), Adele Goldberg (1st OO language), why would anyone think women aren’t in computing?
– @Dan North 12/11
There will always be a shortage of talented, self-motivated creative professionals who will unquestioningly follow orders.
– @Thomas K Nilsson 12/7
Estimation paradox = If something unpredictable happens, predict how long it will take to fix it
– me 12/7
IT systems can be inspired by AK-47 a.k.a. Kalashnikov. The rifle was purposefully designed to be simple and to be tolerant to imperfections in most parts; as a result, it required essentially no maintenance and was extremely reliable.
– summarized from Roman Pichlík’s Odkaz Michaila Kalašnikova softwarovému vývoji
Posted by Jakub Holý on September 5, 2013
Use case: Deploy a new version of a webapp so that all new users are sent to the new
version while users with open sessions continue using the previous version
(so that they don’t loose their precious session state). Users of the new version
can explicitely ask for the previous version in the case that it doesn’t work as expected and vice versa.
Benefits: Get new features to users that need them as soon as possible without affecting
anybody negatively and without risking that a defect will prevent users from achieving their goal
(thanks to being able to fall back to the previous version).
Read the rest of this entry »
Posted in General | Tagged: continuous_deployment, DevOps, java | Comments Off on Webapp Blue-Green Deployment Without Breaking Sessions/With Fallback With HAProxy
Posted by Jakub Holý on September 3, 2013
Are you using Puppet in the client-server setup and want to test the configuration for a particular node without actually changing it? You can do that by fooling Puppet Master into believing that a Vagrant virtual machine (VM) is that node and applying it there. The process is simple: you essentially only need to get the nodes’ cert/private key and supply them to Puppet (and likely make sure that the hostname puppet can be resolved from within the VM). Let’s see it in detail.
Read the rest of this entry »
Posted in Testing | Tagged: DevOps, puppet, vagrant | Comments Off on Test Puppet config of an existing node using Puppet Master via Vagrant