Posts Tagged ‘Git’
Posted by Jakub Holý on October 31, 2013
- Google engineers insist 20% time is not dead—it’s just turned into 120% time – it is interesting to see how has this evolved; “I have done many engineering/coding 20% projects and other non-engineering projects, with probably 20-40% producing “real” results (which over 7 years I think has been more than worth it for the company). But these projects are generally not rewarded.” [highlight mine]
- The Worst Daily Scrum Ever – a story whose bad part is a too common reality; if energy is low, nobody asks for / offers help, and people only report status / plans then you are doing the daily scrum wrong and should stop now (but it also documents a nice example of a good, effective scrum)
- Why Responsive Design is a Waste of Time – a refreshingly critical take on responsive design; the author now aknowledges that it is sometimes worth the pain but the points are still valid – responsive design requires (lot of) extra work, the attempt to create a one-size-fits-all site of course adds considerable complexity (having two separate simple frontends might be better than one that is too complex), also many sites are good enough as they are (especially taking into account the capabilities of mobile browsers)
- How to lose $172,222 a second for 45 minutes – i.e. your bugs are likely not so serious after all :-) A financial company screwed big and ended up bankrupt. The cause? Chaotic DevOps, not removing old unused code, reusing a feature flag instead of creating a new one, lack of monitoring. The story in short: They deployed new trading code but failed to notice that it has not been deployed to one of the 8 servers; due to the flag reuse, the old, 10 years unused code has been activated instead. Due to the lack of monitoring they did not notice the cause, tried to roll back while leaving the flag enabled thus effectively activating the bad code on all the servers. => have proper automated and self-checking deployments, delete old code, do not repurpose old switches.
- 40 Inappropriate Actions to Take Against an Unlocked (Windows) PC – good tips for promoting security and having fun at the same time; I shall keep this at hand :-)
- How to go about ‘proving’ why dynamically typed languages are better – a cultivated and interesting discussion; as argueed, thinking in this direction is itself wrong and in different contexts, different languages will be more appropriate. I also like Phil Lord’s “Programming is a highly fashion-centric occupation for any number of reasons.” and “For me, the main advantage is that you are not forced to build complex hierarchies just to support the type system ([..]), and that having only a few abstractions makes it worthwhile adding lots of functions operating over them.” and L. Petit’s “IMHO, the question is irrelevant. It implicitly assumes that statically typed vs dynamically typed is a black / white choice, and that either ‘static wins over dynamic’ or ‘dynamic wins over static’ will be a true statement whatever the context.” Also a good observation that types are only a subset of function contract enforcement and one of possible implementations.
- The Failure of Governmental IT (Learnings From HealthCare.gov) – links to a few really good articles about the problems with governmental IT in general and my summary of them
- Inside the Arctic Circle, Where Your Facebook Data Lives – the designs of data centers used to be proprietary secrets until Fb developed its own and open-sourced them, enabling many Asian manufactures to start creating cheaper datacenters and thus started a revolution in this domain. Facebook’s data centers are not general purpose but suitable ot the kind of work they need, but it is still widely applicable. Cool to see how they use natural conditions to get energy needs down and make HW that fits best their needs – that is what I call innovation!
- Academia.edu (via @RiczWest) – a rich source of free research papers – just register as an independant researcher; also lean/agile/systems thinking and other interesting topics
+ Username +
- Johannes Brodwall: A canonical web test – a simple web app end-to-end smoke test – using an embedded Jetty, a test DB (preferably in-memory), WebDriver to test it (simple: browser.get(“/people”), assertThat(browser.findElement(<person id>.contains(<person’s name>)); simple, nice, useful
- LearnGitBranching – an online game to learn branching & rebase in git; use the menu in the lower-right corner to navigate between the levels etc. You can also execute commands “show goal”, “hint”, “level” to navigate around; pretty cool and great for learning the command line commands
Society & people
Not a typical topic I share here but really worth it this time.
- The ocean is broken – a saddening story worth reading to learn what does your tuna sandwitch cost and where does all the plastic we use end up. From a sailing trip from Melbourne to US where there were plenty of fish (and birds) 10 years ago – and 2 this year, killed to a noticable degree by huge fishing ships that catch tuna – and kill and throw away all the other “junk” fish. Nowadays fish are replaced by plastic and other waste that actually prevents usage of the engine unless somebody can watch for dangerous nets and ropes leftovers. Earth, where are you falling to?
- The Guardian: Why have young people in Japan stopped having sex? – sad and interesting to observe what happens when the system is set up so that people “can’t be bothered” to have inter-sexual relationships, partnership, and children. Japan needs a good deal of systems thinking to fix its broken society where women do not want children because it would cost them their career and neither men nor women are willing to subjects themselves to the social pressure and demands associated with relationships.
- The Guardian: 29 million people enslaved, says first global index on slavery – welcome to the 21st century! The leading slave countries are India (14M), China (3M), Pakistan (2M). Also, slaves are building the world cup stadion in Qatar.
- They’re Taking Over! – how we managed to destroy sea ecosystems and helped the now unstoppable return of jellyfish – Jellyfish are evidently very veried and extemely resilient and have been hold at bay only by rather complex ecosystems that we managed to destabilize so much that Jellyfish are on their way back to ruling all the sees again (destroying the rests of the ecosystems – i.e. fish – on the way); a sad future for the sea, Earht, and us
- LightTable 0.5.9 got elementary paredit commands
- Amazonica: A comprehensive Clojure client for the entire Amazon AWS api – best with a REPL!
- Clojure Understood: the Rush Hour Platform – application of the design best practices promoted in the Clojure community (separation of concerns, simplicity, …) on a project – “highly accurate vehicle traffic simulations”; I have only started reading it but it looks highly interesting
- Clojure content at InfoQ – articles, news, interviews, presentations etc.
- A bitter taste [of EuroClojure] – we as a community must embrace diversity and stop fostering our egos by mocking other than our holy editor (and, I would add, by mocking other languages and in general mocking whatever); respect and open minds, please!
- Garden – Clojure alternative to scss/less – still needs time to mature and gain tooling support but it is cool that it exists
- C. Emerick’s Austin, the ClojureScript REPL over nREPL – 1) Start nREPL (via lein repl, from your editor…), 2.a) Execute austin’s exec to start a ClojureScript REPL in it, backed by headless PhantomJS or a real browser – or, alternatively, 2.b) create a C.S. REPL connected to your webapp, as described in the browser-connected-repl-sample. You should watch this 8 min demo.
- WhiteHat Aviator – A Safer Web Browser – WhiteHat, a well-known security company, has released a browser that aims at improving privacy by preventing user tracking (f.ex. but not sending referral URL) and blocking ads even at the cost of occassional slight discomfort, i.e. something that the mainstream browsers are not interested in. So far for OS X only.
- EnvPane – a preference pane for environment variables for Mac OS X 10.8 (Mountain Lion) – set env. vars for GUI/terminal apps, no need to log out upon change
Weinberg: Bureaucracy is what we do when we no longer remember why we are doing it
- via Ben Simo, no source specified so it may be fake but anyway it is valid
Posted in General, Languages, Testing, Top links of month | Tagged: aws, clojure, ClojureScript, design, earth, ecology, Git, innovation, privacy, science, scrum, security | 4 Comments »
Posted by Jakub Holý on June 30, 2013
Agile, process, SW dev, people etc.
- Real Options—a Mindset – an intro into the Real Options approach, which has been quite a hot topic and a transformational way of thinking for a number of inspiring people (Dan North, Liz Keogh etc.). “Real Options help us to better make decisions and commitments with three simple principles: Options have value. Options expire. Never commit early unless you know why.” We can “pay” to keep our options open longer, i.e. to avoid commiting prematurely.
- Demystifying the CHAOS report’s claim of ~ 1/2 features being unused: the Standish Group’s CHAOS report has been often quoted for its “finding” that a large percentage of features in applications is never/rarely used. However this claim seems to have never been confirmed, their “research” is reportedly not very scientific and not publicly available for scrutiny. Critique by Laurent Bossavit (2013), Jorge Aranda’s Standish, the CHAOS report, and science. Thx to @smalltalk80 for pointing this out! However there is one research, Online Experimentation at Microsoft, that supports the claim, in a different context but the same problem applies to features: “Evaluating well-designed and executed experiments that were designed to improve a key metric, only about one-third were successful at improving the key metric!”
- Why Yammer believes the traditional engineering organizational structure is dead – small teams, small projects (2-10 people, 2-10 weeks), no separation into front/middle tier/backend team (=> communication, design obstacle); have instead people specializing in these areas and construct feature teams from them based on the actual needs; engineers, not managers do eng. decisions; all aligned via focusing on the same 3 key metrics. Small projects => constant sense of urgency (and excitement): Often very long projects cause engineers to lose track of the end goal. Think of it in terms of hiking: start fresh & excited, get tired and losing track of the goal, excited again at the end => cut out the middle part, keep them in the exciting state where they can measure progress and see it visually; it’s the only way to maintain urgency and morale. Focus: people alwasy work only at one (short) project at a time (there are special bug-fixing teams for maintenance tasks with people rotating in&out).
- Agile development is more culture than process – Why thinking of agile as culture and not just process explains resistance and difficulty in teaching and learning the approach – and should be taught so => 1. Underscore agile values that motivate practice; 2. Identify organization values that compete with agile values, conflict of values; 3. Be sensitive to culture shock.
- Mark Zuckerberg’s Letter to Investors: ‘The Hacker Way’ (quite long, you might want to read only “The Hacker Way” part at the end) – about Facebook’s “unique culture and management approach” – “Hackers believe that something can always be better, and that nothing is ever complete.” “Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once.” “Instead of debating for days [..], hackers would rather just prototype something and see what works.” “Hacker culture is also extremely open and meritocratic.” “Many of our most successful products came out of hackathons, [..].” <=> five core values: Focus on Impact (focus on solving the most important problems, be good at finding the biggest problems to work on); Move Fast (“[..] if you never break anything, you’re probably not moving fast enough.”); Be Bold (“Building great things means taking risks.”); Be Open (=> effort to make as much info as possible visible to all); Build Social Value (“[..] Facebook exists to make the world more open and connected, and not just to build a company. “)
- Dave Nicolett: I know how to tie my shoes – on the difficulty of convincing people to try unfamiliar software development techniques – “People change the way they operate when they are experiencing some sort of inconvenience or negative feedback. As long as things are going along reasonably well, people don’t go out of their way to change the way they work.” (with few exceptions) You can learn to tie your shoes in a split second, but why to invest the effort? You’d need to set aside assumptions, suppress habits, practice. You can argument there are many inconveniences (bugs, criticism for slow delivery, …) but “Unfortunately, that’s all pretty normal, and most people in the software field are accustomed to it. They don’t see it as a problem that calls for them to change their practices. Most of them probably have a hard time visualizing a different reality.” => Maybe that’s the reason there’s been no satisfactory answer to the question of how to convince people to adopt different practices. We shouldn’t be trying to convince people to do anything. We should be helping people solve their problems and achieve their goals. If they are satisfied with the outcomes they achieve using their current methods, then there is no problem to solve.
- Kent Beck: Pace of Progress = Pace of Feedback – ‘”The pace of my progress is completely constrained by the pace of my feedback”. If I want to go faster, it’s hard to achieve by going faster. I can almost always optimize my feedback loop, though.’ “The second lesson from this episode is that it’s not just the duration of the feedback loop that matters, it’s also the quality. All week I was working in tiny little iterations. Without producing useful information, though, those iterations could be as small or as large as I liked, I was still just going to spin my wheels.” => “The next time I seem to be going slow, I’m going to look at my whole feedback loop–duration, quality and my ability to respond to the information.“
- What Google Has Learned About How to Hire People – interview results have no relation to actual performance on the job: “We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship. It’s a complete random mess.” “Instead, what works well are structured behavioral interviews, where you have a consistent rubric for how you assess people, [..]” ‘Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.”’ Link to an interesting book, Hiring Geeks That Fit.
Cool tech stuff
- The Elixir language – Clojure + Ruby + Erlang - a functional meta-programming aware language built on top of the Erlang VM; a dynamic language with flexible syntax with macros support that leverages Erlang’s abilities to build concurrent, distributed, fault-tolerant applications with hot code upgrades. First-class support for pattern matching, polymorphism via protocols, etc. (via @bodil)
- Random Testing seems to be gaining popularity and looks very interesting; at NDC Oslo, John Hughes has presented how QuickCheck, which generates random sequences of API calls, has been successfully used to find bugs in the Riak DB and a file system that a human would never think of, and Stuart Halloway has presented simulation testing with Simulant, which runs predefined actions according to a probabilistic model (e.g. 100 traders, each having 1h mean time between trades and mean traded amount 100, the test runs for 4 simulated hours). Something worth exploring!
- Dmytro Navrotskyy’s collection of Frontend Development resources and learning materials for tools (grunt, unused css detection,..), best practices (Atomic Design, …), JS/CSS frameworks, typography, animation, visualization, useful on-line services, and many more (via Herman Schistad)
- The Secret To 10 Million Concurrent Connections – The Kernel Is The Problem, Not The Solution: To have really fast SW, you need to implement your own core services (FS, net driver (packet handling), thread scheduling, ..) tuned for your app. You need to be aware of the clock-time cost of cache misses, memory access etc.. Custom solutions are times faster than what the general OS kernel can offer. => “data plane oriented system” Core areas and solutions for them: packet scalability, multi-core scalability (locks are expensive), memory scalability.
- M. Fowler: EmbeddedDocument – a pattern for working with JSON flowing in/out of our services (REST <-> JSON-friendly DB) without unnecessary conversions but with good encapsulation; naive approach: json -> object graph -> (processing) -> json; “In many of these situtiations a better way to proceed is to keep the data in a JSONish form, but still wrap it with objects to coordinate manipulation.” – use a lib to parse the JSON into a generic structure (e.g. a structure of lists, and maps/dicts) and store in a field of an object defining methods that encapsulate it – f.ex. for an Order we could have a method returning the customer and another computing the cost, accessing the underlying generic structure. The user of the wrapper object doesn’t need to know/care about the underlying structure. “The sweet spot for an embedded document is when you’re providing the document in the same form that you get it from the data store, but still want to do some manipulation of that data. [..] The order object needs only a constructor and a method to return its JSON representaiton. On the other hand as you do more work on the data – more server side logic, transforming into different representations – then it’s worth considering whether it’s easier to turn the data into an object graph.”
- ThoughtWorks’ Approach To Big Data Analytics – an inspiring, brief read. Some really good points such as “It’s not about Data. It’s about Insight and Impact” => “focus on the questions you’d love to answer for your business” => “changing big data from a technological problem to a business solution.” Also “The value of data is only realised through insight. And insight is useless until it’s turned into action.” Measure the value you gain at each step. See Introducing Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing by Ken Collier
- Wired.com, Nassim Taleb: Beware the Big Errors of ‘Big Data’ – in big data, noise has much stronger effect and in a large enough dataset we will always find spurious (i.e. false) relationships => beware! “Well, if I generate (by simulation) a set of 200 variables — completely random and totally unrelated to each other — with about 1,000 data points for each, then it would be near impossible not to find in it a certain number of “significant” correlations of sorts. But these correlations would be entirely spurious.”
- A Taste of Salt: Like Puppet, Except It Doesn’t Suck – a deescription of Salt and the tools around by an enthusiastic user with deep experience with Puppet. Highlights: Light-weight communication over ZeroMQ, very active community, simplicity, configuration is YAML, Salt-cloud can spin instances in EC2/Openstack/…, Salt-virt does the same for virtual machines (KVM/Xen/…), Salt-vagrant, Salt-monitor (work in progess) can ask all the server for their stats. “Having stood up a number of different configuration management systems across a wide variety of environments, I’ve yet to find a solution that’s as rapid to deploy, simple to scale, or as well architected as Salt.”
- Trash Your Servers and Burn Your Code: Immutable Infrastructure and Disposable Components – leveraging the lectures of PF to have a stable infrastructure – instead of updating servers, throw them away and create a new one from scratch (requires virtualization/cloud); this is something that Netlfix is doing and also Comoyo is moving towards
- Robin Ward: AngularJS vs Ember – a nice overview of the different approaches of the two; the author is strongly pro-Ember, claiming that AngularJS is much closer to low-end libraris like Backbone/Knockout and that you will often need the additional features of Ember. The comments provide the right countrweight to the biased post and form thus a good whole together.
- Scala Productivity. A Survey of the Community – people (that asnwered) seem to be productive with Scala right from the start
- After Your Job Is Gone – an interesting essay on the future, which, according to the author, we can already see happening, when technology will take away most of our work and we will not need to work all day. Not very optimistic, though (the author predicts few reach and many poor people).
- Clojure Cup 2013, Sept 28-29 – create something cool with Clojure/ClojureScript within 48h and perhaps win a price! #fun
- Clojure use in the industry – examples at an e-mail forum – Netflix, Puppet Labs (e.g. PuppetDB), UBS (talk), Deutsche Bank (talk, some details), Citigroup (reportedly “the largest private sector deployment of Clojure to date,” 11/2012), getprismatic.com (with frontend moving to ClojureScript; -> Why Prismatic Goes Faster With Clojure), Roomkey.com (details in a Relevance podcast), MastodonC.com (big data), Trend Micro, Walmart, beanstalkapp.com, ReadyForZero.com (50kLoC), www.cognician.com (20kLoC), World Singles (13kLoC) and more… (another similar thread)
- Stuart Sierra’s My Clojure Workflow, Reloaded (6/2013) – mainly about reloading changes into REPL, working around things that are not reloaded/left over => restart the app from scratch after significant changes => the app as a transient object => no global state, careful management of resources, :dev profile with :source-paths to a dir with user.clj (autoloaded by repl, pre-loading useful stuff) and dev util deps
- Adam Bard’s walk-through useful Clojure libs – f.ex. clojure.[data.[csv xml json] inspector java.shell java.browse xml], tools.logging, clojure.core.[match logic typed contracts ...]
- Juxt.pro: Jon Pither’s and Malcolm Sparks’ “network of experienced IT professionals who specialise in the Clojure programming language,” providing training, consulting, talks
- Anthony Grimes: The Clojure Community and Me (2011) – an exciting insight into the embracing and supportive Clojure community
- In Clojure-based Machine Learning: “Our backend is 99.4% coded in Clojure, and 66% of the team [of 3] had never programmed seriously in any Lisp, let alone Haskell or Prolog (heck, not even I (the remaining 33%) had actually tried anything non-mainstream for real in a big project!) Maybe some Ruby, and lots and lots of Java and C and C++. But they accepted the challenge after reading around and learning the basics, and 3 months later you couldn’t take Clojure from their prying hands.”
- J. Pither: TDD and Clojure – “If you were to create a shopping list of things you really want for your development experience then what would you put at the top?” => 1. rapid feedback on changes, 2. REPL (place to explore and to play with your code <=> TDD), 3. FP and Immutability (“FP and dynamic languages lead to a lot less code. There’s less ceremony, less modeling. Because you’re managing less code you do less large scale refactorings.” => TDD needed less), 4. Regression Tests (“It’s my current opinion that what you get left out of TDD once you have amazingly fast feedback and a REPL is regression testing.”)
- More beautiful and colorful git log, by Filipe Kiss (via @lcdutoit) You may also want to have a look at tig, which is a text-based UI for git with the default view similar to git log.
- jq (via lcdutoit) – sed/awk/grep for JSON – slice, filter, map, transform structured data
- SemanticMerge (Windows) – a Java-aware merge tool – free beta (I haven’t tried it)
- JetLang – a high performance java threading library for in-memory messaging, based upon Retlang (via @tastapod, used likely in a trading SW).
Agile [is] in NOT a process — it’s a philosophy.
- Joe Wroblewski in a comment to a blog post
Teach culture first, then process and techniques
- Jeff Patton in Agile development is more culture than process
If a candidate came telling me that s/he wanted to program only in, say, Java because that’s what s/he knows best and that s/he doesn’t really feel prepared or interested in learning and using, say, Clojure (or any other language, really), I wouldn’t hire her/him in a million years, no matter what language my project were using, and no matter how many thousands of candidates like this one I had at my disposal.
- José Antonio Ortega in Clojure-based Machine Learning
We shouldn’t be trying to convince people to do anything. We should be helping people solve their problems and achieve their goals. If they are satisfied with the outcomes they achieve using their current methods, then there is no problem to solve.
- Dave Nicolett in I know how to tie my shoes
Posted in General, Languages, Testing, Tools, Top links of month | Tagged: bigdata, clojure, DevOps, fp, frontend, Git, java, lean, performance, simulation, waste | Comments Off
Posted by Jakub Holý on May 31, 2011
Acceptance testing / Specification by example:
- Gojko Adzic: Anatomy of a good acceptance test - an example of refactoring a bad acceptance test into a good one – good for learning about pitfalls and how a good one should look like
- Gojko: Top 10 reasons why teams fail with Acceptance Testing – acceptance testing is great and brings lot of value but must not be underestimated; some of the problems are bad collaboration, focusing on “how” instead of “what,” confusing AT with full regression tests. Brief, worth reading.
- Specification by Example: a love story (go directly to the PDF with the story): A nice, made-up story of going from low-level, workflow-based Selenium tests through similar Cucumber ones to true BDD tests describidng clearly what, not how – very well shows the point of specification by example and how it should (and should not) look like
(Enterprise) Java best practices:
- Clean code, clean logs: 10 brief posts on logging best-practices – nothing really new here for me but in total it is a very good overview that every developer should know
- Make Large Scale Changes Incrementally with Branch By Abstraction – Continuous integration doesn’t work well with branches but as this article shows, you can manage even large-scale refactorings without branches using “branch by abstraction,” an approach reminding me of Fowler’s “strangler application” (an incremental replacement of a legacy system). The idea is: 1. Create an abstraction over the part of code to be changed; 2. Refactor the code to use it; 3. Implement new functionality using the new way / step by step move old functionality too, the abstraction layer delegating either to the new or old implementation … . It may be more work but: 1) your software is always working and deliverable; 2) (side-effect) in the end it will be more decoupled
- John Wiegley’s Git from the bottom upp (31p, Git 184.108.40.206, PDF) – a useful explanation of the fundamentals of Git, i.e. how it is constructed and how it works, which makes it much easier to understand how to use it properly (recommended by Pål R.). Reading the The Git Parable first may be a good idea for an easy introduction into the fundamentals, though absolutely not necessary. This document introduces very well the important Git concepts (blob, index, commit, commit names such as branches, reflog) and how they cooperate to provide the rich set of functionality it has. It also explains well the value and usage of rebase. Among others I’ve appreciated the tip to use checkout, branch -m <new-branch> master, branch -D instead of the much more dangerous reset –hard and the tip to use stash / stash apply to create daily backups of your working tree in the reflog (with clearing it with ‘git reflog expire –expire=30.days refs/stash‘ instead of stash clear). Also git diff/log master..[HEAD] for reviewing work done in the current branch and and git diff/log ..master for checking the changes since the last merge/rebase after a fetch are interesting.
- The secret power of bookmarklets - bookmarklets are an indispensable tool for every developer who works with web applications (to fill in test data, speed up log in, …), yet I’m sometimes surprised by meeting people who don’t know or use them; this blog explains them nicely, links to some useful ones and some useful tools for building them
- (*****) Implementing Lean Software Development: From Concept to Cash by Mary Poppendieck, Tom Poppendieck – A great introduction into lean thinking (the values and principles it is build upon), clearly communicated with the help of “war stories”. I absolutely recommend it to anybody interested in lean/agile.
- (**** ) Agile Project Management with Scrum (Microsoft Professional) by Ken Schwaber – Even though you can’t understand Scrum without experiencing it, this book full of war stories will help you to avoid many Scrum implementation pitfalls and to understand its mantra of “the art of the possible” and will show you how to adapt Scrum to various situations. It’s very easy to read thanks to its format of brief case studies organized by topics (team, product owner, …).
Favourite Quotes of the Month
@unclebobmartin: Cleaning code does NOT take time. NOT cleaning code does take time.
Posted by Jakub Holý on February 28, 2011
Articles, links etc.
- The Git Parable (thx to Alexander) – a good and easy to understand explanation of the story behind Git and thus also its main goals and concepts. It really helps in understanding what makes Git different from Subversion and thus empowers you to use it to its full capabilities and in accordance with its philosophy and not (painfully) against it. Though I’d appreciate little more coverage of merging and cooperation in the Git world.
- Some thoughts on stress testing web applications with JMeter (part 2) – what to measure, when to stop, what listeners to use, some practical tips e.g. for using remote slave JMeter instances, how to interpret the results (explanation of mean, std. deviation, confidence interval). By Nicolas Vahlas, 3/2010.
- Scalability Factors of JMeter in Performance Testing projects (conference paper, 2008) – what factors determine what number of virtual users (VU) a load test tool can efficiently simulate and experiments to determine the impact of those factors on JMeter. Each VU has an associated cost in terms of CPU and memory (send, process, store request/response etc.) => number of VUs depends on the HW and the complexity of the messages and protocol and on the load test script complexity.
- For example one commercial tool claims to support up to nearly 2k VUs on 1GHz PIII w/ 1GB but w/o stating the other factors.
- JMeter scalability experiments results (Def.: Scalability limit reached when additional VUs aren’t increasing the server’s load (i.e. the test tool has become the bottleneck)):
- Response time: “The optimal number of virtual users increases with increase in response time. It increases from around 180 virtual users to around 700 optimal virtual users when the response time changes from 500 ms to 1.5 seconds.”
- Application response size: “Application response size has a massive effect on the load generation capabilities of JMeter. The optimal # of virtual users drops from around 500 virtual users to a measly 115 virtual users when the application response size increases form 20 kb to 100kb.”
- Communication protocol (HTTP x HTTPS): The load generation capability decreases by 50% or more when the protocol is HTTPS
- HW used for the test machine: P4 2.4GHz, 2GB RAM.
- A. Bien: Can Stateful Java EE 6 Apps Scale? – Another Question Of The Week – the answer: Yes. A.B. always starts with Gateway+PDO and measures the performance/overhead with VisualVM, JMeter. Don’t forget that stateless applications usually just move the state and therefore scalability problem to the DB.
- Switching to Plan J – isn’t Scala good enough (yet)? – by Jonathan Edwards. Really an interesting read including the comments (well, the first half – esp. Martin, Stuart R., Vincent etc.) – though very impressive, is Scala too complex and do we need syntax subsets for normal people? The state of IDE support is bad but should be getting better.
- Opinion: The unspoken truth about managing geeks – a great article on corporate culture and psychology, which turn people into stereotypical geeks – invaluable advices for managing IT professionals. And it’s fun to read! (Well, at least if you’re a geek observeing the very things he speaks about around you.)
- ThoughtWorks Technology Radar, Januar 2011 – Scala up to Trial, DevOps, …. . Unchanged: ESB, GWT & RIA on hold, Apache Camel (integration library) on Trial. Other interesting: CSS supersets supporting variables etc. like LESS.
Quotes of the month
I’ve decided to add this occassional section to capture interesting or inspirational comments of famous as well as ordinary people.
- “if i learn, i end up going fast. if i just try to go fast, i don’t learn & only go fast briefly.” (Kent Beck’s Twitter, 2/11). Reflects nicely a discussion I recently had with a colleague :-) I also like the follow-ups:
- “The computer industry is the only industry that is more fashion-driven than women’s fashion.” – by Richard Stallman in Cloud computing is a trap, 2008-09-29. I might not agree with everything he says but there certainly is a seed of truth in that!
- “Shipping is a feature. A really important feature. Your product must have it.” by Joel Spolsky in The Duct Tape Programmer, 2009-09-23. This is something we should really keep in mind and explain to the product owners too :-)
Posted in General, Testing, Tools, Top links of month | Tagged: EJB, Git, javaEE, performance, scala, technology, Testing, trends | Comments Off
Posted by Jakub Holý on June 2, 2010
The most interesting stuff I’ve read in May, in no particular order. You can easily guess I’ve been working on performance troubleshooting this month :-)
- NoSQL is About… – all the things NoSql databases are said to be about (and perhaps are not) and a good overview of the different goals and thus also features of the various implementations
- Bulletproof of Mind Mapping: Overview, Benefits, Tips and Tools – the article not only introduces mind maps (a structured way of recording ideas, much less limited than lists) but also describes over 30 desktop and web-based MM tools, both free and commercial (some of the descriptions come from the SW’s web, some from the author – the distinction isn’t clear)
- Java vs. C Performance….Again. (9/2009) – When C(++) is better than Java, when Java is more appropriate, and common flaws in comparions methodologies/false arguments.
- Why Learning Git is really, really hard part 1 and part 2 with actual reasons – because it doesn’t care enough for usability (unusual commands, cryptic error messages, impossibly to go to a “simpler use mode”). I’m intrigued by distributed SCM systems and tired of not-so-easy branching & merging in SVN and its lovely problems with corrupted metadata (when you delete a folder…) and thus I was considering switching to Git that everybody is so excited about. I still plan that but these articles warned me that it may be not so painless and easy. A good read.
- Java VisualVM Blogging Contest results – the best posts -
- VisualVM – tool for profiling Java applications – nice, short intro with many pictures
- Analyzing Memory Leak in Java Applications using VisualVM
- (and others … )
- How to compute running mean/standard deviation - this page explains and in C implements an algorithm for computing a running estimate of mean and standard deviation, which minimizes accumulation of precision errors. A running estimation has the advantage that you do not need to store all the numbers and is thus suitable e.g. for continuous performance monitoring with a low memory overhead (buth the performance overhead of a division and multiplication it introduces is perhaps also something to consider – though for most application it’s negligible)
- (Java) Web performance in seven steps – a great article about the “management of performance” of a Web/JEE application from the definition of performance requirements up to continual performance monitoring with interesting war stories and links to various useful tools. I can sign the author’s maxim “measure, don’t guess!”. The Java monitoring API Java Simon mentioned in the article is worth a look.
Posted in General, Languages, Top links of month | Tagged: Git, java, monitoring, nosql, performance | Comments Off