Posts Tagged ‘architecture’
Posted by Jakub Holý on July 22, 2015
By Michael T. Nygard, 2007, ISBN: 978-0-9787-3921-8
My digest and review of the book.
Of the books I have read, Release It! is the one I would require all “senior” developers to read (together with something like Architecting Enterprise Solutions: Patterns for High-Capability Internet-based Systems). Especially the first part on stability with its patterns and anti-patterns is a must read. Without knowing and applying them, we create systems that react to problems like a dry savannah to a burning match. I found also to next to last chapter, #17 Transparency, very valuable, especially the metrics and design of the OpsDB and observation practices.
One thing I have left out of the digest which is really worth reading are the war stories that introduce each section, they are really interesting, inspiring, and educational.
Stability x longevity bugs
Integration point = call to a DB, WS, … . Stability risk #1.
Read the rest of this entry »
Posted in [Dev]Ops | Tagged: architecture, book, ops, performance | Comments Off on Book Review & Digest: Release It! Design and Deploy Production-Ready Software
Posted by Jakub Holý on July 31, 2013
This month focuses on languages and approaches (reactive programming, F#, Erlang, FP talks etc.), agile (need for speed, recommended books), Clojure/Linux/cloud tools and libs.
- The Need For Speed – the top 10 reasons for fast development flow (with time to market being one of the less important) – more learning, focus on the MVP, focus on the puprose/goal, happier customers/leadership, better quality (sic!), higher morale (I concur!), push for cotninuous improvement, “one of the only sustainable differentiators”; => “sense of urgency and motivation”; “[..] I continue to meet people and teams that not only move very slow, they don’t understand the relationship between speed and innovation, or speed and quality.”
- agile42 Summer Reading List 2013 – books recommanded by experienced people/agile experts – lot of interesting stuff! Topics: Communication and Coaching (f.ex. Practicing Nonviolent Communication), Business (How Great Companies Get Their Mojo from Maslow ,..), Learning From the Military, Agile and Technology (e.g. The Art of Action: How Leaders Close the Gaps between Plans, Actions and Results), Agile and Technology (f.ex. The people’s Scrum)
- Dan North: Are we nearly there yet? – optimize for time to business impact; SW dev as mountaineering (impossible to estimate correctly, many unknown details, dead ends, …); go fast – but sustainable; the tyranny of backglog (there are multiple paths to the top yet backlog defines only one; have you ever considerably changed it?) “Instead we could embrace the fact that today we always know more than we did yesterday, and that tomorrow we will know even more. We can take a fresh look up the mountain every time we pause to regroup, to plan.” => we ask 1) what gives us the shortest lead time to business impact? 2) what can help us to learn/invalidate more? 3) how to assure our stakeholders we are approaching the goal?
- Joel on Software: Software Inventory (7/2012) – a classical article about the evilness of software inventory (backlogs, issue trackers, undeployed features, …) ‘When I hear about product teams that regularly have “backlog grooming” sessions, in which they carefully waste a tiny amount of time and mental energy every day or every week thinking about every single feature which will never be implemented, I want to poke my eyes out.‘
- Job satisfaction self-test: Twelve questions that define a great place to work – check yourself how satisfied you are with your job (example questions: How well do I know what is expected of me? How often in the past seven days have I received recognition or feedback on my work? How much does the mission/purpose of the company make me feel like my work is important?)
- Coaching Anti-Patterns: Prescriptive Agile – a prescriptive coach “knows” what is “right” and forces it onto the client, without listening to her; instead, we should “Meet them where they are and leave them in a better place” => “[..] my first responsibility is to understand how and why they came to this practice. How did they come to this decision? What challenges does this approach address? What benefits are they optimizing for?” Worth remembering AND practicing
Languages, paradigms, approaches
- The Trouble with Erlang (or Erlang is a ghetto) – an objective criticism of Erlang by somebody who seems to be quite experienced with it; as I know very little about Erlang, it was interesting to learn about its weaknesses (no map/dict data structure, slow memory management, poor “JIT,” not usable for shared-state concurrency (contrary to e.g. Clojure), immutable state is not necessary and makes some things bad, inconsistent and ugly standard lib, …)
- Adventures in Multi Paradigm Programming – different programming paradigms/approaches re-implemented in Emacs Lisp – interesting 1) to see and compare these different approaches and 2) the flexibility of Lisp. Including iteration – Ruby’s map, Python’s list comprehension ([an_expression for x in list]), Scala’s default argument (_); search – Java’s for; arguments: direct, variadic (i.e. any number of args), named args; destructuring and pattern matching in CoffeScript/OCaml style; Haskell-like monads; objects with mixins; namespaces.
- Why bugs don’t like F# – no nulls, immutable data, strong type system, composition of small functions, asynchronous programming abstractions, higher-order functions over collections (no off-by-one), units of measure
- IBM high-fives Netflix open-source tools – it is interesting to see the spreading of Netflix’s open source tools for better cloud infrastructures; f.ex. “Karyon, is what Netflix calls the base container for applications and services built using the NetflixOSS ecosystem; Eureka is mid-tier load balancing; Hystrix controls interactions between myriad distributed services to nip cascading failures in the bud; and Ribbon is a Remote Procedure Call library.”
- ZeroMQ instead of HTTP, for internal services (with implementation in Clojure) – an interesting idea of using ZeroMQ – the sockets on steroids library – instead of HTTP in a way compatible with existing HTTP routing libs; advantages of ZeroMQ: automatic retrial (=> can restart the target service withou noticing), speed, reuse of a connection. The trick is to send a http-like structure (i.e. with method, uri, body) and pass that to Compojure or similar (update: there are now Clojure/core.async bindings for ZeroMQ)
- Joel on Software: Victory Lap for Ask Patents – killing a bad Microsoft patent request in 15 minutes – Ask Patents is a new StackExchange site that enables experts to look at SW patent requests and point to previous existing works that invalidate them; as Joel describes in his successful patent kill story, it is not difficult at all. Hopefully this will manage to really help the patent office and hit woul-be patent trolls hard! #victory
- The Dangers Of “Gamification” In Education by Kathy Sierra (a former game designer, a trainer of trainers at Sun, author of the Head First book series) – gamification is often regarded as something very desirable that will improve our lives; however, as Kathy discusses, it has also dark sides and, applied unappropriately, can actually decrease our intrinsic motivation (therefore it should be nearly never used in e.g. education)
- Choosing an OSS license doesn’t need to be scary (by GitHub) – a human-readable overview of OSS licenses; you should always assign a license to your GitHub account (Add A License can help with that; otherwise it is considered to be “all rights reserved” and you are not giving back to the community (I use the same as Clojure, Eclipse Public License)
- FunctionalTalks.org – “Brilliant people giving brilliant talks on functional programming” – f.ex. Wilkes Joiner: Functional Reactive Programming, Alexander Gounares: All your cores are belong to us, Katie Miller: Superhero monads, Bryan O’Sullivan: Running A Startup On Haskell, Rich Hickey: Introduction To Clojure, John A. De Goes: Building a Data Science Platform in Scala and many more.
- Types vs. Tests: An Epic Battle? – “Amanda Laucher and Paul Snively debate solving problems through types and tests using different approaches.” – can type system replace tests or vice versa? Interesting intro into the discussion for me. Using F#, Scala & more. Same claims: types don’t pay out so much for “small” codebases but scale better than tests. Types – Tests is a spectrum, not two single extremes. When a property should hold “for all,” a type would be a good match. Inductive types (Scala, Haskell?) can become quite complex, dependant types (as in Coq) would be much nicer [if I got that right].
- Paul Irish on Web Application Development Workflow (via M. Noddeland) – if you need to do some web development but are not up to date on the state of art, this might be useful – an overview of tools, utilities, services by a Googler and the person behind Modernizr, HTML5 Boilerplate, Yeoman etc. Including effective shell & dotfiles.GH, better ssh via .ssh/config and authorized_hosts, the all-in-one dev/build tool Yeoman with live reload, BrowserStack for testing, LocalTunnel to easily share anything running locally, Chrome Dev Tools support for SASS and testing devices (emulate touch events, screen sizes), JetBrains’ WebStorm, sharing tools via setapp.me. A genous idea to use GoogleAnalytics to track usage of features in a CLI app!
Other Interesting Stuff
Azul Systems’ high-performance JVM on the Vega architecture (from The Trouble with Erlang (or Erlang is a ghetto)) looks very interesting:
The other night I tweeted “If you’re looking for a language that gets multicore concurrency right, look at how Azul implemented Java on their Vega architecture” and I definitely stand by that. Azul is a company that got a lot of smart hardware and software people together and had them work on designing a custom system which would scale to hundreds of CPU cores (up to 768 of them), heaps that topped 500 GB (up to 768GB), and had the GC pause only 10-20ms at a time. The realtime performance characteristics Azul managed to eek out of their system lead them to often describe their GC as “pauseless”.
- Discussion: How core.async compares to agents, future and promise? – future/promise: 1 producer, 1 value, multiple consumers; agent: an unbounded queue of functions mutating a single value, with multiple producers and consumers (reading the latest value produced); channel: multiple 1:1 producers/consumers, i.e. a value can only be taken once from the channel, using a bounded queue (=> slow consumers can block fast producers). As mentioned elsewhere, channels is a relatively low-level abstraction and other things can be built on the top of it.
- Clojure Tradeoffs (design implications and why you should care) – perhaps not very unbiased but interesting anyway (shared-memory over other computing paradigms, i.e. message-passing, dynamic over static, speed over convenience, composition over IoC, …)
- Rich Hickey’s post introducing core.async with its Go-like channels as a better alternative to a collback hell (I know everybody has already read it but it is still an important link :))
- Tools etc.
- Faster Clojure Startup with Class Data Sharing – use JVM’s capability to include any classes in its boot image and include clojure in it
- lein-ancient – checks for outdated dependencies and plugins => run “lein ancient :all”
- lein-try – a Leiningen plugin that enables you to try a library in a REPL in the context of your project without having to add it to project.clj; simply run “lein try clj-time 0.5.1” and then in the REPL “(require ‘[clj-time.core :refer :all])” and e.g. “(date-time 1986 10 14)”
- Lemur: tool to launch a Hadoop job locally/on EMR from a job definition file + actions before/after
- Emacs: sexp fold/expand is very useful for exploring source code (hide all but the first lines of all top-level forms with hs-hide-all) – the built-in hs-minor-mode can hide/show all, or hide/show/toggle one but the keys for it are cumbersome; hideshow-org makes it possible to toggle hide/show with TAB, while preserving the original TAB behavior (it does the normal TAB first only only if nothing changes does it expand/fold); very useful!
- devdocs.io (via @palruud): “an all-in-one API documentation reader for [web] developers,” navigable via keyboard – JS, HTML, CSS, DOM, DOM events, jQuery, Underscore.js
- Kilim – a message-passing framework for Java that provides ultra-lightweight threads and facilities for fast, safe, zero-copy messaging between these threads.
- AssertJ – a library of assertions similar to fest-assert but providing a richer set of assertions (nicer API then fest-assert, according to a friend)
- NetflixOSS – Netflix, the online streaming gigant, has open-sourced many fascinating components of its cloud infrastructure such as Karyon, a blueprint for web-ready components with many features (monitoring,…), Genie/Hadoop as a Service, Servo for monitoring, Archaius for configuration management – too many to list. Check out Chris Fregly’s fluxcapacitor, a demo distributed application that uses many of the components
- Tools to keep a daemon running:
Posted by Jakub Holý on June 22, 2013
Highlights from Dan North‘s excellent, inspiring, and insightful talk Patterns of Effective Delivery at RootConf 2011. North has a unique take on what agile development is, going beyond the established (and rather limitied and rigid) views. I really recommend this talk to learn more about effective teams, about North’s “shocking,” beyond-agile experience, and for great ideas on improving your team.
The talk challenges the absolutism of some widely accepted principles of “right” software development such as TDD, naming, the evilness of copy&paste. However the challenge is in a positive way: it makes us think in which contexts these principles really help (in many) and when it might be more effective to (temporarily) postpone them. The result is a much more balanced view and better undestanding of their value. A lot of it is inspired by the theory (and practice) of Real Options.
What are Patterns of Effective Delivery?
- Patterns – strategies that work in a particular context – and not in another (too often we forget the context and to consider the context where a strategy doesn’t work / is contra-productive); beware: a part of the context is the experience of the developer; for unexperienced devs it might be better to just stick to a process and applying TDD etc. all the time than trying to guess when they are appropriate and when not without having the experience to decide it right
- Effective – optimize for something: volume of SW produced? time to market? learning/discovery? certanity? user experience?
- Delivery – get stuff that is useful out of the door; software is not important, the utility it provides is; know why you write the SW to be able to get better at it
Read the rest of this entry »
Posted in General | Tagged: agile, architecture, effectivity, favourite, lean, patterns, review, team | Comments Off on Patterns of Effective Delivery – Challenge Your Understanding Of Agile (RootsConf 2011)
Posted by Jakub Holý on November 30, 2011
- Recommended Reading by Poppendiecks – an excellent selection, starting with Lean from Trenches, Management 3.0, Specification by Example, The Lean Startup etc.
- Eric Allman says that Programming Isn’t Fun Any More because problem solving has been replaced with learning, configuring, and integrating tons of libraries, frameworks, and tools and many people agree with that (as discussion on reddit proves). In other words we tend to go for any benefit we can have without considering the costs and for “easy” solutions without considering the true enemy: complexity. Perhaps we should always listen to the Rich Hickey’s Simple Made Easy talk before we add a lib/tool/framework?
- Dean Wampler claims that functional programming can bring the joy back – “[..] a functional language, Scala, Clojure, Haskell, etc. will greatly reduce the amount of code you create. That won’t solve the problem of trying to integrate with too many libraries, but you’ll be less tempted. I also believe those libraries will be less bulky, etc.“
- Few quotes from a related article by M. Taylor: To put it another way, libraries make excellent servants, but terrible masters. | [..] frameworks [..] do keep their promise of making things very quick and easy … so long as you do things in exactly the way the framework author intended | On libraries: [..] we all assume (I know I do) that “plug in solutions X1 and X3″ is going to be trivial. But it never is — it’s a tedious exercise in impedance-matching, requiring lots of time spent grubbing around in poorly-written manuals [..] | On the effect of language choice: [..] different languages, with their different expressive power and especially their different culture, yield very different experiences.
- To sum it up: Choose your tools and libraries wisely and always mind the global complexity. More usually means worse.
- Java Magazine – Adam Bien: Stress Testing Java EE 6 Applications (page 41+) – do developer stress testing! Using: JMeter, VisualVM to find out resource consumption and behavior in the application, VisualVM’s Sampler profiling tool [cca 20% overhead], a webapp to extract metrics from GF (STM)
- Java Magazine – Polyglot Programming on the JVM (page 50; excerpt from The Well-Grounded Java Developer) – why you should consider polyglot programming and how to decide whether to use it and what languages to pick, f.ex.: “These [Java’s] qualities make the language a great choice for implementing functionality in the stable layer [of the polyglot programming pyramid]. However, these same attributes become a burden in the middle and upper [lower, DSL, on the linked image] tiers of the pyramid; for example: Recompilation is laborious; Static typing can be inflexible and lead to long refactoring times; Deployment is a heavyweight process; Java’s syntax is not a natural fit for producing DSLs.” “There is a wide range of natural use cases for *alternative languages*. [after identifying such a UC] You *now need to evaluate* whether using an alternative language is appropriate.”
- Intrusion Detection for Web Apps – Detection Points – If security is a concern of your web application then you should build intrusion detection into the application f.ex. leveraging the OWASP AppSensor project. The key is to detect malicious/unexpected behavior and proactively do something such as locking the user out or alerting the admins. The page linked above lists some common suspicious behaviors such as the use of multiple usernames, unexpected HTTP command/method, additional/duplicated data in request. Worth checking out!
- Yammer Moving From Scala to Java– Scala is a cool language but sometimes its cost is higher than the benefits. Snippets from the post: “…the friction and complexity that
comes with using Scala instead of Java isn’t offset by enough productivity benefit or reduction of maintenance burden …”. “Scala, as a language, has some profoundly interesting ideas in it. […] But it’s also a very complex language. The number of concepts I had to explain to new members of our team for even the simplest usage of a collection was surprising: implicit parameters, builder typeclasses, ‘operator overloading’, return type inference, etc. etc.” (It’s claimed that only library authors need to know some of that but if it’s a part of library APIs, the users need to understand it too.) Notice that the author isn’t saying “Scala is bad” but only that Scala isn’t the best balance of their needs at this time, as Alex Miller put it*
: The text wasn’t intended for publication and it is a private opinion of a Yammer developer, not the company itself. You should read the official Yammer’s position
where Coda puts it into the right context.
- Opportunistic Refactoring by Martin Fowler – refactor on the go – how & why
- Michael Feathers: Getting Empirical about Refactoring – gather information that helps us understand the impact of our refactoring decisions using data from a SCM, namely File Churn (frequency of changes, i.e. commits) vs. Complexity – files with both high really need refactoring. Summary: “If we refactor as we make changes to our code, we end up working in progressively better code. Sometimes, however, it’s nice to take a high-level view of a code base so that we can discover where the dragons are. I’ve been finding that this churn-vs.-complexity view helps me find good refactoring candidates and also gives me a good snapshot view of the design, commit, and refactoring styles of a team.“
UIs and Web Frameworks
- Devoxx 2011 – WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks (slides) – the authors did extensive performance testing of some of the most popular web frameworks. Of course it’s always hard to guess how general their results are, if/how they apply to one’s particular situation, and if they aren’t distorted in some way but it’s worth for their approach alone (AWS with its CloudWatch monitoring, WebDriver, additional measurement of page load with HAR and a browser plugin). In their particular tests GWT scored best, followed by Spring MVC, with JSF and Wicket lagging far behind (especially the MyFaces implementation). Conclusion: A web framework may have strong impact on performance and scalability, if they are important for you then do test the performance early with as realistic code and load as possible.
- JSF2 – Benchmark datatable by N. Labrot, 2/2011 – performance comparison of PrimeFaces 2.2.1, IceFaces 2.0, Richfaces 4.0.0M4 on a simple page with Ajax. I do not trust any benchmark that I don’t fake myself (for there are always too many factors that influence the conclusions to be drawn) but it’s interesting anyway – and perhaps a good thing to do before you decide for a JSF component library.
- Alex MacCaw: Asynchronous UIs – the future of web user interfaces and the Spine framework – users in 2011 shouldn’t anymore wait for pages to load and operations to complete, we should build asynchronous UIs where changes to the UI are performed immediately while a request to the server is sent in the background, similarly to sending e-mail in GMail, which returns at once displaying a non-intrusive “Sending…” notification. As a user I very much agree with Alex.
- Matt Raible’s 20 criteria for evaluating web frameworks, 2010 (detailed description, here’s a brief list) – Matt’s results are disputable and as he himself says you should always do your own evaluation and spikes but the criteria are pretty useful: Developer Productivity, Developer Perception, Learning Curve, Project Health, Developer Availability, Job Trends, Templating, Components, Ajax, Plugins or Add-Ons, Scalability, Testing, i18n and l10n, Validation, Multi-language Support (Groovy / Scala), Quality of Documentation/Tutorials, Books Published, REST Support (client and server), Mobile / iPhone Support, Degree of Risk.
Don’t use MongoDB via @nicolaiarocci – a (fake?!) bad experience with MongoDB – the text is not credible (the author is anonymous, s/he doesn’t explicitely state which version of MongoDB they used, the 10gen CTO can’t find a matching client and any evidence for some of the issues mentioned) but it gives context for the read-worthy response from the 10gen CTO, and a post that nicely explains how to correctly design for MongoDB. A comment about MongoDB experience at Forsquare: “Currently we have dozens of MongoDB instances across several different data clusters storing over a TB of data and handling 10s of thousands of requests per second (mostly reads but the write load is reasonably high as well).Have we run into problems with MongoDB along the way? Yes, of course we have. It is a new technology and problems happen.Have they been problematic enough to seriously threaten our data? No they have not.“
- Martin Fowler on Polyglot Persistence – the are when will be choosing persistence solution with respect to our needs instead of mindlessly picking RDBMS is coming. Applications will combine multiple, specific solutions, f.ex. we could pick Redis (key-value) for caching, MongoDB (document DB) for product catalog, Neo4J (graph DB) for recommendations, RDBMS for financial data and reporting… (of course not all in one project!). Polyglot persistence will come at a cost (complexity, learning) – but it will come because the benefits are worth it – performance, data storage model and behavior more aligned with the business logic (NoSql databases ofer various models and tradeoffs and thus we can find a much better fit than with general-purpose RDBMs).
Talks & Video
- Adam Bien’s JavaOne talk Java EE 6: The Cool Parts (1h) – absolutely worth the time – a very practical fly through the cool features of Java EE (eventing, ..), most of the time is spent actually coding. Don’t forget to check also the interesting discussion below the video (JEE and other frameworks, Java FX and JSF 2, …).
- Jurgen Appelo’s keynote How to Change the World at Smidig 2011 is well done and highly useful. We all strive to change the world around us – as consultants we want to make our clients more agile, as team members we want to make our Scrum teams more self-organizing, as employees we want to help building knowledge-sharing and open culture, … . However it isn’t easy to influence or change people and culture and if we aren’t aware of all the dimensions of a change (system, individuals, interactions, environment) and how to work along each of them, we are much less likely to succeed. The knowledge and experience that Jurgen shares with us can help us a lot in having an impact. You can also download the slides and change management questions.
- Project X: What is being a programmer like? (5min) If ever again a non-geek asks you what you as a developer are doing, just show him this short and extremely funny video (created by my ex-employer – perhaps they estimated how much time and energy developers loose trying to explain it to normal people and decided to prevent this great waste :-))
- RSA Animate – Drive: The surprising truth about what motivates us (10 min) – entertaining and enlightening; once we’ve enough money to cover our needs, it’s autonomy (self-direction), mastery, and purpose what motivates us (money actually decrease our performance). Now this is a great evidence for lean/agile – for they’re based on making people self-directing and encourage mastery (as in continous integration and top quality to enable steady pace). Autonomy enables engagement as does a higher purpose (“make the world a better place”) – Steve Jobs with his visions was able to provide such a purpose. Atlassian’s FedEx Days are a good example of what engagement and benefits autonomy brings.
- Simon Sinek: How great leaders inspire action (18 min, subtitles in 37 languages) – do you want to succeed, to change the world around you for the better, to start a new company? Then you must start by communicating “why” you do what you do, not “what” – like M. L. King, bro Wrights, and Apple. Very inspiring! (More in his Why book.)
Links to Keep
Refactoring is like advertising: it doesn’t cost, it pays.
– Mary & Tom Poppendiecks, Implementing Lean Software Development, p.166
Posted by Jakub Holý on November 21, 2011
Having [automated] unit/integration/functional/… tests is great but it is too easy for them to become a hindrance, making any change to the system painful and slow – up to the point where you throw them away. How to avoid this curse of rigid tests, too brittle, too intertwined, too coupled to the implementation details? Surely following the principles of clean code not only for production code but also for tests will help but is it enough? No, it is not. Based on a discussion on our recent course with Kent Beck, I think that the following three principles below are important to have decoupled, easy to evolve tests:
- Tests tell a story
- True unit tests + decoupled higher-level integration tests (-> Mike Cohn’s Layers of the Test Automation Pyramid)
- More functional composition of the processing
Read the rest of this entry »
Posted in Testing | Tagged: architecture, opinion, tdd, Testing | 3 Comments »
Posted by Jakub Holý on November 20, 2011
What I like most at conferences are reports of someone’s failure to do or implement something for they’re the best sources of learning. And How to Fail with Drools (in Norwegian) by C. Dannevig of Know IT at JavaZone 2011 is one of them. I’d like to summarize what they learned and extend it for introduction of a tool, framework, or library in general based on my own painful experiences.
They decided to switch to the Drools rule management system (a.k.a. JBoss Rules) v.4 from their homegrown rules implementation to centralize all the rules code at one place, to get something simpler and easier to understand, and to improve the time to market by not requiring a redeploy when a rule is added. However Drools turned out to be more of a burden than help for the following reasons: Read the rest of this entry »
Posted in General | Tagged: architecture, opinion | 1 Comment »
Posted by Jakub Holý on July 31, 2011
- Martin Fowler, M. Mason: Why not to use feature branches and prefer feature toggles instead, when branches can actually be used (video, 12min) – feature branches are pretty common yet they are a hindrance for a good and stable development pace due to “merging hells”. With trusted developers, feature toggles are a much better choice.
- M. Fowler: The LMAX Architecture – Martin describes the innovative and paradigm shaking architecture of the high-performance, high-volume financial trading platform LMAX. The platform can handle 6 million orders per second – using only a single java thread and commodity hardware. I highly recommend the article for two reasons: First, it crashes the common view that to handle such volumes you need multithreading. Second, for the rigorous, scientific approach used to arrive to this architecture. The key enablers are: 1) The main processing component does no blocking operations (I/O), those are done outside (in other threads). 2) There is no database – the state of the processor can be recreated by replaying the (persistent) input events. 3) To get further from 10k to 100k TPS they “just” wrote good code – well-factored, small methods (=> Hotspot more efficient, CPU can cache better). 4) To gain another multitude they implemented more efficient, cache-friendlier collections. All that was done based on evidence, enabled by thorough performance testing. 5) The processor and input/output components communicate without locking, using a shared (cyclic) array, where each of them operates on sum range of indexes and no element can ever be written by more than one component. Their internal range indexes do ever only increase so it is safe to read them without synchronization (at worst you will get old, lower value). The developers also tried Agents but found them in conflict with modern CPUs for their require context switch leading to emptying of the fast CPU caches.
Updated: Martin has published the post titled Memory Image which discusses the LMAX approach to persistence in a more general way.
- S. Mancuso: Working with legacy code with the goal of continual quality improvement – this was quite interesting for me as our team is in the same situation and arrived to quite similar approach. According to the author, the basic rule is “always first write tests for the piece code to be changed,” even though it takes so much time – he justifies it saying “when people think we are spending too much time to write a feature because we were writing tests for the existing code first, they are rarely considering the time spend elsewhere .. more time is created [lost] when bugs are found and the QA phase needs to be extended”. But it is also important to remember when to stop with refactoring to get also to creating business value and the rule for that is that quality improvements are done only with focus on a particular task. I like one of the concluding sentences: “Constantly increasing the quality level in a legacy system can make a massive difference in the amount of time and money spend on that project.”
- Uncle Bob: The Land that Scrum Forgot – Scrum makes it possible to be very productive at the beginning but to be able to keep the productivity and continue meeting the expectations that are thus created we need to concentrate on some essential technical practices and code quality. Because otherwise we create a mess of growing complexity – the ultimate killer of productivity. Uncle Bob advices us what practices and how to apply to attain both high, sustainable productivity and (as required for it) high code quality. It’s essential to realize that people do what they are incented to do and thus we must measure and reward both going fast and staying clean.
How do we measure quality? There is no perfect measure but we can build on the available established metrics – coverage, # (new) tests, # defects, size of tests (~ size of production code, 5-20 lines per method), test speed, cyclomatic complexity, function (< 20) and class (< 500) sizes, Brathwaite Correlation (> 2), dependency metrics (no cycles, in the direction abstraction).
The practices that enable us to stay clean include among others TDD, using tools like Chceckstyle, FindBugs to find problems and duplication, implementing Continuous Integration.
- Getting Started: Testing Concurrent Java Code – very good and worthy overview of tools for checking and testing of concurrent code with links to valuable resources. The author mentions among others FindBugs, concurrent test coverage (critical sections examined by multiple threads) measurement with IBM’s ConTest, multithreaded testing with ConTest (randomly tries to create thread interleaving situations; trial version – contact the authors for the full one) and MultithreadedTC (which divides time into “ticks” and enables you to fine-configure the interactions)
- The top 9+7 things every programmer or architect should know – quite good selection of nine, respectively 7 things from the famous (on-line available) books 97 Things every programmer/architect should know.
- Beans and noses – J. Spool reveals the First Rule of Consultancy: “No matter how much you try, you can’t stop people from sticking beans up their nose.” In other words, sometimes your clients decide to do some very unwise thing and no amount of reasoning can discourage them from that (quite understandably, as already the way they got to this decision defies logic). “The only thing I can do in a beans-and-noses situation is wait. Wait until the bean is in its final resting place.” Then you ask the person how it is working for him and “… if sticking a bean deep into their nostril doesn’t meet the very high expectations they’d had, I can now start talking alternative approaches to reaching those expectations.” Already before you can actually ask them about their expectations, in some cases (50:50) this discussion can lead them to realize they could achieve them with an alternative, less painful approach. Now, if you have read up to this point, you clearly have enough time, so go an read the article because it’s really orth it!
Posted in Languages, Testing, Tools, Top links of month | Tagged: architecture, continuous_deployment, java, Jenkins, legacy, quality, scrum | Comments Off on Most interesting links of July
Posted by Jakub Holý on March 27, 2011
I’ve recently stumbled upon an interesting Eclipse project called ObjectTeams/Java (OT/J), which promises improved reusability and maintenance and support for evolvable architectures by creating well-encapsulated bundles of behavior – modules – that can be applied to existing classes (via AOP), when they are in the appropriate context of interaction (and not simply always, as is the case with AOP). An example application is the addition of NonNull constraint to JDT via an OT/Equinox plugin, without the necessity to modify JDT’s base classes. I’ve decided to write down my discoveries as the project is missing a clear and brief introduction (though it has otherwise very good documentation). This blog borrows heavily from . Read the rest of this entry »
Posted in Languages, Tools | Tagged: AOP, architecture, eclipse, java | Comments Off on Introduction to ObjectTeams/Java, a Role-Based Approach to Modularity With AOP
Posted by Jakub Holý on July 21, 2010
I’d like to share with you my experience with the Sun Certified Enterprise Architect for the Java Platform, Enterprise Edition 5 (SCEA 5)  certification. There was a couple of unclear things regarding the assignment and its deliverables and I’ve learned some interesting things (mostly about hardware estimation and deployment environments such as the “clouds”), both of which may be of an interest to somebody aiming at this certification. I only know that I passed but not how well, so my way of doing things, though sufficient, may not be the best one.
Read the rest of this entry »
Posted in j2ee, Languages | Tagged: architecture, certification, java, javaEE, sun | 13 Comments »
Posted by Jakub Holý on September 25, 2009
PatternTesting is a mature open-source project that enables you to automatically check at the compile- or run-time that architectural/design/implementation decisions are implemented and bad practices avoided in the code. The main differences from tools like FindBugs and PMD are that you can implement tests spanning multiple files (classes) and that aside of compile-time checks there are also run-time checks (using AOP with ApsectJ) to do things like “ensuring that there are no more than 10 calls to the database per user case” and providing better error messages in the case of e.g. IOException.
Read more in a The Server Side’s introductory article and on the project’s page.
I haven’t yet the chance to try this out but it really intrigues me.
Posted in Languages, Testing | Tagged: AOP, architecture, java, patterns, Testing | Comments Off on PatternTesting: Automatically verifying the good application of architectural/design patterns in code