The Holy Java

Building the right thing, building it right, fast

Posts Tagged ‘security’

Most interesting links of November ’13

Posted by Jakub Holý on November 30, 2013

Recommended Readings

Some interesting topics this time despite me spending lot of time on the Principles of Reactive Programming course: Java x Node.js, REST x other future-proof architectures, scary legacy code. Of course, also plenty of Clojure.

People, organizations, teams, development:

  • Chris Argyris (1923-2013): An Appreciation – Thinkers 50 – recently departed Ch. Argyris is a person whose work you should know, if a bit interested in learning and organizations and how they (dis)function; and since we all work in organizations and want our work to be pleasant, this means all of us. We all want to work in orgs that do double-loop learning, i.e. they actually evovle as they learn. “Argyris argued that organizations depend fundamentally on people and that personal development is and can be related to work.” Now stop and go read it!
  • Bob Marshall: The Antimatter Principle – “the only principle we need for becoming wildly effective at collaborative knowledge work” – can be summarized as “attend to folks’ needs” (importantly, including your own) => find out what people actually need, interpret their behavior (including anger, seemingly irrational or stupid requests etc.) in terms of needs; mastering this will make you excell in communication and effective work. Read the post to find out more.
  • It’s a state of mind: some practical indicators on doing agile vs. being agile – are you agile or are just “doing agile”? Read on ti find out, if you dare! F.ex. “Non Agile teams have a process that slows the review of the changes.” Cocnlusion: “An Agile mindset is just that – a state of mind, a set of values. Its a constant questioning, and an opening up to possibilities. Its a predisposition to produce great things.
  • Johannes Brodwall:  Humble architects – how to avoid being an architect that does more harm than good, as so many out there? Some tips: Don’t assume stupidity, Be aware that you might be wrong, Be careful with technology (i.e. simplicity beats everything; applies so much to us developers too!), Consistency isn’t as important as you think (or beware context), Tactical reuse in across systems is suboptimization (i.e. reuse has a cost), separate between (coding) rules and dogma (i.e. is that way unsafe, incomprehensible, or just a heresy w.r.t. a dogma?) Very valuable insights into creating good technical solutions and teams that work.
  • Liz Keogh’s The Dream Team Nightmare: a Review – a very good review of this adventure-style book about coaching “agile” teams through (around?) common pitfalls, provides a good base for deciding whether you shall read the book (Liz essentially says yes)
  • Fibonacci Kittens: One Idea One Commit – a short story of coming from biannual releases to frequent release of individual features; I link to this primarily to spread optimism, if this company managed it then, perhaps, we other can too?
  • The Eternal Struggle Between Business and Programmers – “Business: More features! Now! Programmers: More refactoring! Now!” How can we resolve this eternal conflict of needs? This post reveals how the two parties can find a common ground and mutual understanding (beware, everybody must give up on something) and thus work together rather than against each other.

Coding, architecture, legacy

  • Why the future is NOT RESTful – always refreshing to read something against the mainstream; “REST is not fit for the next generation of smart client applications because it has not been designed for smart clients.” According to the author, a smart client app stack needs: “1. persistence (storage and query), 2. documents/orm (conversion to tree-like datastructures), 3. data authorization (once authenticated), 4. pub/sub (websocket communications), 5. client db (client-side caching and querying), 6. templating (presentation level)” Meteor.js has nearly all but #3 thanks to mongodb (1+2), dpp (4), mongo on the client (5), spark (6). The author considers a similar but Clojure-based stack (with Datomic, Angular etc.) and looks at authorization possibilities. “Secured, personalised, CRUD operations are the future to a more simplified web.” We may agree or not but it certainly is worth reading.
  • Michael Feathers (of Working Effectively With Legacy fame): Unconditional Programming – “Over and over again, I find that better code has fewer if-statements, fewer switches, and fewer loops. Often this happens because developers are using languages with better abstractions. [..] The problem with control structures is that they often make it easy to modify code in bad ways.” Includes a nice example of replacing if-else with much more readable and safer code.
  • The Quality of Embedded Software, or the Mess Has Happened – an interesting and scary read about terrible spaghetti code (and hardware) that makes some Toyotas to accelerate when the driver tries to break; 11,000 global variables, the key function showing so high cyclomatix complexity that “makes it impossible not only to test but even maintain this program in any way.” Then 80k violations of the car industry coding standard cannot surprise. And a safety control that does not work. Interesting that a great manufacturer may have so terrible IT (and Toyota isn’t the only one).
  • The string type is broken – the String type is M. Feathers’ favourite example of a leaky abstraction – most languages fail to process/split/count less common Unicode characters properly, the fact that String is implemented as a series of bytes leaks through (UTF-16 langs like Java); worth reading to be aware of the limitations

Languages

  • Why I’m Productive In Clojure – some interesting points about simplicity, consciousness, interactive development, power without overwhelming fatures, etc. “With it [Clojure] I can always easily derive a solution to a particular problem from a small set of general patterns. [..] However, the number of ways that these concepts can be combined to solve all manner of problems appears to be inexhaustible.
  • Node.js at PayPal – PayPal is switching from Java to Node.js (among others to promote language consistency) and, as a part of that, has implemented the same app in Node and Java; results: Node was done earlier, had less code, performed better (though, as Daniel Kvasnička pointed out, “Comparing Node.js and servlet-based archs is not fair… compare Node with @vertx_project and you’ll get a whole different story ;)”; also, as Charles Nutter said, “The @PayPal numbers for their Java to Node move are absurd. A JVM app doing 1.8 pages/s isn’t slow…it’s broken.“)
  • IBM: Developing mobile apps with Node.js and MongoDB, Part 1: A team’s methods and results – also IBM has implemented the same (REST) app once with Java and DB2, once with Node and Mongo where Node+Mongo required less work and performed better; one of the great wins was having JSON as a native structure everywhere instead of transforming from/to it so Mongo is, in my opinion, an important factor in this particular case
  • Dynamics of Programming: Benefits of Scala in CS1 – reasons for and experiences with using Scala in an introductory computer science course, worth reading; some of the advantages over Java are consistency (.asInt on String and Double vs. casting/parsing, no “primitive” types), REPL with time inference good for learning, functional style enables focus on what rather than how; quite persuasive arguments

Security

  • The New Threat: Targeted Internet Traffic Misdirection – did you know that internet traffic to any site can be made to go through a particular server without anybody noticing? This has been observed repeatedly in the wild, for banks and other sites. Rather make sure you use strong encryption (NSA-approved, of course ;)).
  • A (relatively easy to understand) primer on elliptic curve cryptography – Everything you wanted to know about the next generation of public key crypto – you cannot just read this, you have to study it, which I did not; but it looks good and I guess the time will come when I will come back to it

Other

Clojure Corner

  • Stuart Sierra’s Component – a library for making it easier to implement Stuart’s reloadable workflow; a component is something that can be started, stopped, and depend on other components so that it is easier to do interactive REPL development
  • Logan Linn: Clojure/conj 2013 – a pretty good overview of the conference
  • Caribou – “the kernel of usefulness that has emerged from years of this basic practice“- a new Clojure web framework – seems to be interesting
  • Results of the 2013 State of Clojure & ClojureScript survey and drill-down into what features people want – the most interesting fact is how many more participants use Clojure in production than last year and perhaps also the relatively wide adoption of Datomic among the respondents. Light Table has become the 3rd most popular dev. env., after Emacs and Vim. Some of the most mentioned language features requested were types (=> core.typed, Prismatic’s Schema), better error reporting (=> slingshotdireclj-stacktraceio.aviso:pretty, etc.), debuger (though progress is being made)
  • Book: Clojure High Performance Programming
  • Improving Clojure Feedback : Stack Traces – making Clojure stacktraces more usable by filtering out noise and linking to relevant content – io.aviso:pretty,  io.aviso:twixt
  • Clojure Dev discussion: Hashing strategies – Executive summary – “In Clojure, however, it is far more common than in Java to use longs, vectors, sets, maps and compound objects comprised of those components (e.g., a map from vectors of longs to sets) as keys in other hash maps.  It appears that Java’s hash strategy is not well-tuned for this kind of usage.  Clojure’s hashing for longs, vectors, sets, and maps each suffer from some weaknesses that can multiply together to create a crippling number of collisions.” Ex.: An implementation of N-Queens took forever, spending most time on hash collisions. But be calm, smart Clojurians are working on a solution.
  • Datomic Pro Starter Edition – Datomic with all storages, Datomic Console, a year of updates, full Datomic programming model; limitations: no HA transactor, no integrated memcached, max 2 peers and 1 transactor

Tools/Libs

  • AirPair.com – a new site that enables developers to get help from other devs via remote pairing, code review, code mentoring etc. – a good opportunity to get help / help others (and earn something); I haven’t tried it but it sounds pretty interesting

MongoDB web stacks

  • Meteor: JS frontend + MongoDB backend with changes in the DB pushed live to the clients, i.e. MongoDB is used both as the “application server” and storage. It seems great for apps where users need to collaborate in real-time with each other, certainly great for quick proof of concepts etc.; worth checking out; it also comes with free (at least for start?) hosting so really good for prototyping – “an open-source platform for building top-quality web apps in a fraction of the time.” The intro screencast will give you a good overview (10 min).
  • Mean.io – MEAN (Mongo, Express, Angular, Node) stack Boilerplate – frontend, backend and storage using the same language and some of the most popular technologies (not that popular = best fit for you :)); it seems to be very new but since it just glues together 4 popular and documented technologies, that should not be an obstacle. There is an intro on the MongoDB blog.

Other

  • BusyBox (get latest win binary from Tigress.co.uk) – reportedly a better POSIX env for Windows than gow, Cygwin, et al.

Favourite Quotes

[..] no organization should exist unless it is “of service” to its employees, its customers, its community.
- @Tom_Peters 28/11

I hope you’ll agree that there is a certain amount of irony involved in having to write repetitive code
- Dmitri Sotnikov in Why I’m Productive In Clojure

Happy teams are productive teams but:

Morale is 95% a function of the prevailing system (the way the work works). Which in turn is a function of the prevailing collective mindset
- @flowchainsensei Nov 10th

Posted in General, Languages, Top links of month | Tagged: , , , , , , , , , , | Comments Off

Most interesting links of October ’13

Posted by Jakub Holý on October 31, 2013

Recommended Readings

  • Google engineers insist 20% time is not dead—it’s just turned into 120% time – it is interesting to see how has this evolved; “I have done many engineering/coding 20% projects and other non-engineering projects, with probably 20-40% producing “real” results (which over 7 years I think has been more than worth it for the company). But these projects are generally not rewarded.” [highlight mine]
  • The Worst Daily Scrum Ever – a story whose bad part is a too common reality; if energy is low, nobody asks for / offers help, and people only report status / plans then you are doing the daily scrum wrong and should stop now (but it also documents a nice example of a good, effective scrum)
  • Why Responsive Design is a Waste of Time – a refreshingly critical take on responsive design; the author now aknowledges that it is sometimes worth the pain but the points are still valid – responsive design requires (lot of) extra work, the attempt to create a one-size-fits-all site of course adds considerable complexity (having two separate simple frontends might be better than one that is too complex), also many sites are good enough as they are (especially taking into account the capabilities of mobile browsers)
  • How to lose $172,222 a second for 45 minutes – i.e. your bugs are likely not so serious after all :-) A financial company screwed big and ended up bankrupt. The cause? Chaotic DevOps, not removing old unused code, reusing a feature flag instead of creating a new one, lack of monitoring. The story in short: They deployed new trading code but failed to notice that it has not been deployed to one of the 8 servers; due to the flag reuse, the old, 10 years unused code has been activated instead. Due to the lack of monitoring they did not notice the cause, tried to roll back while leaving the flag enabled thus effectively activating the bad code on all the servers. => have proper automated and self-checking deployments, delete old code, do not repurpose old switches.
  • 40 Inappropriate Actions to Take Against an Unlocked (Windows) PC – good tips for promoting security and having fun at the same time; I shall keep this at hand :-)
  • How to go about ‘proving’ why dynamically typed languages are better – a cultivated and interesting discussion; as argueed, thinking in this direction is itself wrong and in different contexts, different languages will be more appropriate. I also like Phil Lord’s “Programming is a highly fashion-centric occupation for any number of reasons.” and “For me, the main advantage is that you are not forced to build complex hierarchies just to support the type system ([..]), and that having only a few abstractions makes it worthwhile adding lots of functions operating over them.” and L. Petit’s “IMHO, the question is irrelevant. It implicitly assumes that statically typed vs dynamically typed is a black / white choice, and that either ‘static wins over dynamic’ or ‘dynamic wins over static’ will be a true statement whatever the context.” Also a good observation that types are only a subset of function contract enforcement and one of possible implementations.
  • The Failure of Governmental IT (Learnings From HealthCare.gov) – links to a few really good articles about the problems with governmental IT in general and my summary of them
  • Inside the Arctic Circle, Where Your Facebook Data Lives – the designs of data centers used to be proprietary secrets until Fb developed its own and open-sourced them, enabling many Asian manufactures to start creating cheaper datacenters and thus started a revolution in this domain. Facebook’s data centers are not general purpose but suitable ot the kind of work they need, but it is still widely applicable. Cool to see how they use natural conditions to get energy needs down and make HW that fits best their needs – that is what I call innovation!
  • Academia.edu (via @RiczWest) – a rich source of free research papers – just register as an independant researcher; also lean/agile/systems thinking and other interesting topics
  • Writing Code? Know Your Boundaries – an inspiring way of thinking; we use many technologies in combination (HTML, CSS, JS, SQL, server-side language, …) and “the risk for picking the wrong tool for the job is strongest near the boundaries“; a discussion of the aforementioned boundaries with examples follows, e.g.: “Avoid putting HTML in JavaScript strings for ‘poor man’s templating‘”, messing up SQL with html (“SELECT '<strong>' + Username + '</strong>' FROM Users“), CSS+HTML: using inline styles, SQL+server-side: string concatenation to create dynamic SQL queries, “writing dynamic JavaScript in a string on the server“. I shall keep this in mind!
  • Johannes Brodwall: A canonical web test – a simple web app end-to-end smoke test – using an embedded Jetty, a test DB (preferably in-memory), WebDriver to test it (simple: browser.get(“/people”), assertThat(browser.findElement(<person id>.contains(<person’s name>)); simple, nice, useful

Learning

  • LearnGitBranching – an online game to learn branching & rebase in git; use the menu in the lower-right corner to navigate between the levels etc. You can also execute commands “show goal”, “hint”, “level” to navigate around; pretty cool and great for learning the command line commands

Society & people

Not a typical topic I share here but really worth it this time.

  • The ocean is broken – a saddening story worth reading to learn what does your tuna sandwitch cost and where does all the plastic we use end up. From a sailing trip from Melbourne to US where there were plenty of fish (and birds) 10 years ago – and 2 this year, killed to a noticable degree by huge fishing ships that catch tuna – and kill and throw away all the other “junk” fish. Nowadays fish are replaced by plastic and other waste that actually prevents usage of the engine unless somebody can watch for dangerous nets and ropes leftovers. Earth, where are you falling to?
  • The Guardian: Why have young people in Japan stopped having sex? – sad and interesting to observe what happens when the system is set up so that people “can’t be bothered” to have inter-sexual relationships, partnership, and children. Japan needs a good deal of systems thinking to fix its broken society where women do not want children because it would cost them their career and neither men nor women are willing to subjects themselves to the social pressure and demands associated with relationships.
  • The Guardian: 29 million people enslaved, says first global index on slavery – welcome to the 21st century! The leading slave countries are India (14M), China (3M), Pakistan (2M). Also, slaves are building the world cup stadion in Qatar.
  • They’re Taking Over! – how we managed to destroy sea ecosystems and helped the now unstoppable return of jellyfish – Jellyfish are evidently very veried and extemely resilient and have been hold at bay only by rather complex ecosystems that we managed to destabilize so much that Jellyfish are on their way back to ruling all the sees again (destroying the rests of the ecosystems – i.e. fish – on the way); a sad future for the sea, Earht, and us

Clojure Corner

Tools/Libs

Mac:

  • WhiteHat Aviator – A Safer Web Browser – WhiteHat, a well-known security company, has released a browser that aims at improving privacy by preventing user tracking (f.ex. but not sending referral URL) and blocking ads even at the cost of occassional slight discomfort, i.e. something that the mainstream browsers are not interested in. So far for OS X only.
  • EnvPane – a preference pane for environment variables for Mac OS X 10.8 (Mountain Lion) – set env. vars for GUI/terminal apps, no need to log out upon change

Favorite Quotes

Weinberg: Bureaucracy is what we do when we no longer remember why we are doing it
- via Ben Simo, no source specified so it may be fake but anyway it is valid

Posted in General, Languages, Testing, Top links of month | Tagged: , , , , , , , , , , , | 4 Comments »

Most interesting links of May ’13

Posted by Jakub Holý on May 31, 2013

Recommended Readings

Agile

  • Discussion: World’s Biggest ‘Agile’ Software Project Close To Failure – what is/isn’t agile, agile vs. waterfall etc; a nice collection of all the possible opinions and misunderstandings. Some of my favorites: waste: ‘[..]An “agile” project cannot fail and cost Billions because it must always deliver runnable software with a maximum of a few weeks delay[..]‘ (runnable = delivering value), separation: ‘[..] my experience has been that separating the designer/architect role from the developer role is fraught with pitfalls. The people writing the code should be the ones designing it, [..]‘, stability: ‘[..] On the successful projects that I’ve worked with in Agile, there’s strong stakeholders, good architecture keeping the vision in place and project management that keeps things well orchestrated. Without those in the mix, it’ll fail just like all software projects. [..]‘, waterfall success, people issue: ‘The problem here isn’t waterfall/agile. The problem here isn’t .Net/Linux. The problem here is the parties involved. [politicians and IT dinosaurs]‘ (learn what’s needed from drunk, bitching employees; ignore official nonsense requirements),: simplicity[..] It would probably be a lot easier if they started by making a simpler tool – instead of trying to calculate everybody’s entitlements everywhere [..]‘, agile suitability: ‘[..] You cannot use Agile to build a 100-mile canal, as the whole thing would be useless even if you completed 99 miles. [..]‘.
    Some people seem to believe that agile means no architecture and no/too little planning. Some believe that agile = hack now, fix later.

Startups etc.

Clojure Corner

  • B. Batsov’s Clojure Style Guide (based on JoC etc.) at GitHub
  • Code quaterly interview with R. Hickey (2011) – motivation behind Clojure (simplicity,..), reusability in C. (vs. Java); many valuable things
  • Clojure in the Enterprise? – about the differences between the Clojure and the enterprise java  (with heavy frameworks such as JPA, JSF, mutable state) ways and difficulties of introducing Clojure due to old-fashioned thinking, limited skills, etc. “Take away objects, mutable state, variables, loops… a lot of Java developers are immediately all at sea and have no idea how to solve even basic problems. They’ll try to bend Clojure to their OOP way of thinking and they’ll most likely fail.
    World Singles’ experience with Clojure: “We love Clojure. It’s made development a lot more fun. We’re able to solve harder problems, make changes faster, leverage multi-core concurrency more effectively, and we have a much smaller code base to maintain.
  • Replace Temp with Query in Clojure – in this post we can follow an interesting refactoring from a deeply nested if & let code to a much flatter one (featuring cond to test multiple conditions (instead of guard conditions used in imperative languages) delay to be ably to bind an expression to a local variable without evaluating it yet)
  • The Why and How of Clojure on Android (4/2013) – about the experience of using Clojure on Android, which is little slow and crazy but fun. From the author of the Nightweb app. Key components:  neko Clojure wrappers for Android and for lein-droid building w/o an IDE. Nicer than Java! (This GSoC proposal shows the limitations/future of neko.)
  • Reconstructing Clojure Macros With Speclj – a good idea to learn macros by trying to create one’s own implementation of existing Clojure macros, driven by tests that define the expected behavior of the macro (using macroexpand, macroexpand-1, macroexpand-all)
  • Advanced inspector middleware for Clojure nREPL and Emacs (nrepl-inspector on GitHub) – C-c C-i or nrepl-inspect to inspect interactively the value of a var (beware: (require ‘nrepl-inspect) fails, however calling nrepl-inspect as a function is ok). Related: Clojure Debugging ’13: Emacs, nREPL, and Ritz (May 17th) – what works, what is missing, how to set up
  • Pithering About » REPL bootstrap pimpage – neat tricks with a custom bootstrap REPL namespace – print last few commits and git branch, pre-loading common libs

Tools

Favorite Quotes

C, C#, and Java:
Applying some of the best ideas of the 1970s to the problems of today.

- Stuart Halloway in Clojure in the Field, 2013 (slide 6)

Posted in General, SW development, Tools, Top links of month | Tagged: , , , , , , | 2 Comments »

Most interesting links of January ’13

Posted by Jakub Holý on January 31, 2013

Recommended Readings

Various

  • Dustin Marx: Significant Software Development Developments of 2012 – Groovy 2.0 with static typing, rise of Git[Hub], NoSQL, mobile development (iOS etc.), Scala and Typesafe stack 2.0, big data, HTML5, security (Java issues etc.), cloud, DevOps.
  • 20 Kick-ass programming quotes – including Bill Gates’ “Measuring programming progress by lines of code is like measuring aircraft building progress by weight.”,  B.W. Kernighan’s “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”, Martin Golding’s “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” (my favorite)
  • How to Have a Year that Matters (via @gbrindusa) – do you want to just survive and collect possessions or do you want to make a difference? Some questions everybody should pose to him/herself.
  • Expression Language Injection – security defect in applications using JSP EL that can sometimes leads to double evaluation of the expressions and thus makes it possible to execute data supplied by the user in request parameters etc. as expressions, affects e.g. unpatched Spring 2.x and 3.

Languages etc.

  • HN discussion about Scala 2.10 – compilation speed and whether it matters, comparison of the speed and type system with Haskell and OCaml, problems with incremental compilation (dependency cycles, fragile base class), some speed up tips such as factoring out subprojects, the pros and cons of implicits etc.
  • Blog Mechanical Sympathy – interesting posts and performance tests regarding “writing software which works in harmony with the underlying hardware to gain great performance” such as Memory Access Patterns Are Important and Compact Off-Heap Structures/Tuples In Java.
  • Neal Ford: Functional thinking: Why functional programming is on the rise – Why you should care about functional programming, even if you don’t plan to change languages any time soon – N. Ford explains the advantages of FP and why FP concepts are spreading into other languages (higher abstractions enabling focus on the results over steps and ceding control to the language, more reusability on a finer level (higher-order functions etc.), few generic data structures with many operations -> better composability, “new” and different tool such as lazy collections, shaping the language towards the problem instead of vice versa, aligning with trends such as immutability)
  • Neal Ford: Java.next: The Java.next languages Leveraging Groovy, Scala, and Clojure in an increasingly polyglot world – a comparison of these languages with focus on what they are [not] suitable for, exploration of their paradigms (static vs. dynamic typing, imperative vs. functional)

SW development

  • How to Completely Fail at BDD – a story of an enthusiastic developer who tried to make everyone’s life better by introducing automated BDD tests and failed due to differences in culture (and inability to change thinking from the traditional testing), a surprising lack of interest in the tool and learning how to write good tests: “Culturally, my current team just isn’t ready or interested in something like this.” Morale: It is hard to change people, good ideas are not enough.
  • M. Feathers: Refactoring is Sloppy – refactoring is often prioritized out of regular development and refactoring sprints/stories aren’t popular due to past failures etc. An counter-intuitive way to get refactoring in is to imagine, during planning, what the code would need to be like to make it easy to implement a story. Then create a task for making it so before the story itself and assign it to somebody else then the story (to force a degree of scrutiny and communication). “Like anything else in process, this is medicine.  It’s not meant to be ‘the way that people do things for all time’ [..]” – i.e. intended for use when you can’t fit refactoring in otherwise. It may also make the cost of the current bad code more visible. Read also the commits (f.ex. the mikado method case).
  • Cyber-dojo: A great way to practice TDD together. Compare your read-green cycle and development over time with other teams. Purposefully minimalistic editor, a number of prepared tdd tasks.
  • On the Dark Side of “Craftsmanship” – an interesting and provoking article. Some developers, the software labouers, want to get work done and go home, they haven’t the motivation and energy to continualy spend time improving themselves. There is nothing wrong with that and we shouldn’t disparge them because of that. We shouldn’t divide people into craftsmen and the bad ones. A summary of and response to the varied reactions follows up in More on “Craftsmanship”. The author is right that we can’t expect everybody to spend nights improving her/his programming skills. Still they should not produce code of poor quality (with few exceptions) since maintaining such code costs a lot. There should be time for enough quality in a 9-5 day and people should be provided with enough guidance and education to be able to write decent code. (Though I’m not sure how feasible it is, how much effort it takes to become an acceptable developer.) Does the increased cost of writing (an learning to write) good code overweight the cost of working with bad code? That is an eternal discussion.

Cloud, web, big data etc.

  • Whom the Gods Would Destroy, They First Give Real-time Analytics (via Leon) – a very reasonable argument against real-time analytics: yes, we want real-time operational metrics but “analytics” only makes sense on a sensible amount of data (for the sake of statistical significance etc.) RT analytics could easily provide misguided results.
    CAP Twelve Years Later: How the “Rules” Have Changed (tl;dr, via @_dagi) – an in-depth discussion of the CAP theorem and the simplification (2 out of 3) that it makes; there are many more nuances. By Eric Brewer, a professor of computer science at the University of California, Berkeley, and vice president of infrastructure at Google.
  • ROCA: Resource-oriented Client Architecture – “A collection of simple recommendations for decent Web application frontends.” Server-side: true REST, no session state, working back/refresh etc. Client: semantic HTML independent of layout, progressive enhancement (usable with older browsers), usable without JS (all logic on the server) etc. Certainly not suitable for all types of apps but worthwile to consider the principles and compare them with your needs.

Clojure Corner

Tools

  • Vaurien, the Chaos TCP Proxy (via @bsvingen) – an extensible proxy that you can control from your tests to simulate network failure or problems such as delays on 20% of the requests; great for testing how an application behaves when facing failures or difficulties with its dependencies. It supports the protocols tcp, http, redis, memcache.
  • Wvanbergen’s request-log-analyzer for Apache, MySQL, PostgreSQL, Rails and more (via Zarko) – generates a performance report from a supported access log to point out requests that might need optimizing
  • Working Effectively With iTerm2 (Mac) – good tips in the body and comments

Favorite Quotes

A very good (though not very scientific) definition of project success applicable for distinguishing truly agile from process-driven projects:

[..] a project is successful if:

  • Something was delivered and put to use
  • The project members, sponsors and users are basically happy with the outcome of the project

- Johannes Brodwall in “How do we become Agile?” and why it doesn’t matter, inspired by Alistair Cockburn

(Notice there isn’t a single word about being “on time and budget”.)

Posted in General, Languages, Testing, Tools, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Creating Custom Login Modules In JBoss AS 7 (and Earlier)

Posted by Jakub Holý on June 21, 2012

JBoss AS 7 is neat but the documentation is still quite lacking (and error messages not as useful as they could be). This post summarizes how you can create your own JavaEE-compliant login module for authenticating users of your webapp deployed on JBoss AS. A working elementary username-password module provided.

Read the rest of this entry »

Posted in j2ee, Languages, Tools | Tagged: , , , | Comments Off

Most interesting links of November

Posted by Jakub Holý on November 30, 2011

Recommended Readings

  • Recommended Reading by Poppendiecks – an excellent selection, starting with Lean from Trenches, Management 3.0, Specification by Example, The Lean Startup etc.
  • Eric Allman says that Programming Isn’t Fun Any More  because problem solving has been replaced with learning, configuring, and integrating tons of libraries, frameworks, and tools and many people agree with that (as discussion on reddit proves). In other words we tend to go for any benefit we can have without considering the costs and for “easy” solutions without considering the true enemy: complexity. Perhaps we should always listen to the Rich Hickey’s Simple Made Easy talk before we add a lib/tool/framework?
    • Dean Wampler claims that functional programming can bring the joy back – “[..] a functional language, Scala, Clojure, Haskell, etc. will greatly reduce the amount of code you create. That won’t solve the problem of trying to integrate with too many libraries, but you’ll be less tempted. I also believe those libraries will be less bulky, etc.
    • Few quotes from a related article by M. Taylor: To put it another way, libraries make excellent servants, but terrible masters. | [..] frameworks [..] do keep their promise of making things very quick and easy … so long as you do things in exactly the way the framework author intended | On libraries: [..] we all assume (I know I do) that “plug in solutions X1 and X3″ is going to be trivial. But it never is — it’s a tedious exercise in impedance-matching, requiring lots of time spent grubbing around in poorly-written manuals [..] | On the effect of language choice: [..] different languages, with their different expressive power and especially their different culture, yield very different experiences.
    • To sum it up: Choose your tools and libraries wisely and always mind the global complexity. More usually means worse.
  • Java Magazine – Adam Bien: Stress Testing Java EE 6 Applications (page 41+) – do developer stress testing! Using: JMeter, VisualVM to find out resource consumption and behavior in the application, VisualVM’s Sampler profiling tool [cca 20% overhead], a webapp to extract metrics from GF (STM)
  • Java Magazine – Polyglot Programming on the JVM (page 50; excerpt from The Well-Grounded Java Developer) – why you should consider polyglot programming and how to decide whether to use it and what languages to pick, f.ex.: “These [Java's] qualities make the language a great choice for implementing functionality in the stable layer [of the polyglot programming pyramid]. However, these same attributes become a burden in the middle and upper [lower, DSL, on the linked image] tiers of the pyramid; for example: Recompilation is laborious; Static typing can be inflexible and lead to long refactoring times; Deployment is a heavyweight process; Java’s syntax is not a natural fit for producing DSLs.” “There is a wide range of natural use cases for *alternative languages*. [after identifying such a UC] You *now need to evaluate* whether using an alternative language is appropriate.”
  • Intrusion Detection for Web Apps – Detection Points – If security is a concern of your web application then you should build intrusion detection into the application f.ex. leveraging the  OWASP AppSensor project. The key is to detect malicious/unexpected behavior and proactively do something such as locking the user out or alerting the admins. The page linked above lists some common suspicious behaviors such as the use of multiple usernames, unexpected HTTP command/method, additional/duplicated data in request. Worth checking out!
  • Yammer Moving From Scala to Java- Scala is a cool language but sometimes its cost is higher than the benefits. Snippets from the post: “…the friction and complexity that
    comes with using Scala instead of Java isn’t offset by enough productivity benefit or reduction of maintenance burden …”. “Scala, as a language, has some profoundly interesting ideas in it. [...] But it’s also a very complex language. The number of concepts I had to explain to new members of our team for even the simplest usage of a collection was surprising: implicit parameters, builder typeclasses, ‘operator overloading’, return type inference, etc. etc.” (It’s claimed that only library authors need to know some of that but if it’s a part of library APIs, the users need to understand it too.) Notice that the author isn’t saying “Scala is bad” but only that Scala isn’t the best balance of their needs at this time, as Alex Miller put it*.
    Important note
    : The text wasn’t intended for publication and it is a private opinion of a Yammer developer, not the company itself. You should read the official Yammer’s position where Coda puts it into the right context.

Refactoring

  • Opportunistic Refactoring by Martin Fowler – refactor on the go – how & why
  • Michael Feathers: Getting Empirical about Refactoring – gather information that helps us understand the impact of our refactoring decisions using data from a SCM, namely File Churn (frequency of changes, i.e. commits) vs. Complexity – files with both high really need refactoring. Summary: “If we refactor as we make changes to our code, we end up working in progressively better code. Sometimes, however, it’s nice to take a high-level view of a code base so that we can discover where the dragons are. I’ve been finding that this churn-vs.-complexity view helps me find good refactoring candidates and also gives me a good snapshot view of the design, commit, and refactoring styles of a team.

UIs and Web Frameworks

  • Devoxx 2011 - WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks (slides) – the authors did extensive performance testing of some of the most popular web frameworks. Of course it’s always hard to guess how general their results are, if/how they apply to one’s particular situation, and if they aren’t distorted in some way but it’s worth for their approach alone (AWS with its CloudWatch monitoring, WebDriver, additional measurement of page load with HAR and a browser plugin). In their particular tests GWT scored best, followed by Spring MVC, with JSF and Wicket lagging far behind (especially the MyFaces implementation). Conclusion: A web framework may have strong impact on performance and scalability, if they are important for you then do test the performance early with as realistic code and load as possible.
  • JSF2 – Benchmark datatable by N. Labrot, 2/2011 – performance comparison of PrimeFaces 2.2.1, IceFaces 2.0, Richfaces 4.0.0M4 on a simple page with Ajax. I do not trust any benchmark that I don’t fake myself :-) (for there are always too many factors that influence the conclusions to be drawn) but it’s interesting anyway – and perhaps a good thing to do before you decide for a JSF component library.
  • Alex MacCaw: Asynchronous UIs – the future of web user interfaces and the Spine framework – users in 2011 shouldn’t anymore wait for pages to load and operations to complete, we should build asynchronous UIs where changes to the UI are performed immediately while a request to the server is sent in the background, similarly to sending e-mail in GMail, which returns at once displaying a non-intrusive “Sending…” notification. As a user I very much agree with Alex.
  • Matt Raible’s 20 criteria for evaluating web frameworks, 2010 (detailed description, here’s a brief list) – Matt’s results are disputable and as he himself says you should always do your own evaluation and spikes but the criteria are pretty useful: Developer Productivity, Developer Perception, Learning Curve, Project Health, Developer Availability, Job Trends, Templating, Components, Ajax, Plugins or Add-Ons, Scalability, Testing, i18n and l10n, Validation, Multi-language Support (Groovy / Scala), Quality of Documentation/Tutorials, Books Published, REST Support (client and server), Mobile / iPhone Support, Degree of Risk.

NoSql

  • Don’t use MongoDB via @nicolaiarocci – a (fake?!) bad experience with MongoDB – the text is not credible (the author is anonymous, s/he doesn’t explicitely state which version of MongoDB they used, the 10gen CTO can’t find a matching client and any evidence for some of the issues mentioned) but it  gives context for the read-worthy response from the 10gen CTO, and a post that nicely explains how to correctly design for MongoDB. A comment about MongoDB experience at Forsquare: “Currently we have dozens of MongoDB instances across several different data clusters storing over a TB of data and handling 10s of thousands of requests per second (mostly reads but the write load is reasonably high as well).Have we run into problems with MongoDB along the way? Yes, of course we have. It is a new technology and problems happen.Have they been problematic enough to seriously threaten our data? No they have not.
  • Martin Fowler on Polyglot Persistence – the are when will be choosing persistence solution with respect to our needs instead of mindlessly picking RDBMS is coming. Applications will combine multiple, specific solutions, f.ex. we could pick Redis (key-value) for caching, MongoDB (document DB) for product catalog, Neo4J (graph DB) for recommendations, RDBMS for financial data and reporting… (of course not all in one project!). Polyglot persistence will come at a cost (complexity, learning) – but it will come because the benefits are worth it – performance, data storage model and behavior more aligned with the business logic (NoSql databases ofer various models and tradeoffs and thus we can find a much better fit than with general-purpose RDBMs).

Talks & Video

  • Adam Bien’s JavaOne talk Java EE 6: The Cool Parts (1h) – absolutely worth the time – a very practical fly through the cool features of Java EE (eventing, ..), most of the time is spent actually coding. Don’t forget to check also the interesting discussion below the video (JEE and other frameworks, Java FX and JSF 2, …).
  • Jurgen Appelo’s keynote How to Change the World at Smidig 2011 is well done and highly useful. We all strive to change the world around us – as consultants we want to make our clients more agile, as team members we want to make our Scrum teams more self-organizing, as employees we want to help building knowledge-sharing and open culture, … . However it isn’t easy to influence or change people and culture and if we aren’t aware of all the dimensions of a change (system, individuals, interactions, environment) and how to work along each of them, we are much less likely to succeed. The knowledge and experience that Jurgen shares with us can help us a lot in having an impact. You can also download the slides and change management questions.
  • Project X: What is being a programmer like? (5min) If ever again a non-geek asks you what you as a developer are doing, just show him this short and extremely funny video (created by my ex-employer – perhaps they estimated how much time and energy developers loose trying to explain it to normal people and decided to prevent this great waste :-))
  • RSA Animate – Drive: The surprising truth about what motivates us (10 min) – entertaining and enlightening; once we’ve enough money to cover our needs, it’s autonomy (self-direction), mastery, and purpose what motivates us (money actually decrease our performance). Now this is a great evidence for lean/agile – for they’re based on making people self-directing and encourage mastery (as in continous integration and top quality to enable steady pace). Autonomy enables engagement as does a higher purpose (“make the world a better place”) – Steve Jobs with his visions was able to provide such a purpose. Atlassian’s FedEx Days are a good example of what engagement and benefits autonomy brings.
  • Simon Sinek: How great leaders inspire action (18 min, subtitles in 37 languages) – do you want to succeed, to change the world around you for the better, to start a new company? Then you must start by communicating “why” you do what you do, not “what” – like M. L. King, bro Wrights, and Apple. Very inspiring! (More in his Why book.)

Links to Keep

Favorite Quotes

Refactoring is like advertising: it doesn’t cost, it pays.
- Mary & Tom Poppendiecks, Implementing Lean Software Development, p.166

Clojure Corner

Posted in Databases, General, j2ee, Top links of month | Tagged: , , , , , , , , , , , | Comments Off

Tip: Retrieving server certificate used in SSL communication (e.g. POP3s)

Posted by Jakub Holý on October 21, 2010

If you would like to get the security certificate used by a server in communication over SSL, such as with the HTTPS or POP3s protocols, for instance to install it on a client device that needs to talk to the server, you can use OpenSSL to retrieve it:


bash$ openssl s_client -connect pop.gmail.com:995 -showcerts

You then store the text starting with “—–BEGIN CERTIFICATE—–” and ending with “—–END CERTIFICATE—–” (inclusive) into a file with the extension .pem.

You can also use openssl to convert the certificate into another format, e.g.:


bash$ openssl x509 -in mycertificate.pem -inform PEM -out mycertificate.der -outform DER

The information originates from the page Gmail POP3 with Fetchmail where you can find a more detailed description and also instructions for testing the certificate.

Posted in General | Tagged: , , | Comments Off

Jetty-maven-plugin: Running a webapp with a DataSource and security

Posted by Jakub Holý on September 10, 2010

This post describes how to configure the jetty-maven-plugin and the Jetty servlet container to run a web application that uses a data source and requires users to log in, which are the basic requirements of most web applications. I use Jetty in development because it’s fast and easy to work with. Read the rest of this entry »

Posted in j2ee, Languages, Tools | Tagged: , , , , , , | Comments Off

RAD, WAS 6 Test Env and J2EE security: getting rid of “No received or invocation credential exist on the thread”

Posted by Jakub Holý on November 24, 2006

Recently I started a new J2EE project that required j2ee security. Unfortunately Rational app. Developer 6 seemed to have some problems running a WAS Test Environment in this way, at least with the default setup. I encountered 2 problems:

  1. RAD wasn’t able to connect to the server at all, throwing some security exceptions.
    The solution (or rather work-around) was to switch the “Server connection type..” in the server’s configuration in RAD from RMI to SOAP. It seems that RMI doesn§t support authetification.
  2. After that, everything seemed to be ok, but an error was logged every few seconds with the cause “No received or invocation credential exist on the thread“. The stack trace pointed to a MBean (a part of JMX) being contacted via RMI.
    Work-around: Disable an RMIConnector for JMX in WAS Console: log in, go to
    Application servers > server1 > Administration  > Administration Services > JMX connectors
    And delete the RMIConnector – make first sure that you don’t need it.

BTW, to enable the security, I enabled it first on the server via WAS admin console and then I created a new server configuration in RAD and set it to use security.

Stack trace for issue 2:
com.ibm.ws.security.role.RoleBasedAuthorizerImpl.checkAccess(RoleBasedAuthorizerImpl.java(Compiled Code)) at com.ibm.ws.management.AdminServiceImpl.preInvoke(AdminServiceImpl.java:1696) at com.ibm.ws.management.AdminServiceImpl.preInvoke(AdminServiceImpl.java:1603) at com.ibm.ws.management.AdminServiceImpl.preInvoke(AdminServiceImpl.java:1533) at com.ibm.ws.management.AdminServiceImpl.preInvoke(AdminServiceImpl.java:1506) at com.ibm.ws.management.AdminServiceImpl.getAttribute(AdminServiceImpl.java:501) at com.ibm.ws.management.connector.AdminServiceDelegator.getAttribute(AdminServiceDelegator.java:99) at com.ibm.ws.management.connector.rmi.RMIConnectorService.getAttribute(RMIConnectorService.java:123) at com.ibm.ws.management.connector.rmi._RMIConnectorService_Tie.getAttribute(_RMIConnectorService_Tie.java:193) at com.ibm.ws.management.connector.rmi._RMIConnectorService_Tie._invoke(_RMIConnectorService_Tie.java:87) at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:608) at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:461) at com.ibm.rmi.iiop.ORB.process(ORB.java:432) at com.ibm.CORBA.iiop.ORB.process(ORB.java:1728) at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2229) at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:65) at com.ibm.ejs.oa.pool.PooledThread.run(ThreadPool.java:95) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code)) . 

Posted in eclipse, WebSphere | Tagged: , , , , | Comments Off