Just finished reading Osborne's actual write up, and it's one of the better tutorials I've read on how to optimize code. Write things in the simplest way, measure, then optimize the bottle necks. We've all heard that, but here is a very concrete example of how to do it in practice. He puts in type annotations and fiddles with the underlying Java bits only where necessary, and wraps it in a way that the code calling the optimized code is mostly idiomatic Clojure.

It also reflects well on the pragmatism behind Clojure's design. The type annotations and dropping into Java, while not as pretty as pure Clojure, are still prettier than regular Java and can be isolated to the performance hot spots.

I appreciated this little bit of editorializing:

"The -> and doto macros save us from having to name each intermediate step and from the unreadable nesting you’d get if you tried to do the same thing in a typical curly-brace language.

    BufferedReader reader = new BufferedReader(new InputStreamReader(new BufferedInputStream(
                          new FileInputStream(filename).skip(startByte), 131072), "US-ASCII"));

Oops, FileInputStream.skip() returns a long, so that doesn’t even work. Now who was complaining about Lisp syntax (looking (like (this)))?"

Nicely done. An informative, enjoyable read. Two thumbs up.

"OK, but a Clojure purist would probably see those occasions as maybe highlighting gaps in that language’s coverage."

A Clojure "purist" is probably still programming Scheme. In other words, "purist" and "Clojure" do not go well together. As Rich Hickey often says, he is a "practitioner." He wants to Get Things Done, and Clojure reflects that. Dropping down into Java is not "cheating," it's one of the reasons Clojure was implemented on the JVM in the first place, and I've seen Rich recommend calling Java code to solve a problem multiple times on the Clojure group.

Amen. I was irked by the "purity" factor as well. Adam's (ato's) response in the comments is dead on: "I needed an Atomic counter. Java has one. I used it."

One of the biggest draws of Clojure is that it runs on the JVM so you get access to Java's libraries. It is baffling to me how someone could describe using Java via Clojure as being a no-no.

This is a great way to describe "The Clojure Way". I think I'm going to co-opt this explanation for future use (I hope you don't mind).

Don't mind at all. Go right ahead.

I find it fascinating that this Clojure implementation is faster than the current best Java one. I wonder how much faster a Java implementation could be.

Any experienced Clojure people care to comment on why atoms were used instead of agents? You don't care when the counter is incremented, just that it gets incremented, so I'd assume that an agent would model the problem more closely. Is it an overhead thing or what?

Agents launch the update function in a separate thread, which seems a bit much for incrementing a counter.

One of these days I'll remember that agents aren't actors, just not today.

Indeed, perhaps it would be a better way to model the problem, I considered it but perhaps rejected it incorrectly. By default Clojure uses a thread-pool for atoms so it wouldn't be a new thread for each request, the overhead is just a queue push which is not that great. It would mean serializing all the updates, but you could use fine-grained agents like I'm using fine-grained atoms.