The Problem With java.util.concurrent

The java.util.concurrent package is a welcome addition to the Java API, but it is a solution to yesterday's concurrency problems and not today's or tomorrow's.

Back in 2000-2005, "concurrency" meant a multi-threaded server process. It would have a number of worker threads in a thread pool, and as each request arrived, a thread would grab it and process it before returning to the pool and the next request. Since every request was serviced by a single thread, the whole architecture was simple and could easily be understood as independent single-threaded processes that just so happened to run concurrently.

Today, concurrency is everywhere - client side as well as server side - and involves putting multiple cores to work whenever possible. Those cores can in turn be of different kinds, such as CPUs or GPUs. The environment has grown more complex[1], and the abstractions of a decade ago are no longer sufficient.

I blame a lot on the designers of the Java API. We get a bunch of nice parts, but not the plan that would enable us to assemble them into larger structures.

For example: Let's say you have a set of CPU-intensive independent tasks - the number isn't important, but let's say that there are more tasks than CPU cores. The solution is obvious: Run one thread per core (to minimize context switching), and divide the work equally among the cores. java.util.concurrent has a ThreadPool implementation for this, so it's not that difficult. The problem comes when you realize that your application uses, say, FooLib and BarLib - each of which may have their own thread pools. A decade ago our server process would have the only thread pool, and everything else would be single-threaded, but with today's fine-grained concurrency we're left to wonder how many cores do we really have access to. At this point the problem has shifted from mechanism to policy. We have the mechanism - thread pools - all sorted. What we lack is a policy that enables us to use the mechanism in such a way as to make our code modular, reusable and composable.

Java may have many concurrency primitives (Phaser?), and a fork-join framework, but the whole package gives more the impression of a student assignment to implement recently-learned concurrency theory than the product of an engineer implementing a solid base for Java concurrency. It's a case of "I wrote all this code, see if you can make it do anything interesting, because I sure can't".