ForkJoin Approach to Concurrency

Fork/Join gets parallel speedup by recursively splitting one big task into halves until each piece is small enough to run sequentially, then combining the partial results back up the tree — and it runs all those pieces on a small fixed pool of worker threads (one per CPU core) where idle workers steal queued subtasks from busy ones, so the cost of "forking" is a cheap object push onto a queue, not the creation of an OS thread.

The mechanism, precisely

The naive mental model is "fork = spawn a thread, join = wait for it." That is wrong at scale, and the correction is the whole point of this design:

• A real OS thread costs roughly 0.5–1 MB of stack plus a kernel-side scheduling entry. Creating one per subtask in a recursive split (which generates thousands of tasks) would exhaust memory and drown the CPU in context switches.
• So ForkJoinPool creates a bounded set of worker threads — by default Runtime.getRuntime().availableProcessors() of them — and keeps them alive for the program's life.
• fork() does not start a thread. It pushes the subtask onto the calling worker's own double-ended queue (deque) and returns immediately. That push is a few instructions.
• Each worker pops its own tasks from the head (LIFO — good cache locality, the most-recently-forked task is usually the one whose data is still hot). When a worker's deque is empty, it steals a task from the tail of a random other worker's deque (FIFO from the victim's side — it grabs the oldest, biggest, least-likely-to-conflict task). This is work-stealing, and it is what keeps all cores busy without a central lock.
• join() on a forked subtask does not block the worker idly. If the result isn't ready, the worker helps — it runs other queued tasks (including the one it's waiting on) until the result is available. A blocked worker is a wasted core, so the pool refuses to waste it.

Worked example: sum an array of 8 elements, threshold 2

Take [3, 1, 4, 1, 5, 9, 2, 6] (sum = 31). The task splits any range longer than the threshold (2) in half and forks; ranges of length ≤ 2 are summed directly. Watch the tree build and collapse:

The leaves (8, 14, 4, 5) sit on worker deques. If a second core is free during step 3, it steals [4,6) from the tail of the first worker's deque and computes 14 in parallel — that is where the speedup comes from. With one core, the same tree runs sequentially and still produces 31, just without the overlap.

Java: RecursiveTask<Long>

This is the canonical Fork/Join shape. Note the order: fork the left, compute the right on the current thread, then join — never fork both and join both, because that wastes the current worker.

import java.util.concurrent.RecursiveTask;
import java.util.concurrent.ForkJoinPool;

class SumTask extends RecursiveTask<Long> {
    private static final int THRESHOLD = 2;
    private final int[] a;
    private final int lo, hi;

    SumTask(int[] a, int lo, int hi) { this.a = a; this.lo = lo; this.hi = hi; }

    @Override
    protected Long compute() {
        if (hi - lo <= THRESHOLD) {          // small enough: do it directly
            long sum = 0;
            for (int i = lo; i < hi; i++) sum += a[i];
            return sum;
        }
        int mid = (lo + hi) >>> 1;
        SumTask left = new SumTask(a, lo, mid);
        left.fork();                          // push left onto THIS worker's deque
        long right = new SumTask(a, mid, hi).compute(); // run right here, no waiting
        long leftResult = left.join();        // help/steal until left is done
        return leftResult + right;
    }
}

// usage
int[] data = {3, 1, 4, 1, 5, 9, 2, 6};
long total = ForkJoinPool.commonPool().invoke(new SumTask(data, 0, data.length));
// total == 31

Why the naive version is wrong

A version that does left.fork(); right.fork(); return left.join() + right.join(); forks both halves and then sits in join() — the current worker has nothing to compute itself and the second forked task often just gets stolen, adding overhead. Worse, calling left.join() before right.fork() serializes the whole thing. And a version that spawns new Thread(...) per split — the old "forking = new thread" idea — creates thousands of OS threads for a deep tree and collapses. The fixed pool + deque + steal is exactly the fix for that cost.

Go: the same divide-and-conquer with goroutines + a WaitGroup

Go has no ForkJoinPool in its standard library because it doesn't need a user-space work-stealing pool — the runtime scheduler itself is a work-stealing scheduler over goroutines. A goroutine is a few KB of growable stack multiplexed onto a small set of OS threads (the M:N model, bounded by GOMAXPROCS), so spawning one really is cheap. You express the split directly:

package main

import (
    "fmt"
    "sync"
)

const threshold = 2

func sum(a []int) int {
    if len(a) <= threshold {
        s := 0
        for _, v := range a { s += v }
        return s
    }
    mid := len(a) / 2
    var left int
    var wg sync.WaitGroup
    wg.Add(1)
    go func() {              // "fork" the left half — a goroutine, not an OS thread
        defer wg.Done()
        left = sum(a[:mid])
    }()
    right := sum(a[mid:])    // compute the right half on this goroutine
    wg.Wait()                // "join": wait for the left half
    return left + right
}

func main() {
    fmt.Println(sum([]int{3, 1, 4, 1, 5, 9, 2, 6})) // 31
}

Where the two runtimes differ

• Unit of work. Java forks ForkJoinTask objects onto a deque you wait on with join(); Go forks goroutines and you wait with a sync.WaitGroup or a channel. Go's work-stealing is hidden inside the runtime; Java's is the explicit ForkJoinPool you call into.
• Cost & bounding. Both bound the OS-thread count (Java pool size ≈ cores; Go GOMAXPROCS ≈ cores). The Java version above with a real-thread-per-split would die; the Go version is fine because a goroutine ≈ a few KB and the runtime, not the kernel, schedules it. Still, an unbounded recursive goroutine fan-out can pile up scheduling pressure, so for huge fan-outs you'd add a worker-pool / semaphore.
• Blocking semantics. Java's join() keeps the worker useful by running other tasks while it waits. Go's wg.Wait() parks the goroutine and the scheduler immediately runs another goroutine on that OS thread — same net effect (no wasted core), different mechanism (channel/parking vs. task helping).

Pitfalls

• Threshold too small. A threshold of 1 forks a task per element. For summing an int array the split/steal bookkeeping costs more than the addition, so the parallel version is slower than a plain loop. Pick a threshold where a leaf does enough real work (often thousands of elements, or a few microseconds) to dwarf the fork cost.
• Blocking inside a Fork/Join task. If a task does I/O, sleeps, or grabs a lock, it parks a pool worker and there are only ~N workers. A few blocked tasks can stall the entire common pool — including everything else in the JVM that shares it (parallel streams, CompletableFuture.supplyAsync defaults). Use a separate ForkJoinPool for blocking work, or wrap it in ManagedBlocker so the pool spins up a compensating thread.
• Wrong fork/join order. Forking both halves and joining both, or joining before forking the sibling, kills the speedup (see "Why the naive version is wrong"). Always: fork left, compute right inline, join left.
• Shared mutable state in subtasks. Fork/Join only parallelizes cleanly when subtasks are independent. If two leaves write the same field you reintroduce the race conditions the next lessons cover — the model gives you parallelism, not safety.
• In Go: capturing the loop variable / racing on a shared result. Writing partial results into a shared slice from many goroutines without distinct indices, or capturing a mutating variable in the closure, is a data race — run with go test -race.

Takeaways

• Fork/Join = recursive split until small, run leaves on a bounded pool, combine results up the tree. fork() is a cheap push, not a thread spawn — that's the whole reason it scales.
• Work-stealing keeps every core busy without a central queue lock: workers run their own tasks LIFO and steal others' oldest tasks FIFO.
• In Java you write RecursiveTask/RecursiveAction and call into a ForkJoinPool; in Go you write goroutines and the runtime is the work-stealing scheduler. Both bound OS threads to ≈ core count.
• The model gives parallelism, not correctness — tune the threshold, never block a pool worker, and keep subtasks independent.

Re-authored and deepened for this guide. Sources: Doug Lea, "A Java Fork/Join Framework" (2000), the original design paper for java.util.concurrent; the OpenJDK ForkJoinPool / RecursiveTask Javadoc and source; Brian Goetz et al., Java Concurrency in Practice; the Go runtime scheduler design (Dmitry Vyukov's work-stealing scheduler) and The Go Programming Language (Donovan & Kernighan), ch. 8–9. The earlier version's claim that "spawning new threads is not resource-intensive" was corrected — real Fork/Join uses a bounded pool with work-stealing precisely because OS-thread creation is expensive.

ForkJoin vs parallel streams vs virtual threads

Worked decision

Sum 100M doubles: use FJ or Arrays.parallelPrefix/parallelStream with threshold-sized chunks — CPU-bound, pure. Fetch 100k URLs: virtual threads (or Go goroutines), not ForkJoin — each task blocks on network and would park FJ workers.

Failure / ops

All app latency spikes when one feature calls parallelStream() that does JDBC — common pool workers stuck in I/O; other parallel streams starve. Fix: dedicated pool, or virtual threads, never block the common FJ pool.

Drill ladder

• L1: What does fork() put on the deque — a thread or a task?
• L2: Why fork-left / compute-right?
• L3: When is parallelStream a production footgun?
• L4: Defend virtual-thread-per-task vs ForkJoin for a CPU matrix multiply.