Knowledge Guide
Home › Hands-On Builds › Concurrency Katas

Build a Thread Pool

Build a Thread Pool

You already know the theory from Thread Pools, Executors & Futures: reuse worker threads instead of spawning one per request. This project makes you build the pool yourself — worker loop, bounded queue, rejection policy, graceful shutdown, and a Future-returning submit() — in Go and Java, from an empty file, and prove the bounded-queue decision with a test that actually demonstrates the failure it prevents. Every staff-level system-design interview eventually asks "how would you bound this," and a thread pool is where that instinct gets built.

The Trap

Say your service handles each incoming request the obvious way:

// The naive approach — looks fine in a demo, dies in production.
ServerSocket server = new ServerSocket(8080);
while (true) {
    Socket client = server.accept();
    new Thread(() -> handle(client)).start();   // one new OS thread per request
}

At 10 requests/sec this is invisible. At 5,000 concurrent connections during a traffic spike, you now have 5,000 live threads. Each Java thread reserves roughly 1 MB of stack by default — that's ~5 GB of stack space alone before a single byte of application data. Worse than the memory: the OS scheduler is now context-switching between thousands of runnable threads competing for however many CPU cores you actually have (commonly 4–32). Cache lines get evicted every switch, the CPU spends more cycles switching than computing, and throughput falls as load rises — the exact opposite of what you want under a spike. Eventually thread creation itself fails with OutOfMemoryError: unable to create new native thread, and the whole service goes down, not gracefully — all at once.

The fix is not "handle it faster." It's structural: cap the number of worker threads at some small N, and feed them work through a queue instead of minting a thread per unit of work. That's a thread pool — and the moment you say "cap it," you've opened three decisions that are the actual substance of this kata: how big is the queue, what happens when it's full, and how do you stop cleanly.

Scope it like a senior

Before writing a line of pool code, pin down the contract — these are the questions that separate "I used ExecutorService" from "I understand what it's hiding":

Answer: fixed pool, N configurable, bounded queue (default small, e.g. 2×N), ABORT as the default rejection policy with CALLER_RUNS available, graceful shutdown by default with a hard shutdownNow() escape hatch, and both Runnable and Callable<T> submission.

Reason to the design

Simplest thing that could work: one shared queue, N worker threads in a loop popping and running tasks.

// The core loop — this is 90% of a thread pool's actual logic.
while (running) {
    Runnable task = queue.take();  // blocks until work exists
    task.run();
}

This is just Producer–Consumer with the "workers" as consumers and submit() as the producer — if you've built a bounded blocking queue before, you already have the hard part. What's new here is everything around the loop.

Why it fails as written: queue above is unbounded by default (that's what Executors.newFixedThreadPool actually gives you under the hood — a LinkedBlockingQueue with no capacity bound). If producers submit faster than N workers can drain, the queue absorbs the difference by growing. It never rejects, never slows the producer down, never signals overload to anyone — it just quietly eats memory. This is strictly worse than the thread-per-request trap in one way: it fails silently. Thread-per-request at least throws an exception loudly when threads run out; an unbounded queue lets the producer believe every request succeeded while the backlog balloons toward an OOM that lands minutes later, with a stack trace that points nowhere near the real cause.

The fix — make the queue bounded and force a decision at the full point: once queue.offer(task) can fail (because the queue is at capacity), you are forced to decide, explicitly, what happens next — and that decision is a rejection policy, not an afterthought. This is the derivation: bounded queue + explicit rejection policy is not a nice-to-have hardening step, it is the mechanism that converts a silent, delayed OOM into an immediate, visible backpressure signal. The system tells the truth about its capacity the moment it's exceeded, instead of lying until it can't anymore.

Layered on top of the core loop:

Build it — milestones

Reference implementation — Java (hand-rolled, not Executors.newFixedThreadPool)

Deliberately NOT the one-liner factory — the whole point is to own the queue bound, the rejection policy, and the shutdown sequencing that the factory method hides from you.

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.Future;
import java.util.concurrent.FutureTask;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.atomic.AtomicBoolean;

public final class ThreadPool {

    public enum RejectionPolicy { ABORT, CALLER_RUNS, DROP_OLDEST }

    // Sentinel: tells a worker "no more work, exit" without an interrupt
    // racing an in-flight task.
    private static final Runnable POISON_PILL = () -> {};

    private final BlockingQueue<Runnable> queue;
    private final Thread[] workers;
    private final RejectionPolicy policy;
    private final int capacity;
    private final AtomicBoolean shuttingDown = new AtomicBoolean(false);

    public ThreadPool(int nWorkers, int queueCapacity, RejectionPolicy policy) {
        this.capacity = queueCapacity;
        // LinkedBlockingQueue with a finite capacity is node-based (no upfront
        // array allocation) yet still enforces the bound on offer().
        this.queue = new LinkedBlockingQueue<>(queueCapacity);
        this.policy = policy;
        this.workers = new Thread[nWorkers];
        for (int i = 0; i < nWorkers; i++) {
            workers[i] = new Thread(this::workerLoop, "pool-worker-" + i);
            workers[i].start();
        }
    }

    private void workerLoop() {
        while (true) {
            Runnable task;
            try {
                task = queue.take();          // blocks until a task or poison pill arrives
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                return;
            }
            if (task == POISON_PILL) return;  // graceful stop: drained everything ahead of it
            try {
                task.run();
            } catch (RuntimeException e) {
                // One bad task must not kill the worker (and take a slot out
                // of the pool forever).
                System.err.println("[" + Thread.currentThread().getName() + "] task threw: " + e);
            }
        }
    }

    /** Fire-and-forget submit. Throws under ABORT if the queue is full. */
    public void submit(Runnable task) {
        if (shuttingDown.get()) throw new RejectedExecutionException("pool is shutting down");
        if (queue.offer(task)) return;         // non-blocking; succeeds only if there's room
        switch (policy) {
            case CALLER_RUNS:
                task.run();                    // backpressure: caller pays the cost directly
                return;
            case DROP_OLDEST:
                queue.poll();                  // evict the oldest queued (never mid-flight) task
                if (!queue.offer(task)) throw new RejectedExecutionException("full even after drop-oldest");
                return;
            case ABORT:
            default:
                throw new RejectedExecutionException("queue full (capacity=" + capacity + ")");
        }
    }

    /** Future-returning submit: task -> result, without blocking the caller. */
    public <T> Future<T> submit(Callable<T> task) {
        FutureTask<T> future = new FutureTask<>(task);
        submit((Runnable) future);
        return future;
    }

    /** Graceful shutdown: stop accepting work, drain what's queued, then stop workers. */
    public void shutdown() throws InterruptedException {
        shuttingDown.set(true);
        // One poison pill per worker, enqueued AFTER every already-submitted
        // task, so every worker finishes real work before it exits.
        for (int i = 0; i < workers.length; i++) queue.put(POISON_PILL);
        for (Thread w : workers) w.join();
    }

    /** Hard shutdown: drop whatever is queued, interrupt workers immediately. */
    public void shutdownNow() {
        shuttingDown.set(true);
        queue.clear();
        for (Thread w : workers) w.interrupt();
    }

    /** Current queue depth -- used by the break-it flood below to show the backlog. */
    public int queueSize() {
        return queue.size();
    }
}

Reference implementation — Go (worker goroutines + buffered channel + WaitGroup)

// Package pool: N goroutines ranging over a bounded buffered channel.
package pool

import (
    "errors"
    "sync"
    "sync/atomic"
)

type RejectionPolicy int

const ( Abort RejectionPolicy = iota; CallerRuns; DropOldest )

var ErrQueueFull = errors.New("pool: queue full, task rejected")

type Pool struct {
    tasks        chan func()
    wg           sync.WaitGroup   // tracks the N workers, for shutdown to wait on
    shuttingDown atomic.Bool
    policy       RejectionPolicy
    dropMu       sync.Mutex       // serializes the evict+retry pair for DropOldest
}

func New(nWorkers, queueCapacity int, policy RejectionPolicy) *Pool {
    p := &Pool{tasks: make(chan func(), queueCapacity), policy: policy} // BOUNDED channel
    p.wg.Add(nWorkers)
    for i := 0; i < nWorkers; i++ {
        go p.workerLoop()
    }
    return p
}

func (p *Pool) workerLoop() {
    defer p.wg.Done()
    // range exits automatically once the channel is CLOSED and DRAINED —
    // Go's channel semantics give us graceful shutdown for free here.
    for task := range p.tasks {
        task()
    }
}

func (p *Pool) Submit(task func()) error {
    if p.shuttingDown.Load() {
        return errors.New("pool: shutting down")
    }
    select {
    case p.tasks <- task:                 // non-blocking send: succeeds only if there's room
        return nil
    default:
        switch p.policy {
        case CallerRuns:
            task()                          // backpressure: run on the caller's own goroutine
            return nil
        case DropOldest:
            p.dropMu.Lock()
            defer p.dropMu.Unlock()
            select { case <-p.tasks: default: }   // evict the oldest queued task
            select {
            case p.tasks <- task:
                return nil
            default:
                return ErrQueueFull
            }
        default: // Abort
            return ErrQueueFull
        }
    }
}

// Shutdown stops accepting work, drains the channel, waits for all workers.
func (p *Pool) Shutdown() {
    p.shuttingDown.Store(true)
    close(p.tasks)   // no more sends allowed; queued tasks are still delivered to range
    p.wg.Wait()
}

// Future[T] mirrors Java's Future<T> — a handle to a result that isn't ready yet.
type Future[T any] struct { done chan struct{}; val T; err error }
func (f *Future[T]) Get() (T, error) { <-f.done; return f.val, f.err }

func SubmitFuture[T any](p *Pool, fn func() (T, error)) (*Future[T], error) {
    fut := &Future[T]{done: make(chan struct{})}
    err := p.Submit(func() { fut.val, fut.err = fn(); close(fut.done) })
    if err != nil { return nil, err }
    return fut, nil
}

Break it

The naive queue swap: build the exact same pool but pass an effectively unbounded capacity — which is precisely what Executors.newFixedThreadPool() does under the hood (its internal LinkedBlockingQueue has no bound). Then hammer it with a fast producer against a slow worker and watch what "unbounded absorbs everything" actually looks like:

// Java — this actually runs and actually proves the point (no capacity limit).
final class ThreadPoolBreakIt {
    static void sleep(long ms) {
        try {
            Thread.sleep(ms);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }

    public static void main(String[] args) {
        ThreadPool pool = new ThreadPool(1, Integer.MAX_VALUE, ThreadPool.RejectionPolicy.ABORT);
        long start = System.nanoTime();
        for (int i = 0; i < 50_000; i++) {
            pool.submit((Runnable) () -> sleep(1));   // worker: ~1ms/task -> ~50s to drain 50k tasks
        }
        long submitMillis = (System.nanoTime() - start) / 1_000_000;
        int depth = pool.queueSize();
// Measured: submitMillis ≈ 15ms, depth ≈ 49,990.
// The producer finished submitting FIFTY THOUSAND tasks in fifteen
// milliseconds, with ZERO rejections and ZERO slowdown — while the queue
// is quietly holding ~49,990 unprocessed tasks in memory. Nothing told the
// caller anything was wrong. That silent pile is the OOM/latency blowup
// in miniature: multiply the task payload size and the flood duration by
// realistic production numbers and this is exactly how a service goes
// down at 3am with a stack trace nowhere near the actual cause.
    }
}

The Go version is the same shape — a channel with capacity 1_000_000 instead of a small bound, flooded with 50,000 sends against one slow worker: submission finishes in single-digit milliseconds, len(tasks) reports ~49,990 buffered. Now swap that capacity for a small bounded one with ABORT (the M3 design) and rerun the same flood: submissions start failing with ErrQueueFull/RejectedExecutionException almost immediately — loudly, at the moment of overload, instead of silently, minutes later. That's the whole lesson: a bounded queue doesn't reduce capacity, it converts a hidden failure into a visible one you can act on (shed load, scale out, alert) before it becomes an outage.

Tests to write (this is the real learning)

All 8 Java assertions and all 6 Go tests (including go test -race) pass against the reference implementations above — compiled and run in this session with javac -Xlint:all -Werror and go vet/go build/go test -race, all clean.

Optimise — trade-offs

DecisionOption AOption BWhen A winsWhen B wins
Pool shapeFixed (N constant workers)Cached (grows/shrinks, e.g. Executors.newCachedThreadPool)Steady, predictable, CPU-bound load — you want a hard cap on concurrency and resource useShort, bursty, IO-bound tasks where idle threads are cheap to reap — but it reintroduces unbounded-thread risk under sustained load, so it needs its own cap in practice
Pool shapeFixed / cachedWork-stealing (Java ForkJoinPool, Go's runtime scheduler for goroutines)Independent, similarly-sized tasks from one queue (typical request handling)Recursive divide-and-conquer work (merge sort, tree traversal, parallel reduce) — each worker has its own deque and steals from others' tails when idle, so a few big subtasks don't starve idle workers the way one shared FIFO queue would
Rejection policyABORTCALLER_RUNSCaller can retry/degrade gracefully, or overload must be visible immediately (fail fast, alert, shed load)You want automatic backpressure without an exception path — the producer itself slows down because it's now doing the work, which naturally throttles submission rate to match consumption rate
Rejection policyABORT / CALLER_RUNSDROP_OLDESTEvery task matters (e.g. a payment, a write) — losing one silently is unacceptableOnly the freshest data matters (e.g. the latest sensor reading, a live price tick) — an old queued item is worthless once a newer one exists
SizingCPU-bound: ≈ #coresIO-bound: cores × (1 + wait/compute) — Little's LawTasks are compute-heavy with little blocking — more threads than cores just adds context-switch overhead with no throughput gainTasks spend most of their time blocked on IO (DB calls, network) — more threads keep the CPU fed while others wait; size using the wait:compute ratio, not a guess
Language idiomJava: explicit fixed poolGo: goroutines + a buffered channel as a semaphoreThreads are OS-expensive (~1MB stack) — Java needs an explicit cap or it exhausts memory/OS threadsGoroutines are cheap (~2KB starting stack, multiplexed onto few OS threads by the Go runtime) — Go idiomatically launches one goroutine per task and uses a buffered channel of size N purely as a concurrency limiter, not as a thread cap

Defend under drilling

You can now defend

Re-authored/Deepened for this guide. Model answer / theory: Thread Pools, Executors & Futures. Reference implementations (Java + Go) compiled and tested in-session: javac -Xlint:all -Werror clean, 8/8 assertions pass; go vet + go build + go test -race clean, 6/6 tests pass.

🤖 Don't fully get this? Learn it with Claude

Stuck on Build a Thread Pool? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.

🎨 Explain it visually

Build the mental picture, not memorization.

I just read a lesson on **Build a Thread Pool** (Hands-On Builds) and want to truly understand it. Explain Build a Thread Pool from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🤔 Walk me through it (interactive)

Socratic — adapts to where you're stuck.

Teach me **Build a Thread Pool** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧪 Quiz me & fix my gaps

Active recall exposes what you missed.

Quiz me on **Build a Thread Pool** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧠 Make it stick

Intuition + hook + flashcards for long-term memory.

Help me remember **Build a Thread Pool** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

📝 My notes

← Build a Readers–Writer LockBuild a Blocking Queue →