What is Multithreading and Concurrency

The mechanism in one sentence

An OS scheduler hands each runnable thread a brief time slice of a CPU core, then preempts it and runs another, so on one core threads take turns (interleaving) while on several cores they literally run at the same instant (parallelism) — concurrency is the program structure that lets either happen, parallelism is the hardware actually doing it.

That single distinction is the whole lesson. The old framing “multithreading helps achieve parallelism” is the trap: you can have concurrency on a single-core machine with zero parallelism, and you can have parallelism (e.g. SIMD vector instructions) with no threads at all. Keep them separate:

• Concurrency = dealing with many tasks at once — how you decompose and structure the program so independent tasks can make progress in any interleaving. A property of the code.
• Parallelism = doing many tasks at the same instant — multiple cores executing instructions simultaneously. A property of the execution.
• A thread = the unit the scheduler moves between these states: one sequential stream of instructions with its own stack and program counter, sharing heap memory with its siblings.
• Multithreading = a single process running multiple such threads. Whether they run concurrently (interleaved) or in parallel (simultaneously) depends entirely on how many cores are free.

Rob Pike's line is the one to remember: “Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.”

A traced example: 3 tasks, 1 core vs 2 cores

Say a request handler must do three independent CPU jobs, each needing 30 ms of CPU work: A (resize image), B (compute checksum), C (render thumbnail). We model the scheduler with a 10 ms time slice (real Linux CFS slices are dynamic, ~1–6 ms; 10 ms keeps the trace readable).

Case 1 — concurrent on 1 core (no parallelism)

The scheduler round-robins one slice at a time. Total work is 90 ms and one core does 1 ms of work per ms, so wall-clock = 90 ms. Concurrency bought us interleaved progress and responsiveness — not speed.

Case 2 — parallel on 2 cores

Now two cores run truly simultaneously. A and B start together; whichever core frees first picks up C. Total CPU work is still 90 ms, but two cores deliver 2 ms of work per wall-clock ms, so the floor is 90/2 = 45 ms.

That naive split finishes in 60 ms because C couldn't start until a core freed at 30 ms — the third task serializes behind the first two. Speedup = 90/60 = 1.5×, not 2×, even with 2 cores. This is Amdahl's law in miniature: with 3 equal tasks on 2 cores the critical path is 2 sequential 30 ms chunks. The lesson: adding cores does not linearly add speed — the dependency/granularity structure caps it.

The same logic in code: Java vs Go

Both run tasks A, B, C concurrently and collect their results. Watch what the runtime schedules onto cores.

Java — platform threads via an executor + futures

import java.util.concurrent.*;
import java.util.List;

public class Tasks {
    // Each task: simulate 30 ms of CPU work, return a labelled result.
    static String work(String name) {
        long deadline = System.nanoTime() + 30_000_000L; // 30 ms
        long spins = 0;
        while (System.nanoTime() < deadline) spins++;     // busy-work
        return name + " done (" + spins + " spins)";
    }

    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newFixedThreadPool(2); // cap at 2 cores
        List<Future<String>> futures = List.of(
            pool.submit(() -> work("A")),
            pool.submit(() -> work("B")),
            pool.submit(() -> work("C")));
        for (Future<String> f : futures) System.out.println(f.get()); // join + collect
        pool.shutdown();
    }
}

A fixedThreadPool(2) means at most two of A/B/C run at once; the third waits — exactly the 60 ms critical path from the trace. Each Java thread here is a platform thread, a 1:1 wrapper over an OS thread (~1 MB stack), which is why you pool them rather than spawning thousands. (Java 21+ adds virtual threads, closer to Go's model — that's a later lesson.)

Go — goroutines + a channel

package main

import (
    "fmt"
    "runtime"
    "time"
)

func work(name string, out chan<- string) {
    deadline := time.Now().Add(30 * time.Millisecond)
    spins := 0
    for time.Now().Before(deadline) { // busy-work, 30 ms
        spins++
    }
    out <- fmt.Sprintf("%s done (%d spins)", name, spins)
}

func main() {
    runtime.GOMAXPROCS(2) // cap parallelism at 2 cores
    out := make(chan string)
    for _, name := range []string{"A", "B", "C"} {
        go work(name, out) // spawn a goroutine
    }
    for i := 0; i < 3; i++ {
        fmt.Println(<-out) // receive blocks until a result arrives
    }
}

Each go work(...) launches a goroutine — a user-space task with a tiny ~2 KB growable stack, multiplexed M:N onto a small pool of OS threads by the Go runtime. The channel out is how goroutines hand results back: the receiver blocks until a value is sent, which both synchronizes and transfers the result in one operation.

Where the two runtimes differ

Pitfalls

• Calling concurrency “parallelism.” Spawning 100 threads on a 4-core box gives you 100-way concurrency but only 4-way parallelism; the other 96 wait for a slice. Saying “I added threads so it's parallel now” is wrong and leads to the next pitfall.
• Expecting N threads → N× speed. The trace showed 2 cores giving only 1.5×. Amdahl's law caps you at the inverse of the sequential fraction; if 20% of the work can't parallelize, the ceiling is 5× no matter how many cores you throw at it.
• Oversubscription thrashing. More CPU-bound threads than cores means the scheduler spends time context-switching (saving/restoring registers, blowing CPU caches) instead of computing. Past the core count, throughput goes down. Pool size ≈ core count for CPU work; only go higher for I/O-bound work that spends most of its time blocked.
• Forgetting threads share the heap. The whole point — shared memory — is also the hazard. Two threads writing the same field with no synchronization is a data race, and the result is non-deterministic. Both f.get() in Java and <-out in Go are doing double duty here: they collect results and impose a happens-before ordering so you read the finished value safely. (Race conditions and the memory model are the next lessons.)
• Confusing “concurrent” with “faster.” On one core, concurrency adds scheduling overhead and is slightly slower in total CPU time. What it buys is responsiveness — the UI thread isn't frozen while a download runs — not raw throughput.

Takeaways

• Concurrency is structure; parallelism is execution. Concurrency lets tasks make progress in any interleaving; parallelism is multiple cores running them at the same instant. You can have either without the other.
• A thread is the schedulable unit — one instruction stream with its own stack, sharing the heap with siblings. Multithreading = many threads in one process; whether they run in parallel depends on free cores, not on the code.
• Cores cap speed, structure caps it harder. Dependencies, task granularity, and the sequential fraction (Amdahl) mean adding cores yields sub-linear, eventually negative, returns.
• Java uses heavyweight 1:1 OS threads (pool them); Go uses lightweight M:N goroutines (spawn freely) coordinated by channels. Same concurrency idea, very different cost and idiom.

Sources: Rob Pike, “Concurrency Is Not Parallelism” (Go talk, 2012) for the structure-vs-execution distinction; Brian Goetz et al., Java Concurrency in Practice (Ch. 1) for the thread/process model and pool sizing; The Go Programming Language (Donovan & Kernighan, Ch. 8–9) for goroutines and channels; Gene Amdahl's 1967 paper for the speedup ceiling; Linux CFS documentation for time-slice behaviour. Re-authored and deepened for this guide — replaced the pure office/cooking analogy and the “multithreading achieves parallelism” conflation with an explicit mechanism, a traced 1-core-vs-2-core example, a timeline diagram, and side-by-side Java/Go code.

Work–span model (Cilk) for the 1.5× miniature

The 90 ms / 60 ms / 1.5× story connects to the classic work–span model used in Cilk and modern DAG-parallel runtimes — but with a subtlety worth getting exactly right, because the 1.5× here is not the work–span lower bound talking.

Mechanism: work W, span T_∞, processors P

• Work W = total CPU-time of all tasks if run on one core. Here A+B+C = 90 ms (this is also T1, the one-core time).
• Span T_∞ (critical path) = length of the longest chain of dependent work in the dependency DAG, i.e. the time on infinitely many cores. It is a property of the DAG, not of the processor count. The three tasks here are independent, so the longest dependency chain is a single task: T∞ = 30 ms.
• Greedy-schedule bound (Brent): max(W/P, T_∞) ≤ T_P ≤ W/P + T_∞. For P=2 the lower bound is max(90/2, 30) = max(45, 30) = 45 ms. So the continuous theory predicts a 2× floor.
• Why we actually get 60 ms, not 45. The bound above assumes work is infinitely divisible. Our tasks are atomic 30 ms units — you cannot run half of A on core 0 and half on core 1. Three indivisible 30 ms tasks on two cores need ⌈3/2⌉ = 2 scheduling rounds = 60 ms, which exceeds the 45 ms continuous floor. Speedup = 90/60 = 1.5×. This gap is a granularity effect, not a dependency and not the span.

Dependency DAG for this miniature: three nodes A,B,C with no edges between them (fully independent), plus a virtual source→{A,B,C}→sink join. Nothing forces serialization at the DAG level; the 1.5× comes purely from too few cores for indivisible tasks.

With infinite cores the same DAG finishes in T_∞ = 30 ms (one layer of three parallel tasks) → speedup ≤ T₁/T_∞ = 90/30 = 3×. So the 1.5× on two cores is a finite-cores-plus-indivisible-tasks effect, not a data dependency. Staff follow-up: if C depended on A's output, an edge A→C would raise the span to T_∞ ≥ 60 ms even with infinite cores — that would be a genuine critical-path limit rather than a granularity one.

When not to use the work–span bound alone

• It ignores cache, NUMA, false sharing, and lock contention — those raise effective W and T_∞.
• I/O-bound work has "span" dominated by wait, not CPU; use Little's Law / concurrency, not W/P.

Failure / ops fingerprint

Dashboard shows 16 cores, p50 latency only ~1.5× better than single-thread for a "fully parallel" job → measure ready-queue depth and critical path (tracing spans), not just core count. Often the join barrier or a single serial reduce is the hidden T_∞.

Drill ladder

• L1: Name W and T_∞ for the 3×30 ms / 2-core schedule.
• L2: What is T_∞ if C must run after A completes, still on 2 cores? On infinite cores?
• L3: Four independent 10 ms tasks on 3 cores — bound T_P and max speedup vs serial.
• L4 (hostile): "We have 100 cores so this should be 100×." What two quantities do you demand before agreeing?