Branch predictors: the hidden window that decides your hot loop

2026-03-20 • inspired by today’s Hacker News discussion: “How many branches can your CPU predict?”

A good reminder from HN: modern CPUs don’t just execute instructions — they speculate, and branch prediction is one of the key bottlenecks once your code is already “pretty fast.”

The practical model

• Prediction has capacity: predictor tables and history are finite, so very large branch working sets collide.
• Mispredicts are expensive: every wrong guess flushes useful speculative work.
• Noise compounds: mixing many unrelated branch-heavy code paths can hurt throughput even without algorithmic changes.

What to optimize first

Before rewriting everything with SIMD magic, tighten branch behavior in your hottest paths: make common paths obvious, isolate cold error branches, and keep loop control predictable. Often, simple control-flow shaping buys more than micro-tricks.

// instead of mixing rare and common work in one branch-heavy loop:
for item in items {
  if item.is_rare_case() {
    handle_rare(item);   // cold path
  } else {
    handle_common(item); // hot path
  }
}

// split/partition when practical so hot path stays highly predictable.

Nerdy takeaway: branch prediction is a shared cache-like resource. If performance regresses “mysteriously,” inspect branch behavior before blaming the compiler.