llm-batcher / inference proxy

Why I added microbatching

A proxy under load shouldn't fire one uncoordinated upstream call per client request. It should group requests that arrive together and release them in controlled waves. Here's the why, the trade, and the proof.

See it

The thundering herd, then the fix

Same burst of requests. Left: every request hits the upstream the instant it arrives. Right: requests are windowed and released under a concurrency cap. Hit play.

Direct path no batching

arrivals
Anthropic
Messages API
Peak concurrent upstream
0
Window calls
0

Microbatch path window + semaphore

arrivals
Anthropic
Messages API
Peak concurrent upstream
0
Added wait
0 ms
cap = 8 concurrent  ·  window = 20 ms / 16 max
The judgment call

I rejected the literal answer on purpose

The roadmap said "accumulate requests into one upstream call." Anthropic literally has an API for that. Using it here would have been wrong.

Message Batches API ✗ (for this endpoint)

  • Offline / asynchronous bulk primitive
  • SLA up to 24 hours (submit, then poll)
  • 50% cheaper, great for evals and bulk jobs
  • Behind a synchronous chat endpoint it would block callers for minutes : surprising and timeout-prone

In-process microbatching ✓

  • Realtime: adds a few ms, not minutes
  • Single choke point for concurrency & failure isolation
  • Protects the upstream from bursts
  • Correct trade for a low-latency, OpenAI-compatible API

Picking the right primitive over the same-named one is the actual engineering. The Batches API still has a home later, as a separate async endpoint for bulk work. See the Batches API docs vs the realtime Messages API.

How

The pipeline

Two flush triggers, one shared client, one concurrency gate, hard timeouts so nothing can hang.

clients
admission window
size OR time
semaphore
cap N in-flight
shared httpx client
Anthropic

Flush when EITHER fires

  • Size: window hits BATCH_MAX_SIZE → flush now
  • Time: BATCH_MAX_WAIT_MS since first item → flush
  • The timer never cancels itself (a real async race I guarded against)

Guarantees on dispatch

  • Concurrency cap via asyncio.Semaphore
  • Failure isolation via gather(return_exceptions=True)
  • No hangs via per-window wait_for timeout
  • Clean shutdown fails any leftover futures
async def run_one(item):
    async with self._sem:                      # concurrency cap
        return await self._dispatch_one(item.params)

results = await asyncio.wait_for(              # per-window timeout
    asyncio.gather(*(run_one(it) for it in batch), return_exceptions=True),
    timeout=self._batch_timeout_s,             # one stuck call can't hang the rest
)
The trade

Latency for stability, on purpose

+ a few ms wait
bounded upstream load, smoother tails

Why a concurrency cap is the right lever: Little's Law says in-flight work L = λ × W. When upstream latency W spikes, holding L fixed with a semaphore stops in-flight work from exploding. You pay a little queueing delay to avoid an unbounded meltdown.

Not exotic

The same pattern, under other names

LLM serving: continuous batching

vLLM / TGI batch tokens across concurrent requests to keep the GPU busy. Same instinct, lower layer.

vLLM · PagedAttention paper

TCP: Nagle's algorithm

Buffer small sends briefly to coalesce them into fuller packets. A size/time window, exactly like ours.

Nagle's algorithm

Backends: DataLoader

Coalesce many field fetches in one tick into a single backend call. Batch-per-tick.

graphql/dataloader

Databases: group commit

Many writes share one fsync. Trade a touch of latency for far more throughput.

Thundering herd
Knobs

Configuration (all opt-in)

VariableDefaultMeaning
BATCH_ENABLED0Master switch. Off keeps the plain realtime path.
BATCH_MAX_SIZE16Flush once the window holds this many requests.
BATCH_MAX_WAIT_MS20Max wait for a window to fill before flushing.
BATCH_MAX_CONCURRENCY8Max simultaneous in-flight upstream calls.
BATCH_TIMEOUT_S60Hard ceiling on one window's dispatch.
Go deeper

Concepts behind this

Every term that matters here, with a primary source.

Admission control Backpressure Little's Law Thundering herd The Tail at Scale Nagle's algorithm PagedAttention / vLLM asyncio.Semaphore asyncio.gather asyncio.wait_for FastAPI lifespan httpx connection pooling Anthropic Batches API Anthropic Messages API