draft
lore:python-async/run-in-executor-default-pool-starvation/0006 (v1.0.0)
loop.run_in_executor(None, ...) all share one bounded default ThreadPoolExecutor; saturating it with blocking calls makes every other default-executor call queue and stall. Give bulk blocking work its own executor.
Verbatim Symptoms (Primary Search Key)
- run_in_executor calls hang or stall under load
- unrelated async task blocked waiting for a free thread pool worker
- quick executor call takes seconds when many blocking calls are running
- asyncio app latency spikes when offloading many blocking functions to the default executor
Problem Statement
loop.run_in_executor(None, func) uses the event loop's single shared default ThreadPoolExecutor, whose max_workers is bounded (min(32, cpu_count+4) by default). When that many blocking calls are in flight, every additional run_in_executor(None, ...) call — including unrelated ones elsewhere in the app — queues behind them and stalls until a worker frees up. Because the pool is shared and its size is small, one blocking workload silently starves the latency of everything else offloaded to the default executor. The size varies by machine, so the bug can hide in dev and surface under production concurrency.
Correct Procedure
- Create a dedicated ThreadPoolExecutor for the bulk/blocking workload and pass it explicitly: `loop.run_in_executor(bulk_ex, func, ...)`, leaving the default executor free for latency-sensitive calls.
- Size the dedicated pool for its workload; use `with ThreadPoolExecutor(...) as ex:` so it is shut down deterministically.
- For CPU-bound work, use a ProcessPoolExecutor instead — threads cannot parallelize CPU work under the GIL and will still starve the pool.
- If you must share the default pool, cap concurrency yourself (e.g. an asyncio.Semaphore) so it never fully saturates.
Anti-Patterns
- Assuming run_in_executor(None, ...) gives unbounded parallelism — it is capped by one small shared pool.
- Offloading many long blocking calls to the default executor alongside latency-sensitive ones — the latter starve.
- Using threads (any executor) for CPU-bound work and expecting speedup — the GIL serializes it and the pool stays saturated.
Documented Dead Ends
- Offloading all blocking work to the default executor via run_in_executor(None, ...) and expecting each call to run promptly: The default pool has only min(32, cpu+4) workers; once saturated, further calls queue. Latency-sensitive calls sharing the default pool stall behind the bulk work with no error raised. (~35 mins wasted)