How Coroutine Dispatchers Actually Work

The question

Most Android documentation explains coroutine dispatchers like this:

Use Dispatchers.Default for CPU-intensive operations like sorting lists or parsing JSON. Use Dispatchers.IO for blocking I/O like reading files or database queries.

From that rule of thumb, it is easy to assume that Kotlin maintains two separate thread pools under the hood: one small pool sized to your CPU cores, and one large pool reserved for I/O operations.

That explanation is convenient, but the actual implementation in kotlinx.coroutines is much more elegant—and fundamentally different.

My initial mental model

For years, I assumed the architecture looked like this:

[Dispatchers.Default] ──► ThreadPool A (Threads = CPU Cores)
[Dispatchers.IO]      ──► ThreadPool B (Threads = up to 64)

In this model:

Switching context from Dispatchers.Default to Dispatchers.IO implies handing off a task to an entirely different thread pool, paying the cost of thread context-switching, cache invalidation, and queue synchronization.
Thread exhaustion in Dispatchers.IO would be completely isolated from Dispatchers.Default.

Neither of those assumptions is true.

What actually happens: The Shared Scheduler

Both Dispatchers.Default and Dispatchers.IO share the exact same instance of an internal class called CoroutineScheduler:

Dispatchers.Default ─┐
                     ├─► CoroutineScheduler ──► Pool of Worker Threads
Dispatchers.IO ──────┘                           (Parked / CPU / Blocking)

The difference between Default and IO is not which threads they own. The difference is the rules under which tasks are permitted to run:

CPU Permits: The scheduler maintains a finite number of “CPU permits,” equal to corePoolSize (typically the number of CPU cores on the device, minimum 2).
Non-blocking tasks (Dispatchers.Default): Must acquire one of these CPU permits before running. A worker holding a CPU permit is guaranteed that no more than corePoolSize threads are executing CPU-bound calculations simultaneously, avoiding excessive OS time-slicing.
Blocking tasks (Dispatchers.IO): Do not consume a CPU permit. Instead, they run up to a higher ceiling (maxPoolSize, defaulting to 64 or core count). When a worker enters a blocking state, it yields its CPU permit back to the scheduler, allowing another worker to wake up and execute Default tasks.

Let’s look at the implementation

If you inspect kotlinx.coroutines.scheduling.CoroutineScheduler.kt, you find the core worker lifecycle:

internal enum class WorkerState {
    CPU_ACQUIRED, // Actively executing a CPU-bound task (Default)
    BLOCKING,     // Executing a blocking task (IO)
    PARKED,       // Sleeping waiting for work
    DORMANT,
    TERMINATED
}

When you dispatch a task via Dispatchers.Default, the scheduler tags the runnable as TASK_NON_BLOCKING. When dispatched via Dispatchers.IO, it is tagged as TASK_PROBABLY_BLOCKING.

Here is what happens inside a worker thread when it picks up a task:

// Simplified from CoroutineScheduler.Worker.runWorker()
fun runWorker() {
    while (!isTerminated) {
        val task = findTask()
        if (task != null) {
            executeTask(task)
        } else {
            park()
        }
    }
}

private fun executeTask(task: Task) {
    val taskMode = task.mode
    if (taskMode == TASK_NON_BLOCKING) {
        // Must acquire CPU permit
        if (state != WorkerState.CPU_ACQUIRED) {
            tryAcquireCpuPermit()
        }
    } else {
        // Blocking task: release CPU permit so another thread can do CPU work!
        if (state == WorkerState.CPU_ACQUIRED) {
            releaseCpuPermit()
            state = WorkerState.BLOCKING
        }
    }
    task.run()
}

The surprising part: Zero-Cost Context Switches

Because both dispatchers share the same thread pool and worker queue, look what happens when you write this standard Android pattern:

suspend fun loadUserData(): UserData = withContext(Dispatchers.Default) {
    val processedId = computeHash(userId) // CPU bound
    
    val data = withContext(Dispatchers.IO) {
        database.load(processedId)        // Blocking IO
    }
    
    transformPayload(data)                // Back on Default
}

What thread runs database.load()?

In many cases, the exact same worker thread!

When execution reaches withContext(Dispatchers.IO):

The worker thread simply calls releaseCpuPermit().
It transitions its internal state from CPU_ACQUIRED to BLOCKING.
It signals the scheduler: “I am going to block on I/O. If there is pending CPU work, wake up another parked thread.”
It continues running your database query on the same hardware thread.

There is no thread hop, no IPC, and no OS context switch. When the query completes and execution returns to Dispatchers.Default, the worker re-acquires a CPU permit (parking only if all CPU permits are currently claimed by active threads).

Work-Stealing Mechanics

The CoroutineScheduler uses a classic work-stealing algorithm inspired by Java’s ForkJoinPool and Go’s scheduler:

Each worker thread owns a lock-free, single-producer circular array queue of capacity 128 (WorkQueue).
When a coroutine resumes, it is pushed directly to the current thread’s local queue (cache locality).
If a worker drains its own queue, it attempts to steal tasks from sibling workers before falling back to the global shared queue.
Only if all queues are empty does the thread park via LockSupport.parkNanos().

Practical consequences for Android developers

1. `Dispatchers.IO` starvation can affect `Dispatchers.Default`

Because both dispatchers share a thread pool ceiling (maxPoolSize), if rogue code spawns 64 blocking tasks that block indefinitely without using suspending APIs, you can saturate the scheduler’s ability to create new workers. Always use proper timeouts on blocking calls.

2. The power of `limitedParallelism`

In modern kotlinx.coroutines, Dispatchers.IO.limitedParallelism(n) creates a view that restricts concurrent execution to n tasks without allocating a new thread pool:

// Limits disk writes to 2 concurrent operations without creating a new Executor:
val diskDispatcher = Dispatchers.IO.limitedParallelism(2)

Tasks dispatched to diskDispatcher are queued internally and fed into the shared CoroutineScheduler only when permits are free.

What I would remember

Default and IO are policies, not pools: They share a single pool of workers managed by CoroutineScheduler.
Context switching can be free: Switching between Default and IO often simply re-labels the current worker thread rather than bouncing to another thread.
CPU permits prevent thrashing: The scheduler dynamically balances CPU-intensive algorithms and blocking I/O on the same hardware.