Async, virtual threads, goroutines: who pays for the wait?

Your service has ten thousand requests in flight, and almost none of them are doing anything. They’re waiting on the database, or on some downstream API that is itself waiting on its database. An OS thread is the wrong tool for waiting. It reserves a stack measured in megabytes, and parking and resuming it means a trip through the kernel scheduler. Ten thousand of them is a memory bill and a context-switching tax you pay for the privilege of doing nothing.

So every mainstream runtime has built machinery for waiting without a thread. C# rewrites your method into a state machine. Java gives you a million cheap threads. Go never gave you real threads in the first place. Rust hands you a state machine and refuses to ship a scheduler for it. They all solve the same problem, and they disagree about everything else.

The disagreement makes sense once you follow the money: the cost of suspending a call never disappears; each runtime only chooses who pays it. Sometimes the caller pays, in annotations that spread through the codebase. Sometimes the foreign-function boundary pays, and sometimes it’s the runtime team, for a decade. This post walks the ledger across .NET (including Runtime Async in .NET 11 and the green-thread experiment whose death produced it), Java’s virtual threads (including the pinning saga), Go, Kotlin, and Rust, and ends with a table for choosing and a verdict.

One function, five runtimes

To keep the comparison honest, every section implements the same function: fetch a URL, wait on the network, return the body length. The interesting part is never the code. It’s what the runtime does at the moment the response hasn’t arrived yet.

There are only two mechanisms for parking that moment, plus one discipline that rides on top. Naming them now saves re-explaining them five times:

1. The rewrite. A compiler (or the runtime itself) transforms your function so its local variables live in an object instead of a stack frame. Suspending means returning; resuming means calling back into the object. C#, Kotlin, and Rust do this.
2. The movable stack. The runtime gives each task a real call stack, but a small one it can grow, shrink, and copy to the heap while the task waits. Your code is not transformed at all. Go, Java’s virtual threads, and Erlang’s processes do this (Erlang’s famous actor model is the discipline of mechanism 3 layered on top of them, and the one platform that made the combination mandatory). The family is often called green threads, a name from the original Java runtime, where Sun’s Green Team multiplexed every Java thread onto a single OS thread. That M:1 design is long dead: every implementation in this post schedules many stacks across many OS threads (M:N), so the old “no real parallelism” objection no longer applies. Fiber is the other name you’ll meet for the same idea: Windows has shipped Win32 fibers since the nineties, PHP and Ruby call their stackful coroutines fibers, and Project Loom itself called its construct fibers before renaming it to virtual threads. Usage is loose, but fiber usually implies the cooperative flavor, a movable stack you schedule yourself rather than hand to a runtime.
3. The mailbox. Not a suspension mechanism but a state-ownership discipline (actors) that sits on top of either one. It gets its own short section, because it keeps being sold as a competitor to the other two when it isn’t.

.NET async/await: the caller pays, in color

Since C# 5 in 2012, the .NET answer has been the rewrite:

public async Task<int> FetchLengthAsync(HttpClient http, string url) {
    string body = await http.GetStringAsync(url);
    return body.Length;
}

The compiler turns this method into a state machine: url and body become fields, the code becomes a MoveNext method with a switch over the current state, and when the method first hits an await that isn’t already complete, the whole thing lands on the heap and the thread walks away. Resumption is a callback invoking MoveNext again. It is efficient, it needed no runtime changes, and it has one famous cost.

That cost is function coloring, named by Bob Nystrom’s 2015 essay What Color Is Your Function?. An async method can only be awaited from another async method, so the keyword propagates up the call chain until Main itself is async. Sync and async versions of the same API split the ecosystem in two (Read and ReadAsync, all the way down the BCL). And the bridge between the colors is a trap: blocking on a task with .Result deadlocks under a synchronization context and starves the thread pool without one. More on that in the pitfalls.

The color is the famous cost, but daily life with the rewrite is a list of smaller taxes. An async Task method allocates the Task, plus the boxed state machine once it actually suspends, so hot paths push you toward ValueTask, which arrives with rules the compiler does not enforce: await it exactly once, never concurrently, never block on it. The pooled machinery underneath (IValueTaskSource, the pooling method builder) is expert-only territory. Library authors write ConfigureAwait(false) on nearly every await to stay usable from context-bound callers. async void turns an escaped exception into a process crash and exists anyway, because event handlers needed it.

Even eliding the keyword is a trap. Returning the inner Task directly is a legitimate optimization that skips one state machine, and it changes semantics in a way the compiler won’t mention:

public Task<int> FetchLengthAsync(string url) {
    using var http = CreateClient();
    return FetchCoreAsync(http, url); // http is disposed while the fetch still runs
}

With async and await in place, the state machine keeps the using scope alive across the suspension. Without them, the method returns immediately and disposes the client under the running task’s feet. The optimization and the bug are the same edit.

So the rewrite’s ledger has two honest columns. On one side, await is a visible record of every point where your function can pause, lose its timing assumptions, and resume on a different thread; coloring is the receipt for the wait. On the other, the receipts come with bookkeeping: allocations, rulebooks, and seams. Both columns matter for the comparison ahead.

The green-thread experiment: .NET priced the alternative

In 2022 and 2023 the .NET team built the other mechanism, a real green-threads prototype: movable stacks in the CLR, wired into sockets and ASP.NET Core far enough to run an end-to-end web API where plain synchronous code did asynchronous I/O. Then they measured, and the ledger came back with three lines on it.

Raw throughput was close but behind: 162,019 requests per second on the ASP.NET plaintext benchmark against 178,620 for async/await, a gap the team thought optimization could narrow. Interop was the disaster. A P/Invoke call from a green thread has to switch off the movable stack onto a real one and back, and 100 million calls went from 300 ms to roughly 1,800 ms, six times slower, before even considering how shadow stacks and thread-local native state interact with stacks that move. And the two models composed badly in the wrong direction: green-threaded code calling the existing async ecosystem needed sync-over-async internally, the exact anti-pattern the platform spends its documentation warning about.

The conclusion, September 2023: “We have chosen to place the green threads experiment on hold” and keep improving async/await instead. The deciding argument wasn’t any single number. It was that .NET already has a colored ecosystem; fifteen years of libraries speak Task. A second, incompatible model wouldn’t replace the first one, it would coexist with it forever, and every boundary between them would be a bug farm.

Runtime Async in .NET 11: same color, new machinery

The promised improvement now has a name. Runtime Async, a preview feature in .NET 11, moves the rewrite out of the C# compiler and into the runtime. Your source doesn’t change; async and await mean what they always meant. But the compiler stops emitting state-machine classes, and the JIT instead generates suspension and resumption logic directly, with the runtime tracking the continuation.

<PropertyGroup>
  <Features>runtime-async=on</Features>
</PropertyGroup>

The practical wins are the ones state machines always obstructed. A live stack trace through three nested awaits shrinks from 13 frames of AsyncMethodBuilderCore.Start plumbing to the 5 frames you actually wrote, which is what your profiler and debugger see. Breakpoints bind inside async methods and stepping no longer detours through generated code. The runtime reuses continuation objects and skips ExecutionContext capture when there’s nothing to restore, shaving allocations in hot async paths. In .NET 11 the runtime libraries themselves are compiled this way; the state machines are gone from the BCL.

Note what did not change: the color stays. Runtime Async clears the mechanical rows of the ledger above (the frames, part of the allocations, the debugger detours) and leaves the model rows untouched: the ValueTask rulebook, the ConfigureAwait boilerplate, async void, and the eliding trap all survive. It is .NET doubling down on the experiment’s verdict. If you were hoping .NET 11 would un-color your functions, it won’t, and after the green-threads numbers above, you know why.

One row should shrink over time, though. ValueTask earned its place by making the synchronous fast path allocation-free, and Runtime Async attacks the same waste at the root: when the consumer directly awaits the callee, the runtime can hand the result across without materializing the intermediate Task machinery, and the JIT inlines await-less fast paths outright. Asked whether that makes ValueTask obsolete, Stephen Toub’s early answer is no with footnotes: pooled IValueTaskSource implementations (sockets), libraries targeting runtimes without the feature, and tasks consumed by something other than a direct await keep their reasons, and official guidance doesn’t exist yet. But if the only reason you were reaching for ValueTask in new code was the sync fast path, this is the feature meant to make that reflex unnecessary.

Java virtual threads: just write the blocking code

Java took the fork .NET declined, and it’s worth being precise about why it could. JEP 444 shipped virtual threads in JDK 21 (September 2023, the same month .NET shelved its prototype): movable stacks managed by the JVM, mounted onto a small pool of OS carrier threads. When a virtual thread blocks, the JVM unmounts it, parks its frames on the heap, and lends the carrier to another virtual thread.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    Future<Integer> length = executor.submit(() -> {
        String body = http.send(request, BodyHandlers.ofString()).body();
        return body.length();   // plain blocking code; the JVM parks the stack
    });
}

Sit with how much of the previous two sections simply does not apply here. No second color, so no viral keyword and no split ecosystem. No ConfigureAwait, because there is no context to lose. No ValueTask rulebook, no async void, no eliding trap, because there is no per-method machinery to optimize away. A stack trace is a stack trace, a thread dump reads top to bottom, and the debugger steps through blocking calls the way it did in 2005. Adoption is configuration rather than rewrite: Jetty and Tomcat accept a virtual-thread executor, Spring Boot 3.2 turns it on with spring.threads.virtual.enabled=true, and twenty-five years of blocking libraries (JDBC, InputStream, servlet filters) scale unmodified. You think about the domain; the runtime thinks about the waiting. As a developer experience, this is the strongest offer on this page.

Java could afford it precisely where .NET couldn’t. The JVM controls almost everything beneath your code, and typical Java stacks make little use of native interop, so the boundary that cost .NET a 6x P/Invoke penalty barely exists. And Java’s ecosystem was still blocking-first, so cheap blocking rescues the majority model instead of competing with it. Concurrency models are ecosystem properties more than language features, and both platforms, looking at opposite ecosystems, made opposite calls correctly.

The devil is in the details, though, and virtual threads have three worth knowing.

The first is pinning, the retrofit’s visible seam. A virtual thread that blocked inside a synchronized block or method got pinned: the JVM couldn’t unmount it, because the monitor implementation tracked the carrier rather than the virtual thread, so the carrier sat hostage for the duration. With a default carrier pool sized to your CPU count, a handful of pinned threads could freeze an entire service (the pitfalls section has the deadlock recipe). JEP 491 fixed this in JDK 24 (March 2025) by making monitors track the virtual thread itself; synchronized and Object.wait() no longer pin. The fix is in the JDK 25 LTS, but code running on the JDK 21 LTS still lives with the old behavior, and native frames (JNI) still pin even today.

The JNI case is not a leftover bug; it is how Java dodged the stack-switching tax .NET measured. A mounted virtual thread executes directly on its carrier’s ordinary OS stack, and the copy to the heap happens only when it parks. So a native call costs exactly what it costs on a platform thread, because by the time it runs, the stack under it is a perfectly normal one. In exchange, the JVM cannot park a stack with native frames on it (it can relocate Java frames, but knows nothing about C frames), so it pins instead. Where the .NET prototype and Go’s cgo pay at every native call, Java pays at every park and, during native calls, in scheduling flexibility rather than time. That bet fits an ecosystem that parks constantly and calls native rarely, the exact inverse of the profile that sank the .NET experiment. The broader lesson stands, though: retrofits take years to stop leaking, and Java spent those years in public.

The second is cooperative scheduling. A virtual thread yields its carrier at blocking points; a CPU-bound loop has none and holds its carrier indefinitely. Go had this exact gap and closed it with signal-based preemption in 2020. The JVM so far asks you to route heavy computation to a separate pool.

The third is the honest version of the trade: virtual threads move the wait out of your signatures and into your tooling. Where C# tells you at the call site that a pause can happen there, Java tells you in a JFR recording or a profiler, after the fact. For most server code that is the right trade, because the async annotations had metastasized to every method anyway and carried no information density. But it is a trade, and one part of the model is still arriving: structured concurrency, which would give lifetimes and cancellation the discipline Kotlin gets from its color, remains in preview as of JDK 25. Today you get the plain code first and the guardrails later.

Go: one color for everything, paid at the border

Go is what a platform looks like when it is born green instead of retrofitted. Since 2009, every goroutine is a movable stack, starting around 2 KB and grown by copying. There is no second kind of thread to be pinned to and no colored subset of the language:

func fetchLength(url string) (int, error) {
    resp, err := http.Get(url)   // parks the goroutine, not an OS thread
    if err != nil {
        return 0, err
    }
    defer resp.Body.Close()
    body, err := io.ReadAll(resp.Body)
    return len(body), err
}

go fetchLength("https://example.org")   // that's the entire concurrency API

The runtime mediates everything: a network poller turns blocking calls into goroutine parks, a work-stealing scheduler multiplexes goroutines over OS threads, and since Go 1.14 the runtime preempts even tight CPU loops with a signal, closing the cooperative-scheduling hole Java still has.

The ledger balances at the border and in the plumbing. Calling C via cgo means leaving the movable stack for a real one, with bookkeeping that makes an FFI call an order of magnitude more expensive than a plain function call; the .NET experiment’s P/Invoke cliff is not a .NET quirk, it is the price of mechanism 2, and Go simply decided its ecosystem would be Go-native enough to absorb it. Every function call carries a stack-growth check. The garbage collector must scan and fix up all those movable stacks. None of this is free; all of it was paid by the runtime team, once, before you arrived. Erlang made the same trade two decades earlier, with preemptive scheduling and per-process heaps on the BEAM, and remains the existence proof that the model scales when the whole world is built for it.

Born green doesn’t mean gotcha-free, though; Go’s classics just live elsewhere. The standard library has no structured concurrency, so a goroutine blocked on a channel nobody will ever read again leaks quietly forever; context discipline and the errgroup package exist because of it. Goroutines share memory, so data races are still yours to make, and the race detector belongs in CI rather than on your laptop. And loop-variable capture in go func closures bit enough people that Go 1.22 changed the language’s semantics to remove the trap.

Kotlin coroutines: coloring, chosen and made to pay rent

Kotlin sits in an instructive spot: a rewrite-family language running on the JVM, designed before Loom existed. A suspend function is a color. The compiler performs a continuation-passing-style transform, appending a hidden Continuation parameter and building a state machine, the same family as C#:

suspend fun fetchLength(client: HttpClient, url: String): Int =
    client.get(url).bodyAsText().length   // callable only from suspend context

Kotlin couldn’t change the JVM, so it couldn’t choose movable stacks. What it did instead was make the annotation earn its keep. Coroutines are structured: every coroutine launches inside a scope, cancellation flows down the hierarchy automatically, and a parent doesn’t complete until its children do. The suspend keyword also marks every cancellation point, so “this call can be cancelled here” is visible in the signature. That is the strongest version of the receipt argument: if you must carry a color, let it carry lifetime and cancellation semantics, not just “this might pause”. Java’s equivalent story is still in preview a decade later; C#’s CancellationToken is manual plumbing by comparison.

The blocking JVM underneath is bridged by Dispatchers.IO, a thread pool for wrapping blocking calls, and on JDK 21+ you can mount coroutines on virtual threads and let the two models cooperate. The color has running costs here too: runBlocking reintroduces the sync-over-async bridge, deadlocks included; a blocking call that sneaks onto Dispatchers.Default (sized to the core count) stalls every coroutine queued behind it; and structured concurrency’s default of cancelling all siblings when one child fails is correct, and still surprises every team exactly once, until they meet SupervisorJob. Outside the JVM, on Android and in multiplatform code, coroutines are simply the only game in town, which is the other reason the color was worth choosing.

Rust: the rewrite, minus the runtime

Rust ran the green-threads experiment too, a decade before .NET did, and reached the same verdict for harsher reasons. Pre-1.0 Rust shipped green threading in libgreen; RFC 230 removed it in 2014 because a mandatory runtime taxed every program, including the embedded and C-interop programs Rust existed for. The principle has a name in Rust culture: you don’t pay for what you don’t use.

So Rust’s async is the rewrite in its purest form. An async fn compiles to an enum state machine that is completely inert: it does nothing until something polls it, and the something is a library, not the language. In practice that library is Tokio:

async fn fetch_length(url: &str) -> Result<usize, reqwest::Error> {
    let body = reqwest::get(url).await?.text().await?;
    Ok(body.len())
}

The upside is real: no allocation per await by construction, no runtime in binaries that don’t want one, executors tailored to embedded targets or io_uring. The cost is the deepest coloring of any language here. The ecosystem splits into sync and async crates. Spawning a task demands Send + 'static bounds that surface as compiler errors two layers from their cause. Self-referential state machines required Pin, an API famous enough for its unfriendliness that the async working group treats it as a known wound. async fn in traits only became usable on stable in Rust 1.75, at the end of 2023, and still has dyn caveats. And cancellation is a drop: any future can simply be discarded at an await point, so a select! loop can lose a half-completed read unless every branch is cancellation-safe, a property the documentation has to warn you about because the compiler can’t check it. Rust made the wait cheapest at runtime and most expensive at the keyboard.

Two mitigations matter in practice. First, ordinary OS threads are a respectable answer in Rust more often than elsewhere: Send and Sync make data races compile-time errors, so threads plus channels cover modest concurrency with none of the async tax. Second, the actor pattern is unusually popular as a library discipline, and the reason is telling. Shared mutable state in async Rust means Arc<Mutex<T>> and the misery of guards held across await points; a mailbox turns the same problem into the one ownership story the borrow checker loves, state owned by exactly one task, messages transferring ownership through a channel. The frameworks also pay Rust’s async taxes once, behind a handler trait, so user code stops touching select! loops and cancellation safety: actix is the veteran (born before async/await stabilized, as structure for the combinator era), ractor transplants Erlang’s supervision idioms, kameo is a newer Tokio-native take. Which raises the question of what actors actually are in this taxonomy.

Actors are a discipline, not a third mechanism

An actor is private state, a mailbox, and the rule that messages are processed one at a time. Notice what’s absent from that definition: any statement about how a waiting actor is parked. Erlang parks it on a movable stack. Akka and Orleans park it with the rewrite. The Rust crates above park it as a Tokio state machine. Actors answer “who may touch this state”, while the mechanisms in this post answer “where does the suspended call live”; they compose rather than compete, which is why “should I use async or actors” is a category error you’ll nonetheless hear in design reviews.

I’ve written a full post on when the actor discipline earns its complexity and when a database row does the same job with less ceremony: The actor model, minus the cluster. The short version: an actor is justified when the consistency unit is a live, in-memory thing, a connection, a stream position, a digital twin, and it’s overhead everywhere else. Elarion, my open-source .NET application framework, builds its actor runtime on colored async/await for exactly the reasons this post lays out: on .NET, that is the mechanism the entire ecosystem already speaks.

The pitfalls that actually happen

Every mechanism has a signature failure. Four come up over and over in real systems.

Blocking on your own async code

The .NET classic. Code with a synchronization context (UI frameworks, legacy ASP.NET) calls .Result on a task whose continuation needs that same context: instant deadlock. ASP.NET Core removed the context, so the same line instead quietly consumes a thread-pool thread per call, and under load the pool starves and requests stall in the queue. The green-threads experiment is the same lesson mirrored: bridging colors is expensive in either direction, and a platform ultimately commits to one.

The pinned-carrier deadlock

JDK 21 through 23. A connection pool guards checkout with synchronized; requests on virtual threads hold a connection, block inside the monitor, and pin their carriers. With all carriers pinned, the virtual thread that would release a connection can never be scheduled. The service doesn’t crash; it stops, with idle CPUs. -Djdk.tracePinnedThreads=full names the culprit on JDK 21. JDK 24’s JEP 491 removes the synchronized case; JNI pinning stays. If you run virtual threads on the 21 LTS, audit the monitors your libraries hold across blocking calls.

Blocking inside the async runtime

The rewrite’s mirror image of pinning. A Tokio worker thread that hits a blocking call (a sync database driver, std::fs, a long zip operation) takes a scheduler lane out of service; a handful of them stall every future in the process. Tokio’s answer is spawn_blocking, .NET’s is to keep sync I/O off the thread pool, and Go’s answer is that the runtime notices and spins up another thread, which is why Go developers have never heard of this problem.

CPU-bound code on a cooperative scheduler

A virtual thread running a tight numeric loop holds its carrier; there is no blocking point at which to unmount it. Pre-1.14 Go had the same failure, down to garbage-collector stalls waiting on a spinning goroutine, and fixed it with forced preemption. On the JVM and on Tokio today, the fix is architectural: route sustained computation to a dedicated pool and keep the cooperative lanes for waiting. The schedulers in this post are optimized for tasks that mostly wait; feed them tasks that mostly compute and they degrade to something worse than plain threads.

Which answer do you actually want?

The rows share one shape: go with the grain of the ecosystem, not the elegance of the model. Java retrofitting movable stacks worked because its world was blocking-first; .NET keeping the color worked because its world was async-first; Rust refusing the runtime worked because its world is FFI-first. Every failed concurrency effort in this post, .NET’s green threads, Rust’s libgreen, is a sound mechanism transplanted into an ecosystem whose libraries spoke the other language.

The verdict, and the outlook

If you make me rank them today, Go is the most finished: one model since 2009, nothing in preview, no rulebook of footnotes. Java has the best outlook: virtual threads plus structured concurrency plus scoped values is the most modern design on the board, but one of the three is still in preview, so it’s an endpoint Java is approaching rather than standing on. It helps that the retrofit club has exactly one member; the JVM is the only runtime that controls its world deeply enough to pull this off against an existing ecosystem.

Genuinely on that level elsewhere: only the born-green platforms. GHC Haskell has run M:N green threads with blocking-style I/O for about two decades, and the BEAM remains the reference. Python’s gevent, Ruby’s fiber scheduler, and PHP’s Swoole chase the same shape but end up cooperative, single-threaded, or bolted on as an extension.

Two ecosystems are modern in a different direction. OCaml 5’s effect handlers give you direct-style concurrency without coloring and without stack copying, the most elegant answer on the board and the least battle-tested. Swift kept the color but shipped structured concurrency and actors as language constructs from day one: the rewrite, designed knowing everything we know now.

My rule of thumb: the wait always costs, so decide where you can afford to pay. If you want every suspension visible in the code, take the color and accept its bookkeeping: the allocations, the ValueTask rules, the ConfigureAwait boilerplate. If you want to write plain code and let the runtime carry the machinery, take the movable stacks and accept that the visibility question moves into your tooling. Both are honest deals, and where the ecosystem supports it, the second one is easier to live with day to day. The only dishonest deal is believing the cost disappeared. It moved, and knowing where it went is the whole game.