Conventional semantics for acquiring a stream from a resource combined with drop

[alexcrichton]

Many upcoming WASIp3 APIs are based on stream and future. Many APIs also work with a resource such as an HTTP body. This gives rise to a situation where a guest can receive an HTTP request, acquire the body as a stream<u8>, and then drop the original request resource itself. The question then faced is what happens? This basic scenario is applicable to a broad number of other situations such as receiving a stream of bytes from a TCP socket or generalizing to passing a stream<u8> to a "transformer" component which performs compression/encryption/etc.

In WASIp2 this question was resolved with the WASI-specific concept of a "child resource" where if the parent was dropped before the child a trap would be generated. A question for WASIp3 is whether we want to perpetuate this. I had some discussion today with @vados-cosmonic, @dicej, and @rvolosatovs and we discussed four possible ways to answer the question of what to do in this situation:

1. Do what WASIp2 did and trap. This would likely require some form of officially component model integration as streams are managed by Wasmtime where body resources, for example, are managed by WASI implementations. The downside of this approach is that this is already not necessarily easy to integrate with WASIp2 today. Put another way if a trap is generated then guests have to be precise about the exact order handles are dropped in. This is not always easy for GC languages for example. Another example this is not easy to do is in a composition use case where a stream is passed to an entirely separate component. In such a situation it's not always clear when the receiver is done with the stream so the caller may not ever know when it's safe to close the "parent" resource.

2. Reference-count all resources. This is different from WASIp2 where closing a parent before the child wouldn't do anything, the child would still be valid. Whatever the child needed to keep alive to stay operational would be kept alive. Embedders would be required to do some form of reference counting to ensure that resources/streams stay working even when the "parent" goes away. This would be easier for GC languages and composition use cases in that drop order doesn't matter, and everyone's just responsible for dropping their own handle. A downside of this approach is that it's more difficult to ensure that host state is indeed deallocated when a resource is dropped. Host state is still kept alive by any child handles, and a guest may not know where those are or how to close them.

3. "Close children on parent drop" where when a parent resource is closed it doesn't generate a trap but it does forcibly close all child resources. For example a stream would be terminated, a future would resolve to a "default value", child resources would "start returning errors", etc. Basically the trap is no longer present but semantically the meaning/state of all child handles is different after the parent resource has dropped. This has the benefit of guaranteeing that when a resource is dropped that any host state associated with it is possible to drop. A downside of this approach though is similar to (1) where while drops can happen in any order it's still ambiguous to know when a drop of a parent resource is possible. For example if a component wants to hand off a body stream to another component it still has to wait to drop the original request resource until the body is done being processed. This may not always be as simple as waiting on an async func for example.

4. Dropping a parent resource blocks until all children are dropped. This is sort of like "async drop" but it semantically means that when a resource is dropped that it'll wait for all children to go away and/or finish their I/O. This has the benefit of a resource can be dropped at any time. This has a downside, though, that by blocking a drop of a parent it could block a component for an indefinitely long time while I/O completes.

WASIp3 doesn't today take a firm stance on which of these possible routes is chosen for various APIs. The goal here is to make a high-level decision about how best to manage this situation and determine if it's best solved in the Component Model, WASI, or runtimes. Personally I'd like to advocate for option (2) above. While not without its downsides I feel that it has the fewest downsides and what remains is possible to solve with new WASI APIs. For example an API could be added to take an own<tcp-socket> which forcibly closes it, peforming the semantics of option (3) but in a TCP-specific context. Effectively the benefits of (2) would remain (anyone can close whenever) also with the benefits of (3) of being able to close something and know the host-side resource has been disposed of.

[pchickey]

If I got to redesign the way child resources work in P2 today, I wouldn't implement a trap if a parent resource is dropped while any children were still alive, but instead I'd have the parent drop swap out any children for a gravestone, and trap if there is any use of the gravestone besides dropping it. This would help guests because bindings generation doesn't have any mechanical encoding of the child/parent relationships (just human readable in the comments) and so its hard to make bindings that automatically enforce drops in the "right" order. Permitting drops in any order is safe and sensible.

Is there a way to modify approach 3 to trap if the child streams or futures have any operations on them besides removing them from the waitable set and dropping them?

[alexcrichton]

Assuming it were possible to implement that, personally I'd still advocate for avoiding that option (option 3 above) and sticking to option (2). Assuming the expected prevalence of composition I'm worried that parent handles would have to unnecessarily be kept open for arbitrary periods of time just to keep children running. One example would be a connection-pool style component where when you hand it a stream it may pool the stream or not, but you're generally not sure so you wouldn't know when to close the parent resource. This is a pretty nebulous example, but I'm basically worried about components where the closing the parent resource semantically shouldn't happen til all children are done and the component may not be able to deduce when the children are done if they're handed off to other components.

[pchickey]

I think I need more detail to work through the design of your connection pool example to see exactly how that works out.

In wasi-http, we used the guarantee that all child resources were dropped before the parent could be dropped as part of the API design, in particular to make illegal states unrepresentable - e.g. https://github.com/WebAssembly/wasi-http/blob/main/wit/types.wit#L514-L523 where the implementation needs to be able to determine whether its appropriate to put backpressure on the delivery of the body or not. When we designed this, we were particularly concerned that reference counting would be the incorrect choice, because the whatwg fetch (js) interface is effectively stuck unable to implement trailers because it has no way to disambiguate between the users intent to put backpressure on body contents vs throw away body to consume trailers.

[lukewagner]

Agreed that, in the absence of an explicit child annotation in WIT that bindings generators can see and use, Option 1 has serious usability problems for GC languages.

Previously, I had thought that Option 3 was the right way to go since it emulates host-side what a bindings generator would do guest-side if we had child. But that's a really good point that if I hand off a child stream, we still get this surprising behavior (that, in a GC setting, will manifest or not based on GC timing). And really, this can even happen in intra-component settings too where client code unaware of the child/parent relationship lets the parent reference become unreachable and it mostly works (b/c GC timing) until it doesn't.

Option 4 seems like it would introduce non-obvious deadlocks, so I suppose, by process of elimination, that Option 2 is the way to go. It's not ideal and has some subtle component-composition hazards, but it seems like the best we can do without child in WIT.

[[For future reference: this helps clarify for me that there are two rather distinct things we need bindings generators doing for child in a GC language setting: (1) auto-tombstoning children when the parent is explicitly dropped (e.g., via dispose) or transferred, (2) otherwise keeping the parent reachable as long as any child is reachable.]]

[alexcrichton]

@pchickey I'm new to this problem space for wasi-http, but for trailers-vs-body could that be served/satisfied because the body and trailers are separate handles? Independent of reference-counting I'd naively expect that expressing disinterest in the body would be done by closing the resource (possibly prematurely) which would signal intent that trailers are still interesting (the future is still there) but the body can be entirely thrown away. This naively seems disconnected to resource lifetime issues of whether or not the stream<u8> body is alive before/after the request or response object.

I do in general agree though that my connection pooling example is quite amibiguous so I don't want to over-rotate on it for sure.

---

I also don't want to give the impression that the above list of options are necessarily exhaustive. It's what we could think of, but there might be yet other alternatives too