[v0.3] `handle` implementations cannot forward trailers without trapping while handling transmit errors

[rvolosatovs]

Components exporting wasi:http@0.3.0-draft/handle.handle cannot correctly forward incoming requests while tracking the body transmit error.

Assume an application, which forwards the incoming request and returns a successful (200 OK) response with empty body iff forwarding of the incoming request in it's entirety has succeeded to the best of it's knowledge (we can never know for sure due to OS-level buffering).

Pseudo-code (in pseudo-Rust):

async fn handle(req: Request) -> Result<Response, ErrorCode> {
    let headers = req.headers().clone();
    headers.set("X-Forwarded-For", origin_of(headers));

    let req_body = req.body()?;
    let (req_stream, req_result) = req_body.stream()?;
    let (trailers_tx, trailers_rx) = future::new();
    // We don't know if we will receive trailers at this point, so we are forced to use this API
    let (out_body, out_result) = body::new_with_trailers(req_stream, trailers_rx);

    // NOTE: this is incorrect, trailers might not be present and we'll only know about it once we have received the body
    // We should not send empty trailers if none were given, but closing the future without a result is not valid according to the spec

    let res = http::handle(Request::new(headers, Some(out_body), None)).await?;

    let trailers = body::finish(req_body).await;
    // TRAP!
}

• The application will only find out whether there are trailers to forward once the full incoming body has been read, however the body constructor forces us to make a choice before that has happened (since we want to stream the body being forwarded)
• The only way for the application to get a handle to the trailers is by calling finish on the incoming body, however we cannot do that, since there is an active stream handle, that we must pass to the body constructor.
  • Even if [v0.3] model bodies after stream<u8, trailers, error-code>, trap less #162 was to be merged, this would still be broken - we'd need to only call finish once the body has been fully streamed, but we don't have a way of knowing when that has happened.

It seems that we need to change the trailers future signature to return option<trailers> and have some way to acquire trailers from the body without moving it

Refs #164

[rvolosatovs]

The solution we discussed with @dicej and @lukewagner is creating a new stream and splicing data.

For the first iteration, this will require explicit copying bytes in the guest, however eventually we'll add splice operation on streams allowing to avoid the copy.

The example would then look roughly like:

fn handle(req: Request) -> Result<Response, ErrorCode> {
    let headers = req.headers();
    headers.set("X-Forwarded-For", origin_of(headers));

    let Some(req_body) = req.body()?;
    let (req_stream, req_result) = req_body.stream()?;
    let (trailers_tx, trailers_rx) = wit_future::new();
    let (splice_tx, splice_rx) = wit_stream::new();
    let (out_body, out_result) = Body::new(splice_rx, trailers_rx);

    let res = http::handle(Request::new(headers, out_body, None)).await?;
    req_stream.splice(splice_tx).await?;
    drop(req_stream);
    trailers_tx.send(Body::finish(req_body).await);
    let (body_tx, body_rx) = wit_stream::new();
    let (res_tx, res_rx) = wit_future::new();
    let (body, _tx) = Body::new(body_rx, res_rx);
    res_tx.send(Ok(None));
    Ok(Response::new(Headers::new(), body))
}