Consider codegen from viceroy wit(x) files

[avidal]

Tracking changes in the Compute ABI is currently a manual process. When this project started there was no documentation on the ABI, and I had to manually inspect the members of the fastly_sys crate, then write and link compatible function signatures in fastlike.

But now things are positioned a little bit better. With viceroy there's now a WIT description of the interface along with a plethora of wit-based tools.

There's a wit-bindgen-go program and library that can take a wasm file or a wit file and generate Go code, although that code is focused on guest programs written in Go (or TinyGo) and may not work directly, but it may be possible to use the wit package as a parser and write our own codegen based on the parsed IDL.

There are a few options for the generated code. One option would be a Go interface for every WIT interface described, and then fastlike implements those interfaces. Those interfaces could then be used to drive a second codegen pass that implements a lower level binding along with standardized logging calls. This is where we could automate lifting and lowering.

Of course, there may be a much simpler way to go about this, and there's the risk of the WIT not being comprehensive. For instance, the fastly_sys crate describes fastly_abi::init but that does not seem to be represented in the WIT description (granted, this may be by design as it's used for version negotiation between the guest program and the host).

[jedisct1]

WIT is used for WASI components. I might be mistaken, so please don’t take my word for it, but I don’t believe regular Compute applications use that. It seems more intended for internal purposes. What’s uploaded are WebAssembly modules, not WASI components.

Viceroy also includes WITX files, which I believe define the ABI actually used by Compute applications.

I maintain a copy of these WITX files that works with witx-codegen, but witx-codegen doesn’t currently have a Go backend.

We can use these files to track any new functions that get introduced. However, new functions aren’t added very often, and what we have right now seems simpler and more readable than auto-generated code.

[avidal]

WIT isn't WASI specific, but it does describe Wasm Components. If a wasm binary is a component, then it implements the Canonical ABI where standard lifting and lowering between core types and higher level types is defined.

Although I do recognize that I'm not entirely sure if Compute programs are actually Components or not. If they are, the main value in the generated code is generating the code that turns an (addr, len) pair into a string and injects the standard logging. If they aren't, why does Viceroy define the wit files?

But you're right that the ABI doesn't change frequently and working on codegen is unlikely to really pay off.

[avidal]

Ah, right: viceroy contains an adapter that adapts the compute abi to the wasm component model, allowing viceroy itself to use wit-bindgen.

I'm curious how it works internally, does viceroy load the adapter wasm alongside the guest program?