Wai bindgen rust is not working with the JS feature

[kajacx]

Scenario:

I want to use wai_bindgen_wasmer to create a Rust program that would import WASM plugins, and then run that program on a webpage. I unserstand that's what the js feature is for, but it is not working for me.

Just to re-itarate, i DO NOT want to import the WASM plugin in JS runtime.

The problem:

When I use the wai_bindgen_wasmer runtime and compile that to WASM using wasm-pack, I get this error at runtime: panicked at 'should create instance: Incompatible Export Type', src\lib.rs:41:14 (source)

fn get_color() -> protocol_plugin::Color {

    let mut store = Store::new(Engine::default());

    let module = Module::new(&store, PLUGIN_BYTES).expect("should create module");

    let (plugin, _) =
        protocol_plugin::ProtocolPlugin::instantiate(&mut store, &module, &mut imports! {})
            .expect("should create instance"); // <-- HERE

    plugin.get_color(&mut store).expect("should get color")
}

I tried cloning the wai repo and using that instead, and it actually throws a different error: panicked at 'not implemented: direct data pointer access is not possible in JavaScript', C:\Programs\Cargo\bin\registry\src\github.com-1ecc6299db9ec823\wasmer-3.1.1\src\js\externals\memory_view.rs:87:9

The code is the same as above.

Steps to reproduce:

Clone this repo at the wasm-pack-example-two-errors tag, and then clone https://github.com/wasmerio/wai.git into the repository. See the run-wasm-only.sh and run-wasm-only-forked.sh scripts on how to run the example.

[kajacx]

Ok, I found the line that throws the error: (source)

    pub unsafe fn data_unchecked_mut(&self) -> &mut [u8] {
        unimplemented!("direct data pointer access is not possible in JavaScript");
    }

Here is the generated method on the instance, using cargo expand --target=wasm32-unknown-unknown:

        pub fn get_color(
            &self,
            store: &mut wasmer::Store,
        ) -> Result<Color, wasmer::RuntimeError> {
            let _memory = &self.memory;
            let result0 = self.func_get_color.call(store)?;
            let _memory_view = _memory.view(&store);
            let load1 = unsafe { _memory_view.data_unchecked_mut() }
                .load::<f32>(result0 + 0)?;
            let _memory_view = _memory.view(&store);
            let load2 = unsafe { _memory_view.data_unchecked_mut() }
                .load::<f32>(result0 + 4)?;
            let _memory_view = _memory.view(&store);
            let load3 = unsafe { _memory_view.data_unchecked_mut() }
                .load::<f32>(result0 + 8)?;
            Ok(Color {
                r: load1,
                g: load2,
                b: load3,
            })
        }

I do not understand why it is not freeing after reading the value, but I guess that is not related to the problem that I am having.

In short, why even have the js feature in wai-bindgen-rust when it is not working?

[kajacx]

After some testing, I have verified that there is a way to get (and modify) the memory of another WASM module on the web with the MemoryView "api".

Proof of concept:

Here is how I pass data into the plugin:

fn export_to_plugin(memory: &Memory, store: &mut Store, instance: &Instance, data: &[u8]) -> u64 {
    let allocate = instance
        .exports
        .get_typed_function::<i32, i64>(&store, "allocate_for_host")
        .unwrap();
    let mut allocate = |size: u32| allocate.call(store, size as i32).unwrap() as u64;

    let fatptr = allocate(data.len() as u32);
    let (addr, _) = from_fatptr(fatptr);
    let view = memory.view(store);
    view.write(addr as u64, data).unwrap();
    fatptr
}

(Note that the plugin has to export a allocate_for_host function that allocated memory for the host to write into).

And here is how to import data from the plugin:

fn import_from_plugin(
    instace: &Instance,
    memory: &Memory,
    store: &mut Store,
    fatptr: u64,
) -> Vec<u8> {
    let (addr, len) = from_fatptr(fatptr);
    let mut bytes = vec![0; len];
    let view = memory.view(store);
    view.read(addr as u64, &mut bytes[0..len]).unwrap();

    let free = instace
        .exports
        .get_typed_function::<i64, ()>(store, "free_from_host")
        .unwrap();
    free.call(store, fatptr as i64).unwrap();

    bytes
}

(Again, plugin must implement free_from_host that frees the memory inside the plugin after the host reads it).

Full code:
In my example, a string gets passed back and forth a million times, and is checked against a second string that is modified directly in the host to make sure the plugin in modifying the string correctly. The code is an absolute mess, but it works. Full repo here.

To run it yourself, clone the repo at the manual-string-passing-with-memory-view tag, and then clone the wasmer repo into the same repository. This needs a bugfix that is wasmer/main, but in on crates.io yet. Then read the run-wasmer3-web.sh script for instruction on how to run it.

Moving forward:
The data_unchecked_mut() method works well on sys and is probably more effective than the MemoryView solution I am using, but it requires direct access to the plugin's memory, which is not available on the web.

The most logical move forward would be to change the generator macro based on the js/sys feature of wai-bindgen-wasmer, with sys feature it would generate the existing data_unchecked_mut() code, and with the js feature, it would generate this new MemoryView code. This would mean there is no penalty at runtime.