MDanceIO

mdanceio is a cross-platform MMD(MikuMikuDance) compatible implementation. It targets at browser though WebGPU on wasm. Rewrite nanoem in Rust.

This project is still in initial development phase.

Motivation

I built this project mainly to learn Rust and WebGPU. I hope to provide a MikuMikuDance implementation in the browser, as well as via cloud rendering and on AR/VR in the future.

mdanceio works as a crate that provides MMD rendering service on a specific Surface, TextureView or Canvas. Or it can directly return a BytesArray.

There is another project build on mdanceio which provides basic MMD remote rendering service via WebRTC.

Getting Started

How can I get supported Models and Motions

You can fetch models and motions from 模之屋(PlayBox), a community sharing character models.

The project build is likely buggy and unfinished. You can try the following model and motion which is tested and welling working to get started.

• Model: 【原神】砂糖Sucrose
  • ◆模型提供：miHoYo
  • ◆模型改造：观海
• Motion: 神里凌华传说任务舞蹈
  • 动作：Lct火红枣

The demo GIFs in this README use the above model and motion.

Example

The example will play a model with specific motion in a native window.

cargo run --package mdanceio --example winit_app -- --model <Model Path> --motion <Motion Path>

You can build as an executable as well.

cargo build --package mdanceio --example winit_app --release

You can also fetch the executable in Actions.

WebGPU Demo

You can visit the demo here(requires Chrome Canary with #enable-unsafe-webgpu enabled).

WebGL Demo

You can visit the demo using WebGL.

Prerequisite

Install wasm-pack, a rust -> wasm workflow tool.

You need nodejs to serve the demo.

You also need Google Chrome Canary that supports WebGPU, and enable the feature flag #enable-unsafe-webgpu.

Build wasm package

wasm-pack build mdanceio --out-dir ../target/pkg

Build requires environment variable: RUSTFLAGS=--cfg=web_sys_unstable_apis

Run Web Demo

cd mdance-demo
npm install
npm run start

You can also fetch prebuilt web bundle in Actions.

Remote Rendering

I have a demo project about how to use mdanceio as a rendering service here.

The service uses WebRTC to communicate with the browser.

You can follow its guidance to play with it.

Target Platform Support

Platform	Support
Windows	✅
Linux	🆗
MacOS	🆗
Browser(WebGPU)	✅
Browser(WebGL)	🆗
Android	🛠️
OpenXR	🛠️

✅ = First Class Support 🆗 = Best Effort Support 🛠️ = Unsupported, but planned

File Format Support

Model Format

Format	Support
PMX	✅
PMD	❌

Motion Format

Format	Support
VMD	✅
NMD	❌

Future Plan

• The core functionality has not yet completed. We will cover all MikuMikuDance features in the future.
• I'm interested in supporting mdanceio in an AR/VR environment. We will extract SDK for AR usage and provide a demo.
• Provide an architecture that supports cloud rendering will.
• Provide support for MME or similar technologies.