CubeMLX

A C++/MLX port of two open-source Roblox generative AI technologies (text-to-mesh, text-to-texture), with a simple editor.

Editor features

• Generation of 3D meshes with a text prompt.
• Texturing of those meshes with a text prompt.
• Addition of textured meshes in a 3D scene.
• Live-preview in a simple game, with hot-reload (changes in the scene affect the game live).
• gRPC client-server architecture, allowing for headless API usage or implementation of other clients.
• Custom integration of Metal to create 3D viewports in Avalonia

Subprojects

Project	Description	Upstream
cubemlx	MLX port of Cube3D - text-to-mesh generation	Roblox/cube
flashtexmlx	MLX port of FlashTex - prompt-based mesh texturing	Roblox/FlashTex
stablediffusionmlx	MLX Stable Diffusion implementation	ml-explore/mlx-examples
nvdiffrastmetal	Metal port of nvdiffrast - differentiable rasterization	NVlabs/nvdiffrast
hashgridmlx	MLX port of Instant-NGP hash grid encoding	NVlabs/instant-ngp
metalbindings	.NET bindings for Apple Metal framework	-
editor	GUI for textured 3D mesh generation and scene construction	-
game	Metal-based game renderer with gRPC communication	-

Critical differences with reference implementations

• Only the subset of features from Stable diffusion, Hashgrid and Nvdiffrast required to run mesh / texture inference end-to-end were ported.
• This repo is dedicated to inference, not training.
• FlashTexMLX uses only DepthControlNet for inference, not LightControlNet.
• FlashTexMLX uses Nvdiffrast across the board for all rendering needs.
• CubeMLX uses dual contouring instead of marching cubes for meshing.

Architecture

┌─────────────────────────────────────────────────────────┐
│                         editor                          │
│                  (Avalonia desktop app)                 │
└────────┬──────────────────┬──────────────────┬──────────┘
         │                  │                  │
         ▼                  ▼                  ▼
   ┌───────────┐     ┌─────────────┐    ┌──────────────┐
   │  cubemlx  │     │ flashtexmlx │    │metalbindings │
   │(text→mesh)│     │ (mesh→tex)  │    │(.NET Metal)  │
   └───────────┘     └──────┬──────┘    └──────────────┘
                            │
          ┌─────────────────┼─────────────────┐
          │                 │                 │
          ▼                 ▼                 ▼
┌──────────────────┐ ┌─────────────┐ ┌───────────────┐
│stablediffusionmlx│ │nvdiffrast-  │ │  hashgridmlx  │
│ (MLX diffusion)  │ │   metal     │ │(hash encoding)│
└──────────────────┘ └─────────────┘ └───────────────┘

Usage

Prerequisites

• macOS with Apple Silicon (M1/M2/M3/M4)
• 32GB RAM minimum
• CMake 3.20+
• Ninja build system
• .NET SDK 10.0+
• Python 3 with pip
• Hugging Face CLI (hf) - installation guide

Build

./bootstrap.sh

This will:

1. Check prerequisites
2. Download model weights from Hugging Face
3. Convert PyTorch weights to SafeTensors
4. Build all C++ components (MLX workspace + game + metalbindings)
5. Copy native libraries to .NET binding directories
6. Build all .NET components

Run

# Launch the game renderer
./run.sh game

# Launch the editor application
./run.sh editor

# Generate a mesh from text
./run.sh cubemlx generate "a wooden chair" output.obj

# Texture an existing mesh
./run.sh flashtexmlx generate --mesh input.obj --prompt "wooden texture"

# Run all tests
./run.sh test

Screenshots

License

This repository contains components with restrictive licenses:

Component	License	Restriction
cubemlx	RAIL-MS	Research/academic only
nvdiffrastmetal	NVIDIA Source Code	Non-commercial only
hashgridmlx	NVIDIA Source Code	Non-commercial only

All other components are Apache 2.0.

See LICENSES.md for full details and third-party attributions.