A comprehensive guide for working with the NVIDIA Cosmos ecosystem—a suite of World Foundation Models (WFMs) for real-world, domain-specific applications across robotics, simulation, autonomous systems, and physical scene understanding.
📚 View the Full Documentation → — Step-by-step workflows, case studies, and technical recipes
carousel.mp4
| Date | Recipe | Model | Description |
|---|---|---|---|
| Jan 6 | Dataset Video Clustering with Time Series K-Means | Cosmos Curator | Advanced video clustering using Time Series K-Means on embedding vector trajectories for robotics behavior analysis |
| Jan 5 | Post-train Cosmos Reason 2 for AV Video Captioning and VQA | Cosmos Reason 2 | Domain adaptation for autonomous vehicle video captioning with multi-benchmark evaluation |
| Jan 1 | Egocentric Social Reasoning for Robotics | Cosmos Reason 2 | Egocentric social and physical reasoning evaluation for social robotics |
| Jan 1 | Reason 2 on Brev | Cosmos Reason 2 | Getting started guide for Cosmos Reason 2 inference and post-training on Brev |
| Dec 22 | Multiview AV Generation with World Scenario Maps | Cosmos Transfer 2.5 | ControlNet post-training for spatially-conditioned multiview AV video generation |
| Dec 20 | Vision AI Gallery | Cosmos Transfer 2.5 | Interactive gallery showcasing weather, lighting, and object augmentations for traffic scenarios |
| Use Case | Linux (Ubuntu) | macOS | Windows |
|---|---|---|---|
| Running cookbook recipes (GPU workflows) | ✅ Supported | ❌ | ❌ |
| Local documentation & contribution | ✅ Supported | ✅ Supported |
- Git with Git LFS
- Python: Version 3.10+
- Internet access for cloning and dependencies
Full GPU workflows require an Ubuntu Linux environment with NVIDIA GPUs.
→ See Getting Started for complete hardware and software requirements.
⚠️ Important: This repository contains many media files (videos, images, demonstrations). Git LFS is required to clone and work with this repository properly.
# Ubuntu/Debian (recommended)
sudo apt update && sudo apt install git-lfs
# Enable Git LFS globally
git lfs installFor other platforms (macOS, Windows, Fedora), see the official installation guide at git-lfs.com.
If you've already cloned without LFS, fetch the media files with:
git lfs pull# Install uv (fast Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
# Install just (command runner)
uv tool install -U rust-justFor other platforms, see astral.sh/uv for installation instructions.
# Clone the repository
git clone https://github.com/nvidia-cosmos/cosmos-cookbook.git
cd cosmos-cookbook
# Install dependencies and setup
just install# Serve documentation locally
just serve-external # For public documentation
# or
just serve-internal # For internal documentation (if applicable)Then open http://localhost:8000 in your browser.
The Cosmos Cookbook is organized into two main directories:
Contains the source documentation in markdown files:
- Technical guides and workflows
- End-to-end examples and case studies
- Step-by-step recipes and tutorials
- Getting started guides
Contains executable scripts referenced throughout the cookbook:
- Data processing and curation pipelines
- Model evaluation and quality control scripts
- Configuration files for post-training tasks
- Automation tools and utilities
This structure separates documentation from implementation, making it easy to navigate between reading about workflows and executing the corresponding scripts.
When contributing media files, prefer .mp4 over .gif:
- Better quality — MP4 supports full color depth vs GIF's 256-color limit
- Smaller file size — Modern video codecs compress far more efficiently
- Audio support — MP4 can include narration when needed
Use H.264 encoding for universal browser compatibility.
# Development
just install # Install dependencies and setup
just setup # Setup pre-commit hooks
just serve-external # Serve public documentation locally
just serve-internal # Serve internal documentation locally
# Quality Control
just lint # Run linting and formatting
just test # Run all tests and validation
# Continuous Integration
just ci-lint # Run CI linting checks
just ci-deploy-internal # Deploy internal documentation
just ci-deploy-external # Deploy external documentation- Contributing Guide - How to contribute to the cookbook
- Report Issues: Use GitHub Issues for bugs and feature requests
- Share Success Stories: We love hearing how you use Cosmos models creatively
This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.
NVIDIA Cosmos source code is released under the Apache 2 License.
NVIDIA Cosmos models are released under the NVIDIA Open Model License. For a custom license, please contact cosmos-license@nvidia.com.