Skip to content

Added SmolVLA and LeRobot integration with configs, validation, and documentation.#407

Merged
baochunli merged 16 commits intomainfrom
smolvla
Mar 10, 2026
Merged

Added SmolVLA and LeRobot integration with configs, validation, and documentation.#407
baochunli merged 16 commits intomainfrom
smolvla

Conversation

@baochunli
Copy link
Collaborator

@baochunli baochunli commented Mar 10, 2026

This PR added end-to-end SmolVLA support backed by LeRobot, including optional dependency gating, a LeRobot datasource and trainer path, SmolVLA model loading, runnable experiment configurations, and documentation for setup and operations. It also hardened FedAvg state exchange so adapter-style fine-tuning, mixed dtypes, and transport-safe checkpoint payloads continue to work within Plato's existing aggregation flow.

Changes

  • Added a extra and guarded import paths so SmolVLA and LeRobot support can be enabled without changing the default Plato installation path.

  • Added a LeRobot datasource with deterministic episode-aware partitioning, dataset validation, canonical Plato sample mapping, and configurable image-transform and temporal-window handling.

  • Added a SmolVLA model wrapper that resolves pretrained policies from config, supports full and adapter fine-tuning modes, and validates checkpoint compatibility for federated training.

  • Added a LeRobot trainer backend that integrates policy preprocessing, device and precision controls, custom collate behavior, and loss normalization for robotics workloads.

  • Added runnable LeRobot experiment profiles for single-client smoke runs, two-client FedAvg smoke runs, and longer full fine-tuning workflows.

  • Hardened FedAvg payload extraction and loading to support adapter-only exchanges, payload-size safeguards, bf16 transport casting, and dtype-safe restoration for non-floating tensors.

  • Added unit and smoke integration coverage for configuration loading, datasource wiring, model and trainer behavior, and the end-to-end SmolVLA LeRobot runtime path.

  • Added installation guidance, configuration parameter documentation, and an operator runbook for setting up and running SmolVLA training with LeRobot datasets.

Validation

  • Ran ============================= test session starts ==============================
    platform darwin -- Python 3.13.12, pytest-8.4.2, pluggy-1.6.0
    rootdir: /Users/bli/Playground/plato
    configfile: pyproject.toml
    plugins: hydra-core-1.3.2
    collected 24 items

tests/algorithms/test_fedavg_algorithm.py ..... [ 20%]
tests/datasources/test_lerobot_datasource.py ... [ 33%]
tests/models/test_smolvla_model.py .. [ 41%]
tests/trainers/test_lerobot_trainer.py ... [ 54%]
tests/test_config_loader.py .......... [ 95%]
tests/integration/test_lerobot_smolvla_smoke.py . [100%]

============================== 24 passed in 0.10s ==============================.

  • Observed .

@netlify
Copy link

netlify bot commented Mar 10, 2026

Deploy Preview for platodocs ready!

Name Link
🔨 Latest commit deaa0c4
🔍 Latest deploy log https://app.netlify.com/projects/platodocs/deploys/69b06fa2b69e340007b0fc56
😎 Deploy Preview https://deploy-preview-407--platodocs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@baochunli baochunli merged commit da88695 into main Mar 10, 2026
8 checks passed
@baochunli baochunli deleted the smolvla branch March 10, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant