- 
-
-
Meet HopeJR – A humanoid robot arm and hand for dexterous manipulation!
@@ -51,20 +47,12 @@
- 
-  |
+  |
+
Meet the updated SO100, the SO-101 – Just €114 per arm!
Train it in minutes with a few simple moves on your laptop.
@@ -76,7 +64,7 @@Want to take it to the next level? Make your SO-101 mobile by building LeKiwi!
Check out the LeKiwi tutorial and bring your robot to life on wheels.
-
+ 
 |
-  |
-  |
+  |
+  |
+  |
| ACT policy on ALOHA env | @@ -110,24 +98,9 @@
Note: For efficiency, during training every checkpoint is evaluated on a low number of episodes. You may use `--eval.n_episodes=500` to evaluate on more episodes than the default. Or, after training, you may want to re-evaluate your best checkpoints on more episodes or change the evaluation settings. See `python -m lerobot.scripts.eval --help` for more instructions.
@@ -305,26 +278,6 @@ reproduces SOTA results for Diffusion Policy on the PushT task.
If you would like to contribute to 🤗 LeRobot, please check out our [contribution guide](https://github.com/huggingface/lerobot/blob/main/CONTRIBUTING.md).
-
-
### Add a pretrained policy
Once you have trained a policy you may upload it to the Hugging Face hub using a hub id that looks like `${hf_user}/${repo_name}` (e.g. [lerobot/diffusion_pusht](https://huggingface.co/lerobot/diffusion_pusht)).
@@ -341,34 +294,16 @@ To upload these to the hub, run the following:
huggingface-cli upload ${hf_user}/${repo_name} path/to/pretrained_model
```
-See [eval.py](https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/eval.py) for an example of how other people may use your policy.
-
-### Improve your code with profiling
-
-An example of a code snippet to profile the evaluation of a policy:
+See [eval.py](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/eval.py) for an example of how other people may use your policy.
-
-```python
-from torch.profiler import profile, record_function, ProfilerActivity
-
-def trace_handler(prof):
- prof.export_chrome_trace(f"tmp/trace_schedule_{prof.step_num}.json")
+### Acknowledgment
-with profile(
- activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA],
- schedule=torch.profiler.schedule(
- wait=2,
- warmup=2,
- active=3,
- ),
- on_trace_ready=trace_handler
-) as prof:
- with record_function("eval_policy"):
- for i in range(num_episodes):
- prof.step()
- # insert code to profile, potentially whole body of eval_policy function
-```
-
+- The LeRobot team 🤗 for building SmolVLA [Paper](https://arxiv.org/abs/2506.01844), [Blog](https://huggingface.co/blog/smolvla).
+- Thanks to Tony Zhao, Zipeng Fu and colleagues for open sourcing ACT policy, ALOHA environments and datasets. Ours are adapted from [ALOHA](https://tonyzhaozh.github.io/aloha) and [Mobile ALOHA](https://mobile-aloha.github.io).
+- Thanks to Cheng Chi, Zhenjia Xu and colleagues for open sourcing Diffusion policy, Pusht environment and datasets, as well as UMI datasets. Ours are adapted from [Diffusion Policy](https://diffusion-policy.cs.columbia.edu) and [UMI Gripper](https://umi-gripper.github.io).
+- Thanks to Nicklas Hansen, Yunhai Feng and colleagues for open sourcing TDMPC policy, Simxarm environments and datasets. Ours are adapted from [TDMPC](https://github.com/nicklashansen/tdmpc) and [FOWM](https://www.yunhaifeng.com/FOWM).
+- Thanks to Antonio Loquercio and Ashish Kumar for their early support.
+- Thanks to [Seungjae (Jay) Lee](https://sjlee.cc/), [Mahi Shafiullah](https://mahis.life/) and colleagues for open sourcing [VQ-BeT](https://sjlee.cc/vq-bet/) policy and helping us adapt the codebase to our repository. The policy is adapted from [VQ-BeT repo](https://github.com/jayLEE0301/vq_bet_official).
## Citation
@@ -376,83 +311,13 @@ If you want, you can cite this work with:
```bibtex
@misc{cadene2024lerobot,
- author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascale, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
+ author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
howpublished = "\url{https://github.com/huggingface/lerobot}",
year = {2024}
}
```
-Additionally, if you are using any of the particular policy architecture, pretrained models, or datasets, it is recommended to cite the original authors of the work as they appear below:
-
-- [SmolVLA](https://arxiv.org/abs/2506.01844)
-
-```bibtex
-@article{shukor2025smolvla,
- title={SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics},
- author={Shukor, Mustafa and Aubakirova, Dana and Capuano, Francesco and Kooijmans, Pepijn and Palma, Steven and Zouitine, Adil and Aractingi, Michel and Pascal, Caroline and Russi, Martino and Marafioti, Andres and Alibert, Simon and Cord, Matthieu and Wolf, Thomas and Cadene, Remi},
- journal={arXiv preprint arXiv:2506.01844},
- year={2025}
-}
-```
-
-- [Diffusion Policy](https://diffusion-policy.cs.columbia.edu)
-
-```bibtex
-@article{chi2024diffusionpolicy,
- author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song},
- title ={Diffusion Policy: Visuomotor Policy Learning via Action Diffusion},
- journal = {The International Journal of Robotics Research},
- year = {2024},
-}
-```
-
-- [ACT or ALOHA](https://tonyzhaozh.github.io/aloha)
-
-```bibtex
-@article{zhao2023learning,
- title={Learning fine-grained bimanual manipulation with low-cost hardware},
- author={Zhao, Tony Z and Kumar, Vikash and Levine, Sergey and Finn, Chelsea},
- journal={arXiv preprint arXiv:2304.13705},
- year={2023}
-}
-```
-
-- [TDMPC](https://www.nicklashansen.com/td-mpc/)
-
-```bibtex
-@inproceedings{Hansen2022tdmpc,
- title={Temporal Difference Learning for Model Predictive Control},
- author={Nicklas Hansen and Xiaolong Wang and Hao Su},
- booktitle={ICML},
- year={2022}
-}
-```
-
-- [VQ-BeT](https://sjlee.cc/vq-bet/)
-
-```bibtex
-@article{lee2024behavior,
- title={Behavior generation with latent actions},
- author={Lee, Seungjae and Wang, Yibin and Etukuru, Haritheja and Kim, H Jin and Shafiullah, Nur Muhammad Mahi and Pinto, Lerrel},
- journal={arXiv preprint arXiv:2403.03181},
- year={2024}
-}
-```
-
-- [HIL-SERL](https://hil-serl.github.io/)
-
-```bibtex
-@Article{luo2024hilserl,
-title={Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning},
-author={Jianlan Luo and Charles Xu and Jeffrey Wu and Sergey Levine},
-year={2024},
-eprint={2410.21845},
-archivePrefix={arXiv},
-primaryClass={cs.RO}
-}
-```
-
## Star History
[](https://star-history.com/#huggingface/lerobot&Timeline)
diff --git a/docs-requirements.txt b/docs-requirements.txt
new file mode 100644
index 0000000000..e286ad2bb4
--- /dev/null
+++ b/docs-requirements.txt
@@ -0,0 +1,3 @@
+# docs-requirements.txt
+hf-doc-builder @ git+https://github.com/huggingface/doc-builder.git@main
+watchdog>=6.0.0
diff --git a/docs/README.md b/docs/README.md
index 967de7b841..476eb8dce8 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -20,7 +20,7 @@ To generate the documentation, you first have to build it. Several packages are
you can install them with the following command, at the root of the code repository:
```bash
-pip install -e ".[docs]"
+pip install -e . -r docs-requirements.txt
```
You will also need `nodejs`. Please refer to their [installation page](https://nodejs.org/en/download)
diff --git a/docs/source/hilserl.mdx b/docs/source/hilserl.mdx
index f66d8cab7d..2f73d09647 100644
--- a/docs/source/hilserl.mdx
+++ b/docs/source/hilserl.mdx
@@ -28,7 +28,7 @@ This guide provides step-by-step instructions for training a robot policy using
- A gamepad (recommended) or keyboard to control the robot
- A Nvidia GPU
- A real robot with a follower and leader arm (optional if you use the keyboard or the gamepad)
-- A URDF file for the robot for the kinematics package (check `lerobot/common/model/kinematics.py`)
+- A URDF file for the robot for the kinematics package (check `lerobot/model/kinematics.py`)
## What kind of tasks can I train?
@@ -477,7 +477,7 @@ Create a training configuration file (example available [here](https://huggingfa
1. Configure the policy settings (`type="sac"`, `device`, etc.)
2. Set `dataset` to your cropped dataset
3. Configure environment settings with crop parameters
-4. Check the other parameters related to SAC in [configuration_sac.py](https://github.com/huggingface/lerobot/blob/19bb621a7d0a31c20cd3cc08b1dbab68d3031454/lerobot/common/policies/sac/configuration_sac.py#L79).
+4. Check the other parameters related to SAC in [configuration_sac.py](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/sac/configuration_sac.py#L79).
5. Verify that the `policy` config is correct with the right `input_features` and `output_features` for your task.
**Starting the Learner**
diff --git a/docs/source/il_robots.mdx b/docs/source/il_robots.mdx
index ccca6d5082..ec5491b2a9 100644
--- a/docs/source/il_robots.mdx
+++ b/docs/source/il_robots.mdx
@@ -294,7 +294,7 @@ dataset.push_to_hub()
#### Dataset upload
-Locally, your dataset is stored in this folder: `~/.cache/huggingface/lerobot/{repo-id}`. At the end of data recording, your dataset will be uploaded on your Hugging Face page (e.g. https://huggingface.co/datasets/cadene/so101_test) that you can obtain by running:
+Locally, your dataset is stored in this folder: `~/.cache/huggingface/lerobot/{repo-id}`. At the end of data recording, your dataset will be uploaded on your Hugging Face page (e.g. `https://huggingface.co/datasets/${HF_USER}/so101_test`) that you can obtain by running:
```bash
echo https://huggingface.co/datasets/${HF_USER}/so101_test
@@ -323,7 +323,7 @@ The `record` function provides a suite of tools for capturing and managing data
##### 2. Checkpointing and Resuming
- Checkpoints are automatically created during recording.
-- If an issue occurs, you can resume by re-running the same command with `--resume=true`.
+- If an issue occurs, you can resume by re-running the same command with `--resume=true`. When resuming a recording, `--dataset.num_episodes` must be set to the **number of additional episodes to be recorded**, and not to the targeted total number of episodes in the dataset !
- To start recording from scratch, **manually delete** the dataset directory.
##### 3. Recording Parameters
@@ -428,7 +428,7 @@ Your robot should replicate movements similar to those you recorded. For example
## Train a policy
-To train a policy to control your robot, use the [`python -m lerobot.scripts.train`](../src/lerobot/scripts/train.py) script. A few arguments are required. Here is an example command:
+To train a policy to control your robot, use the [`python -m lerobot.scripts.train`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/train.py) script. A few arguments are required. Here is an example command:
```bash
python -m lerobot.scripts.train \
@@ -444,7 +444,7 @@ python -m lerobot.scripts.train \
Let's explain the command:
1. We provided the dataset as argument with `--dataset.repo_id=${HF_USER}/so101_test`.
-2. We provided the policy with `policy.type=act`. This loads configurations from [`configuration_act.py`](../src/lerobot/policies/act/configuration_act.py). Importantly, this policy will automatically adapt to the number of motor states, motor actions and cameras of your robot (e.g. `laptop` and `phone`) which have been saved in your dataset.
+2. We provided the policy with `policy.type=act`. This loads configurations from [`configuration_act.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/act/configuration_act.py). Importantly, this policy will automatically adapt to the number of motor states, motor actions and cameras of your robot (e.g. `laptop` and `phone`) which have been saved in your dataset.
3. We provided `policy.device=cuda` since we are training on a Nvidia GPU, but you could use `policy.device=mps` to train on Apple silicon.
4. We provided `wandb.enable=true` to use [Weights and Biases](https://docs.wandb.ai/quickstart) for visualizing training plots. This is optional but if you use it, make sure you are logged in by running `wandb login`.
@@ -462,9 +462,9 @@ If you do not want to push your model to the hub after training use `--policy.pu
Additionally you can provide extra `tags` or specify a `license` for your model or make the model repo `private` by adding this: `--policy.private=true --policy.tags=\[ppo,rl\] --policy.license=mit`
-#### Train using Collab
+#### Train using Google Colab
-If your local computer doesn't have a powerful GPU you could utilize Google Collab to train your model by following the [ACT training notebook](./notebooks#training-act).
+If your local computer doesn't have a powerful GPU you could utilize Google Colab to train your model by following the [ACT training notebook](./notebooks#training-act).
#### Upload policy checkpoints
diff --git a/docs/source/il_sim.mdx b/docs/source/il_sim.mdx
index 193b09b1bf..761e24e0fd 100644
--- a/docs/source/il_sim.mdx
+++ b/docs/source/il_sim.mdx
@@ -96,7 +96,7 @@ If you uploaded your dataset to the hub you can [visualize your dataset online](
## Train a policy
-To train a policy to control your robot, use the [`python -m lerobot.scripts.train`](../src/lerobot/scripts/train.py) script. A few arguments are required. Here is an example command:
+To train a policy to control your robot, use the [`python -m lerobot.scripts.train`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/train.py) script. A few arguments are required. Here is an example command:
```bash
python -m lerobot.scripts.train \
@@ -111,7 +111,7 @@ python -m lerobot.scripts.train \
Let's explain the command:
1. We provided the dataset as argument with `--dataset.repo_id=${HF_USER}/il_gym`.
-2. We provided the policy with `policy.type=act`. This loads configurations from [`configuration_act.py`](../src/lerobot/policies/act/configuration_act.py). Importantly, this policy will automatically adapt to the number of motor states, motor actions and cameras of your robot (e.g. `laptop` and `phone`) which have been saved in your dataset.
+2. We provided the policy with `policy.type=act`. This loads configurations from [`configuration_act.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/act/configuration_act.py). Importantly, this policy will automatically adapt to the number of motor states, motor actions and cameras of your robot (e.g. `laptop` and `phone`) which have been saved in your dataset.
3. We provided `policy.device=cuda` since we are training on a Nvidia GPU, but you could use `policy.device=mps` to train on Apple silicon.
4. We provided `wandb.enable=true` to use [Weights and Biases](https://docs.wandb.ai/quickstart) for visualizing training plots. This is optional but if you use it, make sure you are logged in by running `wandb login`.
diff --git a/docs/source/lekiwi.mdx b/docs/source/lekiwi.mdx
index bb70fd26b7..a5bdb19cf0 100644
--- a/docs/source/lekiwi.mdx
+++ b/docs/source/lekiwi.mdx
@@ -258,7 +258,7 @@ You should see on your laptop something like this: `[INFO] Connected to remote r
| F | Decrease speed |
> [!TIP]
-> If you use a different keyboard, you can change the keys for each command in the [`LeKiwiConfig`](../src/lerobot/robot_devices/robots/configs.py).
+> If you use a different keyboard, you can change the keys for each command in the [`LeKiwiClientConfig`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/robots/lekiwi/config_lekiwi.py).
### Wired version
diff --git a/docs/source/policy_act_README.md b/docs/source/policy_act_README.md
new file mode 100644
index 0000000000..371a9136fe
--- /dev/null
+++ b/docs/source/policy_act_README.md
@@ -0,0 +1,14 @@
+## Paper
+
+https://tonyzhaozh.github.io/aloha
+
+## Citation
+
+```bibtex
+@article{zhao2023learning,
+ title={Learning fine-grained bimanual manipulation with low-cost hardware},
+ author={Zhao, Tony Z and Kumar, Vikash and Levine, Sergey and Finn, Chelsea},
+ journal={arXiv preprint arXiv:2304.13705},
+ year={2023}
+}
+```
diff --git a/docs/source/policy_diffusion_README.md b/docs/source/policy_diffusion_README.md
new file mode 100644
index 0000000000..9ec934add6
--- /dev/null
+++ b/docs/source/policy_diffusion_README.md
@@ -0,0 +1,14 @@
+## Paper
+
+https://diffusion-policy.cs.columbia.edu
+
+## Citation
+
+```bibtex
+@article{chi2024diffusionpolicy,
+ author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song},
+ title ={Diffusion Policy: Visuomotor Policy Learning via Action Diffusion},
+ journal = {The International Journal of Robotics Research},
+ year = {2024},
+}
+```
diff --git a/docs/source/policy_smolvla_README.md b/docs/source/policy_smolvla_README.md
new file mode 100644
index 0000000000..ee567ee836
--- /dev/null
+++ b/docs/source/policy_smolvla_README.md
@@ -0,0 +1,14 @@
+## Paper
+
+https://arxiv.org/abs/2506.01844
+
+## Citation
+
+```bibtex
+@article{shukor2025smolvla,
+ title={SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics},
+ author={Shukor, Mustafa and Aubakirova, Dana and Capuano, Francesco and Kooijmans, Pepijn and Palma, Steven and Zouitine, Adil and Aractingi, Michel and Pascal, Caroline and Russi, Martino and Marafioti, Andres and Alibert, Simon and Cord, Matthieu and Wolf, Thomas and Cadene, Remi},
+ journal={arXiv preprint arXiv:2506.01844},
+ year={2025}
+}
+```
diff --git a/docs/source/policy_tdmpc_README.md b/docs/source/policy_tdmpc_README.md
new file mode 100644
index 0000000000..804f166c8f
--- /dev/null
+++ b/docs/source/policy_tdmpc_README.md
@@ -0,0 +1,14 @@
+## Paper
+
+https://www.nicklashansen.com/td-mpc/
+
+## Citation
+
+```bibtex
+@inproceedings{Hansen2022tdmpc,
+ title={Temporal Difference Learning for Model Predictive Control},
+ author={Nicklas Hansen and Xiaolong Wang and Hao Su},
+ booktitle={ICML},
+ year={2022}
+}
+```
diff --git a/docs/source/policy_vqbet_README.md b/docs/source/policy_vqbet_README.md
new file mode 100644
index 0000000000..02f95b7c2f
--- /dev/null
+++ b/docs/source/policy_vqbet_README.md
@@ -0,0 +1,14 @@
+## Paper
+
+https://sjlee.cc/vq-bet/
+
+## Citation
+
+```bibtex
+@article{lee2024behavior,
+ title={Behavior generation with latent actions},
+ author={Lee, Seungjae and Wang, Yibin and Etukuru, Haritheja and Kim, H Jin and Shafiullah, Nur Muhammad Mahi and Pinto, Lerrel},
+ journal={arXiv preprint arXiv:2403.03181},
+ year={2024}
+}
+```
diff --git a/examples/hil_serl_simulation_training/hi_rl_test_gamepad.json b/examples/hil_serl_simulation_training/hi_rl_test_gamepad.json
new file mode 100644
index 0000000000..4b040a9df2
--- /dev/null
+++ b/examples/hil_serl_simulation_training/hi_rl_test_gamepad.json
@@ -0,0 +1,92 @@
+{
+ "type": "hil",
+ "wrapper": {
+ "gripper_penalty": -0.02,
+ "display_cameras": false,
+ "add_joint_velocity_to_observation": true,
+ "add_ee_pose_to_observation": true,
+ "crop_params_dict": {
+ "observation.images.front": [
+ 0,
+ 0,
+ 128,
+ 128
+ ],
+ "observation.images.wrist": [
+ 0,
+ 0,
+ 128,
+ 128
+ ]
+ },
+ "resize_size": [
+ 128,
+ 128
+ ],
+ "control_time_s": 40.0,
+ "use_gripper": true,
+ "fixed_reset_joint_positions": [
+ 0.0,
+ 0.195,
+ 0.0,
+ -2.43,
+ 0.0,
+ 2.62,
+ 0.785
+ ],
+ "reset_time_s": 2.0,
+ "control_mode": "gamepad"
+ },
+ "use_viewer": true,
+ "use_gamepad": true,
+ "name": "franka_sim",
+ "mode": "record",
+ "repo_id": "username/panda-pick-gamepad",
+ "dataset_root": "./recordings_static_gamepad",
+ "task": "PandaPickCubeGamepad-v0",
+ "num_episodes": 30,
+ "episode": 0,
+ "pretrained_policy_name_or_path": null,
+ "device": "cuda",
+ "push_to_hub": true,
+ "fps": 10,
+ "random_block_position": true,
+ "features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ },
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 4
+ ]
+ }
+ },
+ "features_map": {
+ "observation.images.front": "observation.images.front",
+ "observation.images.wrist": "observation.images.wrist",
+ "observation.state": "observation.state",
+ "action": "action"
+ },
+ "reward_classifier_pretrained_path": null,
+ "number_of_steps_after_success": 0
+}
diff --git a/examples/hil_serl_simulation_training/hil_serl_simulation_training_guide_README.md b/examples/hil_serl_simulation_training/hil_serl_simulation_training_guide_README.md
new file mode 100644
index 0000000000..1543b7b51e
--- /dev/null
+++ b/examples/hil_serl_simulation_training/hil_serl_simulation_training_guide_README.md
@@ -0,0 +1,98 @@
+# LeRobot Training Guide: Panda Cube Picking
+
+A concise guide for training a robot policy to pick cubes using LeRobot's HIL-SERL framework.
+
+## Installation
+
+Install LeRobot with HIL-SERL support as shown in the [official documentation](https://huggingface.co/docs/lerobot/hilserl):
+
+```bash
+pip install -e ".[hilserl]"
+```
+
+## Recording Demonstrations
+
+Proceed with recording sample episodes for pretraining the reinforcement learning algorithm:
+
+```bash
+python -m lerobot.scripts.rl.gym_manipulator --config_path examples/hil_serl_simulation_training/hi_rl_test_gamepad.json
+```
+
+**Important Notes:**
+
+- **Recommended**: Collect 30-40 episodes including some failed picking attempts
+- **Control**: Use a gamepad controller for better end-effector control (configuration already set up)
+- **Dataset Storage**:
+ - `dataset_root` points to local folder where dataset is saved
+ - `repo_id` points to HuggingFace account for dataset upload
+ - **Prerequisite**: Install and authorize HuggingFace CLI with token to push datasets and replace the username in the `hi_rl_test_gamepad.json` and `train_gym_hil_env_gamepad.json`
+
+## Training the Policy
+
+Start both the learner and actor using these commands:
+
+```bash
+# Terminal 1: Start learner
+python -m lerobot.scripts.rl.learner --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json
+
+# Terminal 2: Start actor
+python -m lerobot.scripts.rl.actor --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json
+```
+
+The learner initializes with parameters from the offline training dataset.
+
+## Human Intervention Strategy
+
+**Important**: Allow the policy a few episodes to test picking on its own before intervening.
+
+**Best practices** for human intervention (see [official guide](https://huggingface.co/docs/lerobot/hilserl#guide-to-human-interventions)):
+
+1. **Guide the end effector back to the object** if it's drifting away from the cube
+2. **Guide the end effector above the cube** and let the policy try to pick it and raise it
+3. **Completely pick the object** if needed
+
+**Progression Strategy:**
+
+- As training progresses, interventions should become less aggressive
+- Once the policy manages to pick the cube on its own, start intervening only to guide the end effector to the approximate area where the object is found
+
+## Training Results
+
+The fully trained policy result can be seen in this video:
+
+
+
+## Resuming Training from Checkpoint
+
+Checkpoints are automatically saved based on the `save_freq` parameter in `train_gym_hil_env_gamepad.json` and are stored in the `outputs` folder.
+
+To resume training, both actor and learner need to be called with the policy path:
+
+```bash
+# Terminal 1: Resume learner
+python -m lerobot.scripts.rl.learner --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json --policy.path=examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model
+
+# Terminal 2: Resume actor
+python -m lerobot.scripts.rl.actor --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json --policy.path=examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model
+```
+
+Where `policy.path` is the full path to the locally saved checkpoints.
+
+## Monitoring Training
+
+Set the Weights & Biases flag to `true` in `train_gym_hil_env_gamepad.json` to follow live training results.
+
+The trained intervention rate should approximately look like this after a while:
+
+
+
+## Configuration Files
+
+- `hi_rl_test_gamepad.json`: Recording configuration
+- `train_gym_hil_env_gamepad.json`: Training configuration
+
+## Key Parameters
+
+- **Save Frequency**: Controlled by `save_freq` in training config
+- **Checkpoint Location**: `outputs/train/YYYY-MM-DD/HH-MM-SS_jobname/checkpoints/`
+- **Policy Path**: Points to `pretrained_model/` directory within checkpoints
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/config.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/config.json
new file mode 100644
index 0000000000..b1806c8fd9
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/config.json
@@ -0,0 +1,185 @@
+{
+ "type": "sac",
+ "n_obs_steps": 1,
+ "normalization_mapping": {
+ "VISUAL": "MEAN_STD",
+ "STATE": "MIN_MAX",
+ "ENV": "MIN_MAX",
+ "ACTION": "MIN_MAX"
+ },
+ "input_features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ }
+ },
+ "output_features": {
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 3
+ ]
+ }
+ },
+ "device": "cuda",
+ "use_amp": false,
+ "push_to_hub": true,
+ "repo_id": "spirosperos/panda-pick-gamepad-policy",
+ "private": null,
+ "tags": null,
+ "license": null,
+ "dataset_stats": {
+ "observation.images.front": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.images.wrist": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.state": {
+ "min": [
+ -0.6897139549255371,
+ -1.1421763896942139,
+ -0.5745007991790771,
+ -2.97829008102417,
+ -0.2710767090320587,
+ 1.3246592283248901,
+ -0.04057434946298599,
+ -0.21261805295944214,
+ -0.4548068344593048,
+ -0.6540042757987976,
+ -0.3644964098930359,
+ -1.1057522296905518,
+ -0.40768879652023315,
+ -0.2220114767551422,
+ 0.0,
+ 0.19176171720027924,
+ -0.3013063669204712,
+ 0.00362197193317115
+ ],
+ "max": [
+ 0.5107022523880005,
+ 0.5516204237937927,
+ 0.5620884299278259,
+ -1.3330878019332886,
+ 0.32758936285972595,
+ 3.119610548019409,
+ 1.8364211320877075,
+ 0.25358933210372925,
+ 0.36316126585006714,
+ 0.14765967428684235,
+ 0.49947625398635864,
+ 0.144814133644104,
+ 0.2820609211921692,
+ 0.7382049560546875,
+ 255.0,
+ 0.6012658476829529,
+ 0.3005995750427246,
+ 0.5004003643989563
+ ]
+ }
+ },
+ "storage_device": "cpu",
+ "vision_encoder_name": "helper2424/resnet10",
+ "freeze_vision_encoder": true,
+ "image_encoder_hidden_dim": 32,
+ "shared_encoder": true,
+ "num_discrete_actions": 3,
+ "image_embedding_pooling_dim": 8,
+ "online_steps": 1000000,
+ "online_env_seed": 10000,
+ "online_buffer_capacity": 100000,
+ "offline_buffer_capacity": 100000,
+ "async_prefetch": false,
+ "online_step_before_learning": 100,
+ "policy_update_freq": 1,
+ "discount": 0.97,
+ "temperature_init": 0.01,
+ "num_critics": 2,
+ "num_subsample_critics": null,
+ "critic_lr": 0.0003,
+ "actor_lr": 0.0003,
+ "temperature_lr": 0.0003,
+ "critic_target_update_weight": 0.005,
+ "utd_ratio": 2,
+ "state_encoder_hidden_dim": 256,
+ "latent_dim": 64,
+ "target_entropy": null,
+ "use_backup_entropy": true,
+ "grad_clip_norm": 10.0,
+ "critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true
+ },
+ "policy_kwargs": {
+ "use_tanh_squash": true,
+ "std_min": 1e-05,
+ "std_max": 5.0,
+ "init_final": 0.05
+ },
+ "discrete_critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_learner_config": {
+ "learner_host": "127.0.0.1",
+ "learner_port": 50051,
+ "policy_parameters_push_frequency": 1,
+ "queue_get_timeout": 2.0
+ },
+ "concurrency": {
+ "actor": "threads",
+ "learner": "threads"
+ },
+ "use_torch_compile": true
+}
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/model.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/model.safetensors
new file mode 100644
index 0000000000..93e1dd0c7f
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a122fcb6fd8ec8d483352e47d007f445505dfebe360724c7835d23700970841
+size 25603220
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/train_config.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/train_config.json
new file mode 100644
index 0000000000..8463c4e8c5
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/train_config.json
@@ -0,0 +1,402 @@
+{
+ "dataset": {
+ "repo_id": "spirosperos/panda-pick-gamepad",
+ "root": null,
+ "episodes": null,
+ "image_transforms": {
+ "enable": false,
+ "max_num_transforms": 3,
+ "random_order": false,
+ "tfs": {
+ "brightness": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "brightness": [
+ 0.8,
+ 1.2
+ ]
+ }
+ },
+ "contrast": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "contrast": [
+ 0.8,
+ 1.2
+ ]
+ }
+ },
+ "saturation": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "saturation": [
+ 0.5,
+ 1.5
+ ]
+ }
+ },
+ "hue": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "hue": [
+ -0.05,
+ 0.05
+ ]
+ }
+ },
+ "sharpness": {
+ "weight": 1.0,
+ "type": "SharpnessJitter",
+ "kwargs": {
+ "sharpness": [
+ 0.5,
+ 1.5
+ ]
+ }
+ }
+ }
+ },
+ "revision": null,
+ "use_imagenet_stats": false,
+ "video_backend": "pyav"
+ },
+ "env": {
+ "type": "hil",
+ "task": "PandaPickCubeGamepad-v0",
+ "fps": 10,
+ "features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ },
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 3
+ ]
+ }
+ },
+ "features_map": {
+ "observation.images.front": "observation.images.front",
+ "observation.images.wrist": "observation.images.wrist",
+ "observation.state": "observation.state",
+ "action": "action"
+ },
+ "name": "franka_sim",
+ "use_viewer": true,
+ "gripper_penalty": 0.0,
+ "use_gamepad": true,
+ "state_dim": 18,
+ "action_dim": 4,
+ "episode_length": 100,
+ "random_block_position": true,
+ "video_record": {
+ "enabled": false,
+ "record_dir": "videos",
+ "trajectory_name": "trajectory"
+ },
+ "reward_classifier_pretrained_path": null,
+ "robot_config": null,
+ "teleop_config": null,
+ "wrapper": {
+ "control_mode": "gamepad",
+ "display_cameras": false,
+ "add_joint_velocity_to_observation": true,
+ "add_current_to_observation": false,
+ "add_ee_pose_to_observation": true,
+ "crop_params_dict": {
+ "observation.images.front": [
+ 0,
+ 0,
+ 128,
+ 128
+ ],
+ "observation.images.wrist": [
+ 0,
+ 0,
+ 128,
+ 128
+ ]
+ },
+ "resize_size": [
+ 128,
+ 128
+ ],
+ "control_time_s": 30.0,
+ "fixed_reset_joint_positions": [
+ 0.0,
+ 0.195,
+ 0.0,
+ -2.43,
+ 0.0,
+ 2.62,
+ 0.785
+ ],
+ "reset_time_s": 2.0,
+ "use_gripper": true,
+ "gripper_quantization_threshold": 0.8,
+ "gripper_penalty": -0.02,
+ "gripper_penalty_in_reward": false
+ },
+ "mode": null,
+ "repo_id": "spirosperos/panda-pick-gamepad",
+ "dataset_root": null,
+ "num_episodes": 30,
+ "episode": 0,
+ "device": "cuda",
+ "push_to_hub": true,
+ "pretrained_policy_name_or_path": null,
+ "number_of_steps_after_success": 0
+ },
+ "policy": {
+ "type": "sac",
+ "n_obs_steps": 1,
+ "normalization_mapping": {
+ "VISUAL": "MEAN_STD",
+ "STATE": "MIN_MAX",
+ "ENV": "MIN_MAX",
+ "ACTION": "MIN_MAX"
+ },
+ "input_features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ }
+ },
+ "output_features": {
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 3
+ ]
+ }
+ },
+ "device": "cuda",
+ "use_amp": false,
+ "push_to_hub": true,
+ "repo_id": "spirosperos/panda-pick-gamepad-policy",
+ "private": null,
+ "tags": null,
+ "license": null,
+ "dataset_stats": {
+ "observation.images.front": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.images.wrist": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.state": {
+ "min": [
+ -0.6897139549255371,
+ -1.1421763896942139,
+ -0.5745007991790771,
+ -2.97829008102417,
+ -0.2710767090320587,
+ 1.3246592283248901,
+ -0.04057434946298599,
+ -0.21261805295944214,
+ -0.4548068344593048,
+ -0.6540042757987976,
+ -0.3644964098930359,
+ -1.1057522296905518,
+ -0.40768879652023315,
+ -0.2220114767551422,
+ 0.0,
+ 0.19176171720027924,
+ -0.3013063669204712,
+ 0.00362197193317115
+ ],
+ "max": [
+ 0.5107022523880005,
+ 0.5516204237937927,
+ 0.5620884299278259,
+ -1.3330878019332886,
+ 0.32758936285972595,
+ 3.119610548019409,
+ 1.8364211320877075,
+ 0.25358933210372925,
+ 0.36316126585006714,
+ 0.14765967428684235,
+ 0.49947625398635864,
+ 0.144814133644104,
+ 0.2820609211921692,
+ 0.7382049560546875,
+ 255.0,
+ 0.6012658476829529,
+ 0.3005995750427246,
+ 0.5004003643989563
+ ]
+ }
+ },
+ "storage_device": "cpu",
+ "vision_encoder_name": "helper2424/resnet10",
+ "freeze_vision_encoder": true,
+ "image_encoder_hidden_dim": 32,
+ "shared_encoder": true,
+ "num_discrete_actions": 3,
+ "image_embedding_pooling_dim": 8,
+ "online_steps": 1000000,
+ "online_env_seed": 10000,
+ "online_buffer_capacity": 100000,
+ "offline_buffer_capacity": 100000,
+ "async_prefetch": false,
+ "online_step_before_learning": 100,
+ "policy_update_freq": 1,
+ "discount": 0.97,
+ "temperature_init": 0.01,
+ "num_critics": 2,
+ "num_subsample_critics": null,
+ "critic_lr": 0.0003,
+ "actor_lr": 0.0003,
+ "temperature_lr": 0.0003,
+ "critic_target_update_weight": 0.005,
+ "utd_ratio": 2,
+ "state_encoder_hidden_dim": 256,
+ "latent_dim": 64,
+ "target_entropy": null,
+ "use_backup_entropy": true,
+ "grad_clip_norm": 10.0,
+ "critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true
+ },
+ "policy_kwargs": {
+ "use_tanh_squash": true,
+ "std_min": 1e-05,
+ "std_max": 5.0,
+ "init_final": 0.05
+ },
+ "discrete_critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_learner_config": {
+ "learner_host": "127.0.0.1",
+ "learner_port": 50051,
+ "policy_parameters_push_frequency": 1,
+ "queue_get_timeout": 2.0
+ },
+ "concurrency": {
+ "actor": "threads",
+ "learner": "threads"
+ },
+ "use_torch_compile": true
+ },
+ "output_dir": "outputs/train/2025-08-05/13-45-02_default",
+ "job_name": "default",
+ "resume": false,
+ "seed": 1000,
+ "num_workers": 4,
+ "batch_size": 256,
+ "steps": 100000,
+ "eval_freq": 20000,
+ "log_freq": 1,
+ "save_checkpoint": true,
+ "save_freq": 2000,
+ "use_policy_training_preset": true,
+ "optimizer": {
+ "type": "multi_adam",
+ "lr": 0.001,
+ "weight_decay": 0.0,
+ "grad_clip_norm": 10.0,
+ "optimizer_groups": {
+ "actor": {
+ "lr": 0.0003
+ },
+ "critic": {
+ "lr": 0.0003
+ },
+ "temperature": {
+ "lr": 0.0003
+ }
+ }
+ },
+ "scheduler": null,
+ "eval": {
+ "n_episodes": 50,
+ "batch_size": 50,
+ "use_async_envs": false
+ },
+ "wandb": {
+ "enable": true,
+ "disable_artifact": true,
+ "project": "franka_sim",
+ "entity": null,
+ "notes": null,
+ "run_id": "vcej2ove",
+ "mode": null
+ }
+}
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_param_groups.json
new file mode 100644
index 0000000000..1862ca0432
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_param_groups.json
@@ -0,0 +1,32 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0,
+ 1,
+ 2,
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_state.safetensors
new file mode 100644
index 0000000000..6da5c8f157
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2a6161778634fb671d504d2a1ddaab6debe545620c3773c48ddd06400f29e35
+size 945016
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_param_groups.json
new file mode 100644
index 0000000000..fa6e015086
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_param_groups.json
@@ -0,0 +1,90 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0,
+ 1,
+ 2,
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11,
+ 12,
+ 13,
+ 14,
+ 15,
+ 16,
+ 17,
+ 18,
+ 19,
+ 20,
+ 21,
+ 22,
+ 23,
+ 24,
+ 25,
+ 26,
+ 27,
+ 28,
+ 29,
+ 30,
+ 31,
+ 32,
+ 33,
+ 34,
+ 35,
+ 36,
+ 37,
+ 38,
+ 39,
+ 40,
+ 41,
+ 42,
+ 43,
+ 44,
+ 45,
+ 46,
+ 47,
+ 48,
+ 49,
+ 50,
+ 51,
+ 52,
+ 53,
+ 54,
+ 55,
+ 56,
+ 57,
+ 58,
+ 59,
+ 60,
+ 61,
+ 62,
+ 63,
+ 64,
+ 65,
+ 66,
+ 67,
+ 68,
+ 69
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_state.safetensors
new file mode 100644
index 0000000000..39cadf5ca6
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdefe9b85637cd0c1d897fea73bf42e8f5d186cd52e22ab1bd48fa53f4657494
+size 7141144
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_param_groups.json
new file mode 100644
index 0000000000..8403db78c1
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_param_groups.json
@@ -0,0 +1,80 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0,
+ 1,
+ 2,
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11,
+ 12,
+ 13,
+ 14,
+ 15,
+ 16,
+ 17,
+ 18,
+ 19,
+ 20,
+ 21,
+ 22,
+ 23,
+ 24,
+ 25,
+ 26,
+ 27,
+ 28,
+ 29,
+ 30,
+ 31,
+ 32,
+ 33,
+ 34,
+ 35,
+ 36,
+ 37,
+ 38,
+ 39,
+ 40,
+ 41,
+ 42,
+ 43,
+ 44,
+ 45,
+ 46,
+ 47,
+ 48,
+ 49,
+ 50,
+ 51,
+ 52,
+ 53,
+ 54,
+ 55,
+ 56,
+ 57,
+ 58,
+ 59
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_state.safetensors
new file mode 100644
index 0000000000..4df296bd39
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1cf9c8277e7f2656cd7be71ea3f13d12a8c7373fe68dce54e6bb1e61d3ae4043
+size 6198632
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/rng_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/rng_state.safetensors
new file mode 100644
index 0000000000..7f769dc138
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/rng_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed8d082a4c8ba4930ef223fc726e932ae8e7835a56bc95b97b787fde40b5a0ea
+size 15708
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_param_groups.json
new file mode 100644
index 0000000000..b93fdca550
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_param_groups.json
@@ -0,0 +1,21 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_state.safetensors
new file mode 100644
index 0000000000..b46bc7a046
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d007eeb289842c38b424534d18ab905a11ec701421192cced69284545f4d94fe
+size 228
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_state.pt b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_state.pt
new file mode 100644
index 0000000000..37d5b0e963
Binary files /dev/null and b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_state.pt differ
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_step.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_step.json
new file mode 100644
index 0000000000..39090bbb98
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_step.json
@@ -0,0 +1,3 @@
+{
+ "step": 2000
+}
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/last b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/last
new file mode 120000
index 0000000000..ec0e1b9d54
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/last
@@ -0,0 +1 @@
+0002000
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log
new file mode 100644
index 0000000000..2ecb985a98
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log
@@ -0,0 +1,362385 @@
+INFO 2025-08-05 13:45:02 /learner.py:154 Learner logging initialized, writing to outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log
+INFO 2025-08-05 13:45:02 /learner.py:155 {'batch_size': 256,
+ 'dataset': {'episodes': None,
+ 'image_transforms': {'enable': False,
+ 'max_num_transforms': 3,
+ 'random_order': False,
+ 'tfs': {'brightness': {'kwargs': {'brightness': [0.8,
+ 1.2]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'contrast': {'kwargs': {'contrast': [0.8,
+ 1.2]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'hue': {'kwargs': {'hue': [-0.05,
+ 0.05]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'saturation': {'kwargs': {'saturation': [0.5,
+ 1.5]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'sharpness': {'kwargs': {'sharpness': [0.5,
+ 1.5]},
+ 'type': 'SharpnessJitter',
+ 'weight': 1.0}}},
+ 'repo_id': 'spirosperos/panda-pick-gamepad',
+ 'revision': None,
+ 'root': None,
+ 'use_imagenet_stats': False,
+ 'video_backend': 'pyav'},
+ 'env': {'action_dim': 4,
+ 'dataset_root': None,
+ 'device': 'cuda',
+ 'episode': 0,
+ 'episode_length': 100,
+ 'features': {'action': {'shape': [3],
+ 'type':