horseRL #34
horseRL #34
Conversation
Horse tweaks
Horse tweaks
sheim
left a comment
sheim
left a comment
There was a problem hiding this comment.
Overall looks good. I put in some comments on things that can be trimmed down from mini_cheetah_osc to horse_osc.
gym/envs/horse/horse_osc.py
Outdated
|
|
||
| self.osc_offset = stop * self.cfg.osc.offset | ||
| self.osc_omega = ( | ||
| stop * self.cfg.osc.omega_stop |
There was a problem hiding this comment.
here also, let's simplify and remove all the _var randomness
There was a problem hiding this comment.
removed torch.randn_like(self.osc_omega/coupling) * self.cfg.osc.omega/coupling_var
gym/envs/horse/horse_osc.py
Outdated
| self.oscillators_vel *= 2 * torch.pi | ||
| self.oscillators += ( | ||
| self.oscillators_vel * dt | ||
| ) # torch.clamp(self.oscillators_vel * dt, min=0) |
There was a problem hiding this comment.
zap the comment, and it'll go onto one line
There was a problem hiding this comment.
removed from both files
gym/envs/horse/horse_osc.py
Outdated
| prod = torch.prod(torch.clip(combined_rew, 0, 1), dim=1) | ||
| return prod - torch.ones_like(prod) | ||
|
|
||
| # this is removed in mini_cheetah_osc |
There was a problem hiding this comment.
what do you mean?
Oh it isn't removed, mini_cheetah_osc inherits from mini_cheetah, so whatever we don't overload is still there.
There was a problem hiding this comment.
got it! cleaned it up, just wanted to make note of differences I saw for easier comparison while working
gym/envs/horse/horse_osc.py
Outdated
| diff = torch.abs(theta1 - theta2) % (2 * torch.pi) | ||
| return torch.min(diff, 2 * torch.pi - diff) | ||
|
|
||
| def _reward_trot(self): |
There was a problem hiding this comment.
all these gait-specific rewards are rubbish that wasn't refactored, zap'em
There was a problem hiding this comment.
removed everything from _reward_trot to _reward_asymmetric
gym/envs/horse/horse_osc.py
Outdated
| similarity *= self._sqrdexp(angle - torch.pi, torch.pi) | ||
| return similarity | ||
|
|
||
| def _reward_asymettric(self): |
There was a problem hiding this comment.
removed everything from _reward_trot to _reward_asymmetric
Modified mini_cheetah for to add height commands.
Added horse model for training and integrated oscillators code. Currently unable to stabilize in a standing pose, still needs further config tuning.
Added helpful logging and plotting scripts. When you run
play.py, logs are saved tojoint_logs.npz. Then runscripts/analyze_joint_scaling.pyor 'scripts/plot_logs_by_joint.py`Added sweep config for automated hyperparameter sweeps for horse_osc
Modified files:
New files: