v1.0.0rc2

Highlights

• Add Grouped PBT Scheduling by @Daraan in #34
• Add EMA variant for Evaluation Return Metric by @Daraan in #29

Further changes

• Prepare for RLlib metrics V2 update by @Daraan in #28
• Add project/group experiment structure for storage_path by @Daraan in #32
• Always add experiment key, stateful PBT and fix trial restore by @Daraan in #33
• Deterministic Environment Seeding and Reset by @Daraan in #35
• Add DQN option by @Daraan in #30
• Patch model_config on restore with setup data by @Daraan in #37
• Improvement of JAX interface and testing its application, see https://github.com/Daraan/SYMPOL
• Other stability increases

Full Changelog: v0.8.0...v1.0.0rc2

v0.8.0 PBT Subparser and Fractional Perturbation

What's Changed

• V0.8.0-Subparser-introduction by @Daraan in #25

Introduces PopulationBasedTrainingParser as a subparser and allows perturbation_interval to be a relative value in 0..1 to perturb in fractions of the total_steps

• improve ExperimentUploader and failure handling
• colored output for Comet logs

Full Changelog: v0.7.0...v0.8.0

Version v0.7.0

Version 0.7.0

• V0.7.0 Restored envs have new seeds by @Daraan in #22

Highlights

• When runs are restarted, e.g. via PBT, the current_step is worked into the env_seed of the trials config. Before restarted trials might have always had the same seed and same first environment observation after a restart.

• Furthermore changed the seeding strategy without controlling the random number generators directly.

• Allow Sequence[int] for env seed as well

• Introduce string constants for seed metrics

• Warn if max_step_size does mot match with perturbation_interval

Further Improvements

• patch test helpers for ray 2.50.0.
• Do not to start WandbMonitor when disabled
• Do not initialize WandB Logging actor multiple times

Warning

ray 2.50.0 is not supported in combination with PBT or from_checkpoint. Metric restore does not work and the workaround provided by ray_utilities does not work on 2.50.0 anymore

Full Changelog: v0.6.0...v0.7.0

v0.6.0 - Streamline Comet and WandB uploads

What's Changed

• V0.6.0 Streamline WandB upload experience by @Daraan in #20

Main changes

• Comet and WandB experiments are uploaded early in "offline+upload" mode
• Better tracking of forks and failed uploads
• Increase success change of Wandb fork uploads.
  • Due to missing feature on the WandB Server side, forked runs can only be uploaded or initialized when the run-parent_id-history file exists on the WandB Server. But, this file is only generated when one actively opens the website to inspect the run.
    To workaround this a Selenium process can be started, for private projects login data can be provided.
  • Viewing the parent run before uploading / wandb.init(online=True, ...) has a very high success chance to create/upload seamless fork runs
• No error when comet .zip has already been uploaded (and moved)
• Improved cleanup, especially on experiment end / termination.
• Add helper methods and classes for experiment tracking an uploads

Other changes

• Support result["should_checkpoint"]=True for callbacks in ray 2.50.0+
• Refactoring:

# Refactor get_current_step
- from ray_utilities.training.helpers import get_current_step
+from ray_utilities.misc import get_current_step

• Colorful comet logger output
• ...

Full Changelog: v0.5.1...v0.6.0

v0.5.1 Sort Population based training correctly and handle NaN values

What's Changed

• v.0.5.1 Add missing top_pbt_scheduler patch by @Daraan in #18, that handles NaN values and correct sorting for max mode.

Full Changelog: v0.5.0...v0.5.1

Version 0.5.0

Upgrade to v0.5.0: Major Features and Improvements

Experiment Management & Configuration

• Refactor experiment key and run_id handling to support forking and lineage tracking
• Store and load config files with experiments; allow config overwrite on load (need to be synced to remote)
• Support shell tab completion for CLI arguments
• Allow new tags via --tag:tag_name and --tag:tag_name=value (with overwrite support)
• Enable new log format for improved readability, log less interesting keys for "more" and "minimal" log stats

Population Based Training & Scheduling

• Add TopPBTTrialScheduler with quantile-based exploitation and grid/cyclic mutation
• Handle NaN results for PBT

Logging, Callbacks, and Offline Storage

• Unify WandB/Comet exclude & non-metrics logic
• Comet: store offline runs in subdirs, create only when needed, and support forked trials
• WandB: sync storage if local_path unavailable, wait for sync after offline trial, and support fork_from feature
• Handle forked trials in CSV, JSON, and (minimally) TBX logger callbacks with remote/local syncing
• Refactor: extract common fork logic into FileLoggerForkMixin
• Improve warning for WandB errors and handle upload failures

Tune & Training Reliability

• Add retries for Tune failures and handle retry errors as strings
• Remove Stopper variant of OptunaSearch when using Scheduler
• Improve total_steps and iterations calculation, also with --use_exact_total_steps option

Testing, Typing, and Miscellaneous

• Typing: clean types of video dicts, unify type hierarchy for metrics and algorithm return types
• Remove setup.py and PEP 728 conditional guards from TypedDict classes
• Minor refactoring, bugfixes, and cleaning

Documentation

Improve Sphinx indexing and update README for v0.5.0

Full Changelog: v0.4.0...v0.5.0

v.0.4.0

Release Notes

Summary

New defaults and parsers for MLP setups, better checkpoint/restore behavior, improved logging & experiment uploads, safer Tune exception handling (via ExceptionGroups), and a number of test and stability fixes. Small tooling and ergonomics improvements (ray_status, --cfg/--lr, customizable trial names) are included.

Highlights

• Default experiment setup switched to MLP (MLPArgumentParser).
• CLI: add --cfg (file) and --lr, support for parsers as well as config_files and load_args for setup classes
• default learning rate lowered to 1e-4 and limit max batch size to 8192
• customizable trial names
• Log model architecture + config; upload to Comet/WandB.
• Improved offline upload and robust Tune exception handling
• Checkpoint/restore fixes: allow late Algorithm creation on restore, implement on_checkpoint_loaded for dynamic callbacks, mark num_env_runners as NeverRestore and add mismatch checks.
• Tool: ray_status.sh to check the status of all ray instances
• Prep work for upcoming learner API changes (training → learners).

Version 0.3.1

What's Changed

• Fix: Gradient Accumulating not working correctly by @Daraan in #10

Full Changelog: v0.3.0...v0.3.1

Version 0.3.0

Version 0.3.0

What's Changed

• add MLPSetup
• improve support for multiple Envs per Runner
• add documentation hosted on https://ray-utilities.readthedocs.io/
• Improve (offline) logging of experiments to Comet and Wandb
• ...

Full Changelog: v0.2.2...v0.3.0