Skip to content

Add periodic WeightWatcher plot generation and optional S3 upload in OpenML 1049 notebook#32

Open
charlesmartin14 wants to merge 1 commit intomainfrom
codex/update-weightwatcher-output-to-s3
Open

Add periodic WeightWatcher plot generation and optional S3 upload in OpenML 1049 notebook#32
charlesmartin14 wants to merge 1 commit intomainfrom
codex/update-weightwatcher-output-to-s3

Conversation

@charlesmartin14
Copy link
Member

Motivation

  • Enable periodic export of WeightWatcher diagnostic figures during training so plots can be inspected later and archived.
  • Provide an optional, configurable S3 sink so saved figures can be uploaded off-host for remote review.

Description

  • Add a plotting cadence configuration WW_PLOT_EVERY_ROUNDS and an optional S3 destination prefix read from the environment via WW_S3_PLOT_PREFIX in XGBWW_OpenML_1049_W7W8_Alpha_Checkpoint.ipynb.
  • Extend ww_stats_for_matrix(...) to accept plot=True and savedir and pass savefig=True/savedir into WeightWatcher.analyze(...) when plotting is requested.
  • Create a local plots root at ww_plots/ and save per-round directories round_XXXXX/{W2,W7,W8} so each round’s figures live in their own folder.
  • Add sync_round_plots_to_s3(...) that runs aws s3 cp --recursive to upload a round folder to the configured S3 prefix, and print status messages after save/upload.
  • Update the main training loop to compute plot_this_round = (r % WW_PLOT_EVERY_ROUNDS == 0), create per-matrix subfolders when plotting, call the updated ww_stats_for_matrix with plotting enabled on those rounds, and trigger the S3 sync when configured.

Testing

  • Compiled the updated notebook cells (3, 7, 11) with compile(...) to verify Python syntax and the fixes to a broken string literal; compilation succeeded for all tested cells.
  • Verified presence of the new configuration keys and functions (WW_PLOT_EVERY_ROUNDS, WW_S3_PLOT_PREFIX, ww_stats_for_matrix, sync_round_plots_to_s3, WW_PLOTS_ROOT) using automated text checks (rg/nl), which confirmed the inserted code locations.
  • Ran a notebook update script end-to-end that wrote the modified XGBWW_OpenML_1049_W7W8_Alpha_Checkpoint.ipynb and reported success (automated edit and save completed).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant