From 4956f24776a8abf0641ba2aa1eefe842b09c9806 Mon Sep 17 00:00:00 2001 From: Stephan Hoyer Date: Mon, 3 Nov 2025 14:35:58 -0800 Subject: [PATCH] Update Fiddle docs and GitHub CI configs The docs now build successfully again on ReadTheDocs: https://fiddle--595.org.readthedocs.build/en/595/index.html I've switched GitHub CI builds to use Python 3.11 or 3.12, which are much more modern than the ancient versions of Python that were previously being tested. Python 3.13 would be great, but some of the test dependencies aren't available. PiperOrigin-RevId: 827654128 --- .github/workflows/testing.yaml | 4 +- .readthedocs.yml | 4 +- docs/api_reference/core.rst | 10 -- docs/colabs.md | 1 - docs/conf.py | 4 + docs/flags_code_lab.md | 247 ++++++++++++++------------------- docs/requirements.txt | 8 +- setup.py | 3 - 8 files changed, 115 insertions(+), 166 deletions(-) diff --git a/.github/workflows/testing.yaml b/.github/workflows/testing.yaml index 43b27069..14ce88b7 100644 --- a/.github/workflows/testing.yaml +++ b/.github/workflows/testing.yaml @@ -10,8 +10,10 @@ jobs: build: runs-on: ubuntu-latest strategy: + fail-fast: false matrix: - python-version: ["3.8", "3.9", "3.10"] + # tensorflow-text (and hence seqio) is not available for python 3.13 + python-version: ["3.11", "3.12"] steps: - uses: actions/checkout@v3 diff --git a/.readthedocs.yml b/.readthedocs.yml index 3a0b3097..61a65b2d 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -7,9 +7,9 @@ version: 2 # Set the version of Python and other tools you might need build: - os: ubuntu-20.04 + os: ubuntu-24.04 tools: - python: "3.9" + python: "3.12" # Build documentation in the docs/ directory with Sphinx sphinx: diff --git a/docs/api_reference/core.rst b/docs/api_reference/core.rst index ccb8114e..1edd2988 100644 --- a/docs/api_reference/core.rst +++ b/docs/api_reference/core.rst @@ -50,13 +50,3 @@ Advanced Functionality ~~~~~~~~~~~~~ .. autoclass:: Buildable :members: __init__, __build__, __flatten__, __unflatten__, __path_elements__, __setattr__, __delattr__, __dir__, __repr__, __copy__, __deepcopy__, __eq__, __getstate__, __setstate__ - - -ArgFactory Internals -~~~~~~~~~~~~~~~~~~~~ -.. currentModule:: fiddle._src.config -.. autoclass:: _BuiltArgFactory - -Casting -~~~~~~~ -.. autofunction:: register_supported_cast diff --git a/docs/colabs.md b/docs/colabs.md index 60f7b943..ff967ea1 100644 --- a/docs/colabs.md +++ b/docs/colabs.md @@ -11,7 +11,6 @@ Please send us feedback if you have any suggestions for improvements! Or if you built something cool and want to share it out, we'd love to link it up at the "Where to go from here" section. -[TOC] ## General colab advice diff --git a/docs/conf.py b/docs/conf.py index 163cc913..6e377565 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -99,3 +99,7 @@ # Enable auto-generation of anchors for headings in markdown files. myst_heading_anchors = 3 + +# Allow non-consecutive header level increase: +# https://stackoverflow.com/questions/73074870/myst-allowing-non-consecutive-header-level-increase +suppress_warnings = ['myst.header'] diff --git a/docs/flags_code_lab.md b/docs/flags_code_lab.md index a8bc720d..b7ce85f3 100644 --- a/docs/flags_code_lab.md +++ b/docs/flags_code_lab.md @@ -7,48 +7,33 @@ to Fiddle's absl_flags integration! This code lab will describe how it all works. Configuration of fiddle config objects via command line arguments is supported using 3 APIs: -| API | Purpose | -| ---------- | -------------------------------------------------------------- | -| New API, | Defines custom configurations per binary using the API | -: single : `DEFINE_fiddle_config()` which returns a config object after : -: config : applying all the command line overrides in order. : -: : The usage of this API is more intuitive than the legacy API, : -: : and it provides the ability to define custom overrides per : -: : binary as well as read serialized configs from a file or : -: : strings on the command line. Additionally, the overrides are : -: : applied in order, which is a more intuitive user experience : -: : than the current order followed by the legacy API. : -| New API, | Defines a 'sweep' of multiple custom configurations per binary | -: multiple : using the API `DEFINE_fiddle_sweep()`. This is similar to : -: configs : `DEFINE_fiddle_config` but returns a sequence of multiple : -: : configs each with some metadata. It allows specifying them : -: : via a sequence of overrides to config attributes, and/or : -: : overrides to arguments of the function that generates configs. : -: : This is intended mainly for use in launch binaries of ML : -: : experiments which perform hyperparameter sweeps. : -| Legacy API | Invoked via `create_buildable_from_flags()` that returns a | -: : config object. Command line overrides are NOT applied in : -: : order; all fiddlers are applied first, followed by all tags, : -: : followed by all overrides. : +* **New API, single config**: Defines custom configurations per binary using + the API `DEFINE_fiddle_config()` which returns a config object after + applying all the command line overrides in order. The usage of this API is + more intuitive than the legacy API, and it provides the ability to define + custom overrides per binary as well as read serialized configs from a file + or strings on the command line. Additionally, the overrides are applied in + order, which is a more intuitive user experience than the current order + followed by the legacy API. +* **New API, multiple configs**: Defines a 'sweep' of multiple custom + configurations per binary using the API `DEFINE_fiddle_sweep()`. This is + similar to `DEFINE_fiddle_config` but returns a sequence of multiple configs + each with some metadata. It allows specifying them via a sequence of + overrides to config attributes, and/or overrides to arguments of the + function that generates configs. This is intended mainly for use in launch + binaries of ML experiments which perform hyperparameter sweeps. +* **Legacy API**: Invoked via `create_buildable_from_flags()` that returns a + config object. Command line overrides are NOT applied in order; all fiddlers + are applied first, followed by all tags, followed by all overrides. > NOTE: New usages of the legacy flags API are discouraged and users should > migrate their legacy usage to the new API. -See some working examples of the APIs below. +See some working examples of the APIs below: -
- -### New API {.new-tab} - -[example](http://github.com/google/fiddle/tree/main/fiddle/_src/absl_flags/sample_test_binary.py) - -### Legacy API {.new-tab} - -[example](http://github.com/google/fiddle/tree/main/fiddle/absl_flags/example) +- [New API](http://github.com/google/fiddle/tree/main/fiddle/_src/absl_flags/sample_test_binary.py) +- [Legacy API](http://github.com/google/fiddle/tree/main/fiddle/absl_flags/example) -
- -[TOC] ## How to structure your code @@ -88,7 +73,7 @@ When our example program is executed, the following steps occur:
-### New API (single config) {.new-tab} +### New API (single config) 1. **Launch**: We run our program on the command line: @@ -109,7 +94,7 @@ When our example program is executed, the following steps occur: arbitrary functions on the built objects to carry out whatever task your application has been designed to do. -### New API (multiple configs) {.new-tab} +### New API (multiple configs) 1. **Launch**: We run our launcher program on the command line to launch multiple configs: @@ -124,13 +109,11 @@ When our example program is executed, the following steps occur: 2. **Flag Parsing**: The custom Fiddle flag parser parses Fiddle-related flags from the command line, and applies any overrides specified in the sweeps to - create a sweep of one or more configs. - If multiple sweeps are specified, the cartesian product of the sweeps is - taken before applying them. - Any fiddler: or set: commands are then applied to all configs in the sweep, - in the order specified. - `_SAMPLE_FLAG.value` returns a sequence of SweepItem dataclasses, each of - which has a `.config` property of type `fdl.Buildable`, and an + create a sweep of one or more configs. If multiple sweeps are specified, the + cartesian product of the sweeps is taken before applying them. Any fiddler: + or set: commands are then applied to all configs in the sweep, in the order + specified. `_SAMPLE_FLAG.value` returns a sequence of SweepItem dataclasses, + each of which has a `.config` property of type `fdl.Buildable`, and an `overrides_applied` property which is the dict of overrides and can be useful to log as metadata attached to each experiment launched. @@ -142,7 +125,7 @@ When our example program is executed, the following steps occur: 4. **Business logic in the main binary**: Is the same as in the single-config case. -### Legacy API {.new-tab} +### Legacy API 1. **Launch**: We run our program on the command line: @@ -196,114 +179,88 @@ When our example program is executed, the following steps occur: ## Flag Syntax -The Fiddle flag integration supports the following flag syntax: - -- **Base Config**: The base configuration function is specified with the flag: +The Fiddle flag integration supports the following flag syntax.
-### New API {.new-tab} - -that was set as the `name` argument for `DEFINE_fiddle_config` or -`DEFINE_fiddle_sweep`, and the command `config`. For example, if the flag object -was instantiated as -`DEFINE_fiddle_config(name="my_flag", ...)`, then the base config is specified -by using `--my_flag -config:some_function_returning_fiddle_config_to_be_overridden()`. -With `DEFINE_fiddle_config` one can also use the command `config_file` to read -from a JSON serialized config written to a file, or the command `config_str` to -read from a JSON serialized config in encoded string form (the additional -encoding involves zlib compression followed by base64 encoding). - -### Legacy API {.new-tab} +### New API + +- **Base Config**: The base configuration function is specified with the + `config` command, following the flag name provided to `DEFINE_fiddle_config` + or `DEFINE_fiddle_sweep`. For example, if the flag object was instantiated + as `DEFINE_fiddle_config(name="my_flag", ...)`, then the base config is + specified by using `--my_flag + config:some_function_returning_fiddle_config_to_be_overridden()`. With + `DEFINE_fiddle_config` one can also use the command `config_file` to read + from a JSON serialized config written to a file, or the command `config_str` + to read from a JSON serialized config in encoded string form (the additional + encoding involves zlib compression followed by base64 encoding). + +- **Fiddlers**: Fiddlers are specified on the command line with the `fiddler` + command after the `name` argument for `DEFINE_fiddle_config` or + `DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as + `DEFINE_fiddle_config(name="my_flag", ...)` then the fiddlers would be + invoked like `--my_flag fiddler:name_of_fiddler(value="new_value")`. -`--fdl_config`. Example: `--fdl_config=base`. Alternatively, a JSON-serialized -configuration can be read from a file via with the flag `--fdl_config_file`. -Example: `--fdl_config_file='/some/path/config.json'`. - -
+- **Specific Overrides**: Specific overrides allow you to specify specific + values to arbitrary fields on the command line. The syntax is the `set` + command after the `name` argument for `DEFINE_fiddle_config` or + `DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as + `DEFINE_fiddle_config(name="my_flag", ...)`, then the specific overrides are + specified using `--my_flag set:some_attr.some_sub_attr=some_value`. + +- **Sweeps**: Sweeps allow you to specify multiple dicts of overrides to + apply, to generate a sweep of one or more configs. The syntax is the `sweep` + command after the `name` argument for `DEFINE_fiddle_sweep`. For example, if + the flag object was instantiated as `DEFINE_fiddle_sweep(name="my_flag", + ...)`, then one or more sweep functions can be specified using `--my_flag + sweep:name_of_sweep(arguments=123)` or just `--my_flag sweep:name_of_sweep` + if no arguments are required to the sweep function. + + A `sweep:` command should specify a function call returning a list of + dictionaries, where each dictionary represents a single item in the sweep. + The entries in the dictionary are the overrides to apply, where keys can be + of the form: + + * `kwarg:foo` -- to specify or override a keyword argument to the base + config function specified by the `config:` command. + * `arg:0` -- to specify or override a positional argument to the base + config function specified by the `config:` command. + * `path.to.some.attr` -- to specify an override to an attribute in the + config returned by the base config function. These paths follow the same + format as is accepted by `set:` commands and can take quite general + forms like `foo.bar['baz'][0].boz`. + + Multiple `sweep:` commands can be specified, which will result in taking the + cartesian product of the separate sweeps before applying them. + +### Legacy API + +- **Base Config**: The base configuration function is specified with the + `--fdl_config` flag. Example: `--fdl_config=base`. Alternatively, a + JSON-serialized configuration can be read from a file with the flag + `--fdl_config_file`. Example: `--fdl_config_file='/some/path/config.json'`. - **Fiddlers**: Fiddlers are specified on the command line with the - -
- -### New API {.new-tab} - -command `fiddler` after the `name` argument for `DEFINE_fiddle_config` or -`DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as -`DEFINE_fiddle_config(name="my_flag", ...)` then the fiddlers would be invoked -like `--my_flag fiddler:name_of_fiddler(value="new_value")`. - -### Legacy API {.new-tab} - -`--fiddler=` flag. This flag can be specified multiple times. Example: -`--fiddler=swap_weight_and_biases --fiddler=other_fiddler`. - -
+ `--fiddler=` flag. This flag can be specified multiple times. Example: + `--fiddler=swap_weight_and_biases --fiddler=other_fiddler`. - **Specific Overrides**: Specific overrides allow you to specify specific values to arbitrary fields on the command line. The syntax is - -
- -### New API {.new-tab} - -the command `set` after the `name` argument for `DEFINE_fiddle_config` or -`DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as -`DEFINE_fiddle_config(name="my_flag", ...)`, then the specific overrides are -specified using `--my_flag set:some_attr.some_sub_attr=some_value`. - -### Legacy API {.new-tab} - -`--fdl.dotted.path.of.fields=3`, where everything after the `fdl.` prefix -corresponds to the name of a field in the Fiddle configuration object -corresponding to exactly the same Python code. For example, if (in Python) we -wanted to set the value of a nested field to 15, I might write: -`cfg.model.dense1.parameters = 15`. The corresponding syntax on the command line -would be: `--fdl.model.dense1.parameters=15`. Due to shell escaping, to specify -a string value, you often need to use two nested quotes, or escape the outer -quotes (depending on your shell). For example: -`--fdl.data.filename='"other.txt"'` (or equivalently: -`--fdl.data.filename=\"other.txt\"`). Only "literal" values may be specified on -the command line like this; if you'd like to set a complex value, please write a -fiddler and invoke it with the previous Fiddlers syntax. - -
- -- **Sweeps**: Allow you to specify multiple dicts of overrides to apply, to - generate a sweep of one or more configs. The syntax is - -
- -### New API {.new-tab} - -the command `sweep` after the `name` argument for `DEFINE_fiddle_sweep`. For -example, if the flag object was instantiated as -`DEFINE_fiddle_sweep(name="my_flag", ...)`, then one or more sweep functions can -be specified using `--my_flag sweep:name_of_sweep(arguments=123)` or just -`--my_flag sweep:name_of_sweep` if no arguments are required to the sweep -function. - -A sweep: command should specify a function call returning a list of -dictionaries, where each dictionary represents a single item in the sweep. -The entries in the dictionary are the overrides to apply, where keys can be of -the form: - - * `kwarg:foo` -- to specify or override a keyword argument to the base config - function specified by the `config:` command. - * `arg:0` -- to specify or override a positional argument to the base config - function specified by the `config:` command. - * `path.to.some.attr` -- to specify an override to an attribute in the - config returned by the base config function. These paths follow the same - format as is accepted by `set:` commands and can take quite general forms - like `foo.bar['baz'][0].boz`. - -Multiple `sweep:` commands can be specified, which will result in taking the -cartesian product of the separate sweeps before applying them. - -### Old API {.new-tab} - -This is not supported in the old API. + `--fdl.dotted.path.of.fields=3`, where everything after the `fdl.` prefix + corresponds to the name of a field in the Fiddle configuration object + corresponding to exactly the same Python code. For example, if (in Python) + we wanted to set the value of a nested field to 15, I might write: + `cfg.model.dense1.parameters = 15`. The corresponding syntax on the command + line would be: `--fdl.model.dense1.parameters=15`. Due to shell escaping, to + specify a string value, you often need to use two nested quotes, or escape + the outer quotes (depending on your shell). For example: + `--fdl.data.filename='"other.txt"'` (or equivalently: + `--fdl.data.filename=\"other.txt\"`). Only "literal" values may be specified + on the command line like this; if you'd like to set a complex value, please + write a fiddler and invoke it with the previous Fiddlers syntax. + +- **Sweeps**: This is not supported in the legacy API.
diff --git a/docs/requirements.txt b/docs/requirements.txt index aeb20f41..cc40a7de 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,10 +1,10 @@ -sphinx >4, <5 +sphinx sphinx-autodoc-typehints -sphinx-book-theme>=0.3.3 -sphinx-copybutton>=0.5.0 +sphinx-book-theme +sphinx-copybutton sphinx-design sphinx-remove-toctrees -jupyter-sphinx>=0.3.2 +jupyter-sphinx myst-nb # Install the Fiddle package itself. diff --git a/setup.py b/setup.py index 2873de85..be49b022 100644 --- a/setup.py +++ b/setup.py @@ -94,9 +94,6 @@ # Specify the Python versions you support here. In particular, ensure # that you indicate whether you support Python 2, Python 3 or both. 'Programming Language :: Python :: 3', - 'Programming Language :: Python :: 3.8', - 'Programming Language :: Python :: 3.9', - 'Programming Language :: Python :: 3.10', 'Topic :: Scientific/Engineering', 'Topic :: Scientific/Engineering :: Mathematics',