-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Labels
enhancementNew feature or requestNew feature or request
Description
The run_benchmark needs to be able to run methods with multiple parameter settings. Perhaps with the addition of something like this?
- name: "--parameter_sets"
type: file
description: |
A yaml file containing the parameter sets for each method. Example format:
- method_id: ...
run_id: ...
parameters:
...
- method_id: ...
run_id: ...
parameters:
...If the run_id is not set, it defaults to the method_id. If no parameters are specified, it defaults to {}.
Alternative formats are possible, such as:
- name: "--parameter_sets"
type: file
description: |
A yaml file containing the parameter sets for each method. Example format:
method_1:
param_set1:
par_a: ...
par_b: ...
param_set2:
par_a: ...
par_b: ...
method_2:
param_set1:
par_a: ...
par_b: ...
param_set2:
par_a: ...
par_b: ...The paramset_id / run_id would need to be stored in the .uns, and downstream components would need to be adapted accordingly, so let's think about how the api specs would change and which downstream components are affected.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request