Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions docs/client_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
"source": [
"### Client | Cluster | Thread Information\n",
"An instance of the Client or Cluster, if they exist, can be retrieved using the client module. These helper functions \n",
"are useful in determining the current Dask client status. It can be useful to check the cluster status becuase sometimes \n",
"are useful in determining the current Dask client status. It can be useful to check the cluster status because sometimes \n",
"even if the client is closed out the cluster instance will persist and multiple ports can be opened if a new client is \n",
"not opened correctly."
]
Expand Down Expand Up @@ -129,7 +129,7 @@
"id": "c3bb1a28-1fcd-4626-905a-2ee4adc8a6d6",
"metadata": {},
"source": [
"There are two ways to get information about the client thread info. If a client instance exists, within the code block the user can simply run the `.thread_info()` method available from the client class. If the user is not usre if a client already exists or was instantiated in another code block, a helper method is available to get the thread info for the client. If a client instance does not exist, an estimation of the available local system resource is returned."
"There are two ways to get information about the client thread info. If a client instance exists, within the code block the user can simply run the `.thread_info()` method available from the client class. If the user is not sure if a client already exists or was instantiated in another code block, a helper method is available to get the thread info for the client. If a client instance does not exist, an estimation of the available local system resources is returned."
]
},
{
Expand Down Expand Up @@ -179,6 +179,14 @@
"source": [
"client"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56394fe4-09e5-4c98-9b2b-cfa4220fcc53",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand All @@ -197,7 +205,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
"version": "3.11.13"
}
},
"nbformat": 4,
Expand Down
29 changes: 23 additions & 6 deletions docs/download_example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,9 @@
"id": "0351dda7-7559-449d-9ebb-c853beb0861e",
"metadata": {},
"source": [
"## Getting available downloadable file in a python list.\n",
"#### This will return an unordered list of the available file on the remote dropbox in a python list. This can be used as an input to thie download function as well."
"## Getting available downloadable files in a python list\n",
"\n",
"This will return an unordered list of the available files on the remote dropbox in a python list. This can be used as an input to the download function as well."
]
},
{
Expand Down Expand Up @@ -55,15 +56,15 @@
"metadata": {},
"source": [
"## Getting download metadata version\n",
"#### This will retireve the current file download metadata version; if the file is not found it will attempt to update to the most recent version."
"This will retrieve the current file download metadata version; if the file is not found it will attempt to update to the most recent version."
]
},
{
"cell_type": "markdown",
"id": "fe18a0d9-bd21-4ce5-91fd-6348e5f1369e",
"metadata": {},
"source": [
"### Manually update metdata info."
"### Manually update metdata info"
]
},
{
Expand All @@ -81,7 +82,7 @@
"id": "fdf5ec65-a98b-412b-b5c6-3b4bb23e8312",
"metadata": {},
"source": [
"## Getting available files and metadata in data frame form.\n",
"## Getting available files and metadata in data frame form\n",
"The output is a polars data frame and has all the functionality contained within. More details can be found on \n",
"the official docs page: https://docs.pola.rs/"
]
Expand Down Expand Up @@ -116,6 +117,14 @@
"files[files.telescope == \"VLA\"]"
]
},
{
"cell_type": "markdown",
"id": "96b7a434-5f47-4cad-a528-fd654dccf9fa",
"metadata": {},
"source": [
"To restrict the number of columns use:"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -125,6 +134,14 @@
"source": [
"files.filter(items=[\"file\", \"path\", \"telescope\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a4b8ff9b-91c9-4faa-ac54-4f1f9c42029c",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand All @@ -143,7 +160,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
"version": "3.11.13"
}
},
"nbformat": 4,
Expand Down
26 changes: 13 additions & 13 deletions docs/parameter.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
The `parameter` module is based off of another well known verification module called [cerberus](https://docs.python-cerberus.org/). Much of the syntax is the same but there is some added functionality specific to the viper and radps frameworks that is only available using `parameter`. A detailed list of validation type that come standard to the `cerberus` package can be found [here](https://docs.python-cerberus.org/validation-rules.html). As a note, custom types can be added as well as detailed below.

### Parameter Configuration File
The parameter configuration files latyout the validation rules for eavh function in a given module and one must be included for the any module that requires validation. The configuration files should be placed in `src/toolviper/config`. The filename should be of the form, `<module_name>.param.json` and is a standard form json file witht he followinf layout.
The parameter configuration files layout the validation rules for each function in a given module and one must be included for any module that requires validation. The configuration files should be placed in `src/toolviper/config`. The filename should be of the form, `<module_name>.param.json` and is a standard form json file witht he following layout.
```
{
"function-name-1":{
Expand Down Expand Up @@ -62,14 +62,14 @@ Then we define a simple configuration file in the following way,
}
```

This will require each parameter to be included as well as enforce the data type of each input parameter. There are many more useless restrictions that can be added to input parameter by referring to the documentaion. Also, custom rules can be made by opening a ticket on the `toolviper` repository under "new feature". Currently, in addition to the standard parameter rules there are two additional features that have been added. The `sequence` and `struct type` parameter checks. The former checsk that a set of inputs are in a given order and the latter works the same as the `type` keword except it defines a type for values that make up a structure. As an example, if we wanted to set a tpye restriction on a list of ddi values to require that they be a list of integers or stirngs we would use the keyword `struct type` as so to define a parameter rule.
This will require each parameter to be included as well as enforce the data type of each input parameter. There are many more useless restrictions that can be added to input parameter by referring to the documentaion. Also, custom rules can be made by opening a ticket on the `toolviper` repository under "new feature". Currently, in addition to the standard parameter rules there are two additional features that have been added. The `sequence` and `struct type` parameter checks. The former checks that a set of inputs are in a given order and the latter works the same as the `type` keword except it defines a type for values that make up a structure. As an example, if we wanted to set a tpye restriction on a list of ddi values to require that they be a list of integers or strings we would use the keyword `struct type` as so to define a parameter rule.

```
"struct type": ["integer", "string"]
```

### Parameter Checking for Class Members
The addition of the paramter module to members of of a class requires only a few simple changes. Let's consider the databse example from above and make the entry funtion a class member of the Database class instead of just being part of the module.
The addition of the parameter module to members of a class requires only a few simple changes. Let's consider the databse example from above and make the entry function a class member of the ``Database`` class instead of just being part of the module.

```
[database.py]
Expand All @@ -83,7 +83,7 @@ class Database:
print(f"{name}: {age}")
```

In this case the changes to the configuration file are simple. We need only make a small change to how we define the funtion-name section.
In this case the changes to the configuration file are simple. We need only make a small change to how we define the function-name section.

```
[database.param.json]
Expand Down Expand Up @@ -125,9 +125,9 @@ function has the following input parameters available.
- `config_dir`: This specifies the configuration directory
- `logger`: This allows the user to pass a specific logger instance to the parameter checking. If this is not done, `toolviper`
will spawn its own internal logger. The only difference will be the logger name in the output.
- `add_data_type`: Not all data types are available from teh default setup more importantly though there are a number of
instance in the framework(s) where custom data objects are passed to function and there needs to be a way to check these
as well. This allows the user to register a custom data ype for checking. All that is needed if to pass an instance of
- `add_data_type`: Not all data types are available from the default setup, more importantly though there are a number of
instances in the framework(s) where custom data objects are passed to functions and there needs to be a way to check these
as well. This allows the user to register a custom data type for checking. All that is needed is to pass an instance of
the data type.
- `custom_checker`: This allows the user to register a function that will return all allowed data types for a parameter.

Expand Down Expand Up @@ -169,10 +169,10 @@ If this was a checker on a class method the only change to be made would be that
`ClassName.snake_danger_checker`.

This is only a very sparse version of a configuration file, the requirements can be made quite strict with some work. A
full list of supported parameter checks at [cerberus::validation](https://docs.python-cerberus.org/validation-rules.html).
full list of supported parameter checks can be found at [cerberus::validation](https://docs.python-cerberus.org/validation-rules.html).
In addition, `toolviper` also supports:

- Custom data types: `ndarray` is supported by default in `toolviper` and any valid data type can be registered.
- Custom data types: `ndarray` is supported by default in `toolviper` and any valid data type can be registered
- Custom checking function
- Sequence validation
- Structure data type
Expand Down Expand Up @@ -209,11 +209,11 @@ def snake_object_danger_checker(number: int, snake_info: SnakeObject) -> Union[f
### Custom Data Checker

This option is for the case that the user wants to define a function that will use some sort of custom logic to decide the
acceptable values for a given parameter and then return them to the validation scheme. The vest example of this is in `astrohack`
acceptable values for a given parameter and then return them to the validation scheme. The best example of this is in `astrohack`
where some of the plotting functions have a different set of acceptable units depending on the plot being made, therefore
we want a easily modifiable list of units that can be returned instead of hard-coding each function separately. The primary
we want an easily modifiable list of units that can be returned instead of hard-coding each function separately. The primary
requirement for a custom checking function is that it take a string parameter and return a list of acceptable values to
check against. In the case of `astrohack` this was a list of acceptable plotting units depending on whether teh function
check against. In the case of `astrohack` this was a list of acceptable plotting units depending on whether the function
needs trigonometric, time or radian units. The custom check function in this case was,

```angular2html
Expand Down Expand Up @@ -265,7 +265,7 @@ In order to add this to the plotting function
...
```

and in the configuration file this function has teh following parameter check for the unit parameter,
and in the configuration file this function has the following parameter check for the unit parameter,

```angular2html
...
Expand Down
16 changes: 12 additions & 4 deletions docs/toolviper-logger-formatting-example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"source": [
"## Color and text formatting for terminal an logging\n",
"\n",
"The colorize module provides a simple way to add text formatting and colors to text output and logging applications. Colorize supports the True color palette. This module was developed specifically for applications within the toolviper framework but is applicable outside as well. There are a number of excellent libraries for these purposes they don't work exactly as needed in the toolviper framework.\n",
"The colorize module provides a simple way to add text formatting and colors to text output and logging applications. Colorize supports the True color palette. This module was developed specifically for applications within the toolviper framework but is applicable outside as well. There are a number of excellent libraries for these purposes but they don't work exactly as needed in the toolviper framework.\n",
"\n",
"**ASCII Escape codes:** \n",
"https://gist.github.com/fnky/458719343aabd01cfb17a3a4f7296797\n",
Expand Down Expand Up @@ -204,7 +204,7 @@
"id": "1545384b-8e5e-4f0f-9d73-a1cb3b036125",
"metadata": {},
"source": [
"## Simple logging using toolviper logger.\n",
"## Simple logging using toolviper logger\n",
"\n",
"The skriba logger is simple to use and will automatically spawn a default logger instance. The setup method is also available for a customized setup,\n",
"\n",
Expand Down Expand Up @@ -276,7 +276,7 @@
"id": "cf7e792c-1bc5-4c86-bd94-cfd1a4138110",
"metadata": {},
"source": [
"### Directly setting verbosity for testing and debugging.\n",
"### Directly setting verbosity for testing and debugging\n",
"This will toggle the verbose flag for all logging levels. The `state` can be either `logger.VERBOSE` or `logger.DEFAULT`."
]
},
Expand All @@ -290,6 +290,14 @@
"logger.set_verbosity(state=logger.VERBOSE)\n",
"verbose_log()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d87b3dbf-1977-4edb-96f2-e027a0e5f396",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand All @@ -308,7 +316,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
"version": "3.11.13"
}
},
"nbformat": 4,
Expand Down