Doc updates

README.md

@@ -1,8 +1,8 @@
 # CLI Wrapper
 
-![Codecov](https://img.shields.io/codecov/c/github/orstensemantics/cli_wrapper)
+[![Codecov](https://img.shields.io/codecov/c/github/orstensemantics/cli_wrapper)](https://app.codecov.io/gh/orstensemantics/cli_wrapper)
 ![PyPI - License](https://img.shields.io/pypi/l/cli_wrapper)
-![PyPI - Version](https://img.shields.io/pypi/v/cli_wrapper)
+[![PyPI - Version](https://img.shields.io/pypi/v/cli_wrapper)](https://pypi.org/project/cli-wrapper)
 
 
 CLI Wrapper uses subprocess to wrap external CLI tools and present an interface that looks more like a python class. CLI
@@ -104,5 +104,6 @@ pip install dotted_dict # for dotted_dict support shown above
 - [ ] Configuration dictionaries for common tools
     - [x] kubectl
     - [x] helm
-    - [ ] docker
-    - [x] cilium
+    - [x] docker
+    - [x] cilium
+    - [ ] ...

doc/callable_serialization.md

@@ -0,0 +1,82 @@
+# Callable serialization
+
+Argument validation and parser configuration are not straightforward to serialize. To get around this, CLI Wrapper uses
+`CallableRegistry` and `CallableChain`. These make it somewhat more straightforward to create more serializable wrapper
+configurations.
+
+## TL;DR
+- Functions that perform validation, argument transformation, or output parsing are registered with a name in a
+  `CallableRegistry`
+- `CallableChain` resolves a serializable structure to a sequence of calls to those functions
+  - a string refers to a function, which will be called directly
+  - a dict is expected to have one key (the function name), with a value that provides additional configuration:
+    - a string as a single positional arg
+    - a list of positional args
+    - a dict of kwargs (the key "args" will be popped and used as positional args if present)
+  - a list of the previous two
+
+
+- A list of validators is treated as a set of conditions which must be true
+- A list of parsers will be piped together in sequence
+- Transformers receive an arg name and value, and return another arg and value. They are not chained.
+
+## Implementation
+
+Here's how these work:
+
+### `CallableRegistry`
+
+Callable registries form the basis of serializing callables by mapping strings to functions. If you are doing custom
+parsers and validators and you want these to be serializable, you will use their respective callable registries to
+associate the code with the serializable name.
+
+```python
+
+def greater_than(a, b):
+  return a > b
+
+
+registry = CallableRegistry(
+  {
+    "core" = {}
+  }
+)
+registry.register("gt", greater_than)
+
+x = registry.get("gt", [2])
+
+assert(not x(1))
+assert(x(3))
+```
+
+### `CallableChain`
+
+A callable chain is a serializable structure that gets converted to a sequence of calls to things in a
+`cli_wrapper.util.callable_registry.CallableRegistry`. It is an abstract base class, and so shouldn't be created directly; subclasses are expected to
+implement `__call__`. We'll use the `.validators.Validator` class as an example. `validators` is a `CallableRegistry` with all of
+the base validators (`is_dict`, `is_list`, `is_str`, `startswith`...)
+
+```python
+# Say we have these validators that we want to run:
+def every_letter_is(v, l):
+    return all((x == l.lower()) or (x == l.upper()) for x in v)
+
+validators.register("every_letter_is", every_letter_is)
+
+my_validation = ["is_str", {"every_letter_is": "a"}]
+
+straight_as = Validator(my_validation)
+assert(straight_as("aaaaAAaa"))
+assert(not straight_as("aaaababa"))
+```
+
+`Validator.__call__` just checks that every validation returns true. Elsewhere, `Parser` pipes inputs in sequence:
+
+```yaml
+parser:
+  - yaml
+  - extract: result 
+```
+
+This would first parse the output as yaml and then extract the "result" key from the dictionary returned by the yaml
+step.

doc/parsers.md

@@ -0,0 +1,45 @@
+# Parsers
+
+Parsers provide a mechanism to convert the output of a CLI tool into a usable structure. They make use of
+`cli_wrapper.util.callable_chain.CallableChain` to be serializable-ish.
+
+## Default Parsers
+
+1. `json`: uses `json.loads` to parse stdout
+2. `extract`: extracts data from the raw output, using the args as a list of nested keys.
+3. `yaml`: if `ruamel.yaml` is installed, uses `YAML().load_all` to read stdout. If `load_all` only returns one
+   document, it returns that document. Otherwise, it returns a list of documents. `pyyaml` is also supported.
+4. `dotted_dict`: if `dotted_dict` is installed, converts an input dict or list to a `PreserveKeysDottedDict` or 
+   a list of them. This lets you refer to most dictionary keys as `a.b.c` instead of `a["b"]["c"]`.
+
+These can be combined in a list in the `parse` argument to `cli_wrapper.cli_wrapper.CLIWrapper.update_command_`,
+allowing the result of the call to be immediately usable.
+
+You can also register your own parsers in `cli_wrapper.parsers.parsers`, which is a 
+`cli_wrapper.util.callable_registry.CallableRegistry`.
+
+## Example
+
+```python
+from cli_wrapper import CLIWrapper
+
+def skip_lists(result): 
+    if result["kind"] == "List":
+        return result["items"]
+    return result
+
+kubectl = CLIWrapper("kubectl")
+# you can use the parser directly, but you won't be able to serialize the
+# wrapper to json
+kubectl.update_command_(
+   "get",
+   parse=["json", skip_lists, "dotted_dict"],
+   default_flags=["--output", "json"]
+)
+
+a = kubectl.get("pods", namespace="kube-system")
+assert isinstance(a, list)
+b = kubectl.get("pods", a[0].metadata.name, namespace="kube-system")
+assert isinstance(b, dict)
+assert b.metadata.name == a[0].metadata.name
+```

doc/pre_packaged.md

@@ -0,0 +1,90 @@
+# Pre-packaged CLIWrappers
+
+As a convenience, some pre-configured wrappers for common tools are included in the package. At the moment, none of
+them are well tested, but they are complete in terms of commands and flags. They are generated by the help_parser tool
+in this repo.
+
+## Usage
+
+```python
+from cli_wrapper.pre_packaged import get_wrapper
+
+kubectl = get_wrapper("kubectl")
+helm = get_wrapper("helm")
+```
+
+## Available Wrappers
+
+- `kubectl`
+- `helm`
+- `cilium`
+- `docker`
+- `terraform`
+
+## Help Parser and Why These Aren't Well Tested
+
+The help parser works by calling the tool with --help, parsing the output, and then calling the tool with `command --help`
+help. The help parser takes some semi-obscure command line arguments, like default parsers:
+
+```shell
+PYTHONPATH=src python -m help_parser helm --parser-default-pairs json:output=json
+```
+
+This will add `--output json` as a default flag and the json parser to every command that seems to support the 
+`--output` flag. And this usually works, except for fun cases like this one:
+
+```
+> kubectl config get-contexts --help
+Display one or many contexts from the kubeconfig file.
+
+Examples:
+  # List all the contexts in your kubeconfig file
+  kubectl config get-contexts
+
+  # Describe one context in your kubeconfig file
+  kubectl config get-contexts my-context
+
+Options:
+    --no-headers=false:
+        When using the default or custom-column output format, don't print headers (default print headers).
+
+    -o, --output='':
+        Output format. One of: (name).
+
+Usage:
+  kubectl config get-contexts [(-o|--output=)name)] [options]
+
+Use "kubectl options" for a list of global command-line options (applies to all commands).
+```
+
+If you want to use the help parser yourself:
+
+```
+PYTHONPATH=src python -m help_parser --help
+usage: __main__.py [-h] [--help-flag HELP_FLAG] [--style {golang,argparse}] [--default-flags DEFAULT_FLAGS [DEFAULT_FLAGS ...]] [--parser-default-pairs PARSER_DEFAULT_PAIRS [PARSER_DEFAULT_PAIRS ...]] [--default-separator DEFAULT_SEPARATOR] [--long-prefix LONG_PREFIX] [--output OUTPUT] command
+
+Parse CLI command help.
+
+positional arguments:
+  command               The CLI command to parse.
+
+options:
+  -h, --help            show this help message and exit
+  --help-flag HELP_FLAG
+                        The flag to use for getting help (default: 'help').
+  --style {golang,argparse}
+                        The style of cli help output (default: 'golang').
+  --default-flags DEFAULT_FLAGS [DEFAULT_FLAGS ...]
+                        Default flags to add to the command, key=value pairs.
+  --parser-default-pairs PARSER_DEFAULT_PAIRS [PARSER_DEFAULT_PAIRS ...]
+                        parser:key=value,... to configure default parsers.
+  --default-separator DEFAULT_SEPARATOR
+                        Default separator to use for command arguments.
+  --long-prefix LONG_PREFIX
+                        Default prefix for long flags.
+  --output OUTPUT, -o OUTPUT
+                        Output file to save the parsed command.
+
+```
+
+For the moment, argparse style isn't actually implemented, since no significant

doc/transformers.md

@@ -0,0 +1,58 @@
+# Transformers
+
+Argument transformers receive an argument (either a numbered positional argument or a string keywork argument/flag) and
+a value. They return a tuple of argument and value that replace the original.
+
+The main transformer used by cli-wrapper is `cli_wrapper.transformers.snake2kebab`, which converts a `an_argument_like_this` to
+`an-argument-like-this` and returns the value unchanged. This is the default transformer for all keyword arguments.
+
+Transformers are added to a callable registry, so they can be refernced as a string after they're registered.
+Transformers are not currently chained.
+
+## Other possibilities for transformers
+
+### 1. Write dictionaries to files and return a flag referencing a file
+
+Consider a command like `kubectl create`: the primary argument is a filename or list of files. Say you have your 
+manifest to create as a dictionary:
+
+```python
+from pathlib import Path
+from ruamel.yaml import YAML
+from cli_wrapper import transformers, CLIWrapper
+
+manifest_count = 0
+base_filename = "my_manifest"
+base_dir = Path()
+y = YAML()
+def write_manifest(manifest: dict | list[dict]):
+    global manifest_count
+    manifest_count += 1
+    file = base_dir / f"{base_filename}_{manifest_count}.yaml"
+    with file.open("w") as f:
+        if isinstance(manifest, list):
+            y.dump_all(manifest, f)
+        else:
+            y.dump(manifest, f)
+    return file.as_posix()
+
+def manifest_transformer(arg, value, writer=write_manifest):
+    return "filename", writer(value)
+
+transformers.register("manifest", manifest_transformer)
+
+# If you had different writer functions (e.g., different base name), you could register those as partials:
+from functools import partial
+transformers.register("other_manifest", partial(manifest_transformer, writer=my_other_writer))
+
+kubectl = CLIWrapper('kubectl')
+kubectl.update_command_("create", args={"data": {"transformer": "manifest"}})
+
+# will write the manifest to "my_manifest_1.yaml" and execute `kubectl create -f my_manifest_1.yaml`
+kubectl.create(data=my_kubernetes_manifest)
+```
+
+## Possible future changes
+
+- it might make sense to make transformers a [`CallableChain`](callable_serialization.md#callablechain) similar to parser so a sequence of things can be done on an arg
+- it might also make sense to support transformers that break individual args into multiple args with separate values

doc/validators.md

@@ -0,0 +1,61 @@
+from curses import wrapper
+
+# Validators
+
+Validators are used to validate argument values. They are implemented as a
+`cli_wrapper.util.callable_chain.CallableChain` for serialization. Callables in the chain are called with the value
+sequentially, stopping at the first callable that returns False.
+
+## Default Validators
+
+The default validators are:
+
+- `is_dict`
+- `is_list`
+- `is_str`
+- `is_str_or_list`
+- `is_int`
+- `is_float`
+- `is_bool`
+- `is_path` - is a `pathlib.Path`
+- `is_alnum` - is alphanumeric
+- `is_alpha` - is alphabetic
+- `starts_alpha` - first digit is a letter
+- `startswith` - checks if the string starts with a given prefix
+
+## Custom Validators
+
+You can register your own validators in `cli_wrapper.validators.validators`:
+
+1. Takes at most one positional argument
+2. When configuring the validator, additional arguments can be supplied using a dictionary:
+
+```python
+wrapper.update_command_("cmd", validators={"arg":["is_str", {"startswith": {"prefix": "prefix"}}]})
+# or
+wrapper.update_command_("cmd", validators={"arg": ["is_str", {"startswith": "prefix"}]})
+```
+## Example
+
+```python
+from cli_wrapper import CLIWrapper
+from cli_wrapper.validators import validators
+
+def is_alnum_or_dash(value):
+    return all(c.isalnum() or c == "-" for c in value)
+validators.register("is_alnum_or_dash", is_alnum_or_dash)
+
+kubectl = CLIWrapper("kubectl")
+# 1 refers to the first positional argument, so in `kubectl.get("pods", "my-pod")` it would refer to `"my-pod"`
+kubectl.update_command_("get", validators={
+ 1: ["is_str", "is_alnum_or_dash", "starts_alpha"],
+})
+
+assert kubectl.get("pods", "my-pod")
+threw = False
+try:
+    kubectl.get("pods", "level-9000-pod!!")
+except ValueError:
+    threw = True
+assert threw
+```