Examples Generator

[krowvin]

Generating examples using the nbformat library to write notebook files from the mock tests files.

This PR adds:

• a generator script, scripts/generate_examples.py
• Generated output files of all existing tests in main
  • Skips the root .\tests\. directory files
• A single edit to tests/timeseries/timeseries_test.py to use real values for notebook testing

Gotchas

Going this route we would want to start using real data in our mock tests files. I.e. instead of using Test as a project we would use KEYS/etc.

This is an initial run through of the test generator, we may want to extend it to show more, change verbiage, or ignore certain method names.

Do we want to include this in a github action? I.e. create these files for the repo when a new test is added?
Personally i'd lean towards running this as needed.

Other thoughts

Should we add links in the pydoc for each function

Examples

Should we also generate generic non Jupyter Notebook files? i.e. the files here

To best view the notebooks below, click the 3 dots and "view file" to render it

[Enovotny]

I think overall this could be useful to get examples of everything. We need to add some more tests. Things that I would like improved, and this isn't necessarily a you thing, as I could look into this as well.

1. we should figure out a way to include the json resources in the notebook so people can see how the json is structure for storing.
2. I don't like the import statements. I think every thing should come from cwms. like cwms.init_session. Or not using
   import cwms.timeseries.timeseries_profile as timeseries
   and then
   timeseries.get_timeseries_profile(office_id, location_id, parameter_id)

when the user can just do
import cwms
cwms.get_timeseries_profile(office_id, location_id, parameter_id)

3. order it as store, get, delete. so we are storing the data then getting it and finally deleting. That might be the structure of the test though so we can work on that.

I do also like the more training examples that I have and keeping them separate. but these are also helpful for the more obscure calls.

[sonarqubecloud]

Quality Gate passed

Issues
21 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[krowvin]

We need to add some more tests.

Test generation could also be automated, but that might be too much. Because at some point we are catering/managing the library manually and flirt the line of automating it all and losing that control.

we should figure out a way to include the json resources in the notebook so people can see how the json is structure for storing.

I have it reading the resources file for the data_in/post body of the responses. But I do not have it reading them as outputs. Instead I was gunning on having the notebooks render the actual live output during the generation and having it store that raw output. But I was not sure if there was a way to store this generated output with the notebook prior to it being ran. I tried. Just left the data.json and data.df statements in for the user to run and see output.

• cwms-python/scripts/generate_examples.py

  Line 87 in 8caacd7

  def load_mock_data(resource_data):

I don't like the import statements.

I think this is due to the tests themselves showing the imports this way.

Here's one of the current tests

• cwms-python/tests/timeseries/timeseries_profile_test.py

  Line 9 in 56b52df

  import cwms.timeseries.timeseries_profile as timeseries

I have it pulling the actual import strings out of each file and putting it at the top, then prepending this import as well.
i.e.

• cwms-python/scripts/generate_examples.py

  Line 163 in 8caacd7

  ["from cwms.api import init_session"] + setup_code

If we change the way things are imported in the tests, it will fix in the output examples. Perhaps when we go to do item # 3 we could do this?

That might be the structure of the test though so we can work on that.

You are right, the order of the items in the tests is the order of the example output.

I don't have it reserving comments, but we could output those to the examples too if we wanted.

[krowvin]

More PR that could come of this

1. Add links to examples in the function pydocs so when users hover / see pydocs in IDE they can click link to examples / keeps it dynamic so we don't have to keep updating pydoc on new changes.

2. Go through all tests and change how the imports are done

3. Go through all tests and change the order of the tests to store, get, delete (proposed above)

4. Fix all tests so that they have actual data to make queries with when they run the notebooks that are generated

5+ ????

Let me know if you approve of any of these and i'll start on them, and if you need any other changes to submit this PR