Run EndToEnd tests Periodically

[davidbrownell]

Is your feature request related to a problem? Please describe.

The EndToEnd tests interact with the GitHub API, which is rate limited. Unfortunately, we exceed those limits when many CI workflows are run within a relatively short period of time.

Describe the solution you'd like

1. The standard CI workflow should execute all tests EXCEPT EndToEnd tests.
2. We should introduce a new CI workflow that executes EndToEnd tests; this workflow should be run periodically. Ideally it will run at least once per day or every hour when changes have been introduced (if possible). It would be nice if failures in this periodic workflow notified people who have made changes since the last time the periodic workflow was run (if possible).

Describe alternatives you've considered

N/A

Additional context

N/A

[varunagrawal]

Would this need restructuring of the tests directory to look like the following?

tests
 |- UnitTests
   |- CommandLineProcessor_UnitTest.py
   |- ...
 |- EndToEndTests
   |- GitHub_EndToEndTest.py
   |- (Any future tests that require an API end point)

Then we could call uv run pytest tests/EndToEndTests in the CI. Using uv run pytest would still run all tests.

[davidbrownell]

I had thought about the change in terms of modifying the command line args used when invoking pytest from the CI workflow rather than modifying the structure of the tests themselves.

For example, when invoking pytest as part of the standard CI workflow, we can update this line to be uv run pytest -k "not EndToEndTest".

The new CI workflow could invoke pytest via uv run pytest -k EndToEndTest.

By doing it this way, the "standard" way of invoking pytest (uv run pytest) will still run all tests when invoked locally (which is the behavior that we want).

[varunagrawal]

A quick note about an issue I am facing: when running uv run pytest -k "not EndToEndTest", the coverage drops from 96% to 83%, which I am guessing is due to the lack of paths covered by the end-to-end tests.

I'll get a draft fix going, but we should strategize on this point.

[varunagrawal]

This has been partially addressed in #97 but @davidbrownell is looking into more specific running criteria.