Zero minutes adoption #4
Description
Thanks to this GHA users report they can now have Nyrkiö running in their workflows in 15 minutes. The demo video can be compressed into 5 minutes: https://www.youtube.com/watch?v=ZTn5WD1vx44 This is great, but we can do better. This ticket lists "v3" improvements that will make this GHA so simple that users literally only need to copy paste 1-2 lines of yaml and everything just works. Defaults work. User doesn't need to create a Nyrkiö JWT token because we integrate and use the permissions of the GITHUB_TOKEN. Ultimately user might never need to even visit nyrkio.com.
Each line item below is a task of its own. Please create subtasks when starting to work on one, or in any case submit each as a separate PR.
- A GHA has access to a GITHUB_TOKEN, that can be used during the time the PR is running the workflows. It seems this can be used to create a second token, again valid only as long as the PR workflows are running, that could be used to post comments and issues as needed. As a result, the NYRKIO_JWT_TOKEN is no longer needed at all.
- Allow a user to post test results to nyrkio.com API without requiring to create a nyrkio.com user account first. This assumes we can somehow read and verify a users GitHub identity from the GITHUB_TOKEN.
- Related to the above, on the backend, separate 404 and 401 error messages when a user is not logged in. In the second case, error message should make it clear that test results/graphs exist at this URL, but user must log in to see them.
- Print naive graphs from the GHA to stdout. The motivation is to show simple graphs in the log that users for sure are looking, even if they did not yet create a nyrkio.com user account. Of course, there should then also be a link to a SVG version of the same graph available at nyrkio.com.
- Auto-detect the correct value for the
tool:field. For example, look for json files in the build/work directory, and if found, check if they are in a known format, such as JMH output files. If yes, parse them as if user had specifiedtool: jmh. Note that with this approach, it is possible for the user to run multiple benchmarks with different frameworks, and this GHA will pick up and detect all of them. - Add a "generic" parser similar to https://github.com/nyrkio/dsi/blob/07ca6c2e21a9516a8d77d7d66657428d11d4a0ff/configurations/analysis/analysis.cratedb.yml#L15