Backlog Features for Buildfarmer tools

[Crola1702]

Migrated from https://github.com/osrf/buildfarmer/issues/475 on Sep 21, 2023

Refactor scripts in database/scripts/ folder

Almost all the features of buildfarmer database tools are stored in this folder, and are run using sql_run.sh script and selecting which sql file you want to run. All the scripts are intended to give information on test regressions, build regressions or the buildfarm in general.

In my case, I don't use some of the scripts that are listed on the folder, such as: errors_match_string.sql, errors_statistical_check_[gazebo|ros].sql, jobs_*.sql, errors_check_[week|month].sql. In fact, some of these scripts are the exact same script with one difference: a string (for example, errors check today, week, month). It would be a good practice to remove duplicated code and parametrize such variables.

In addition, for efficiency purposes, it would be awesome to have default parameters on each of the scripts. e.g., a script called check_errros_in_time_range should be permitted to be called without parameters and show errors in the last day.

The improvement items are:

• Delete unused scripts
• Parametrize scripts to multiple variables (time range, agent, job, domain, builds)
• Create default parameters for scripts

---

Converting buildfarmer scripts into a library

I'm getting more ambitious, but I think it could be a good idea to "Officialize the scripts" making them a real library.

The idea would be creating the library, so it could be easier for anyone to use (JIC we open source this). In this case, we should use a console tool to use the library (known as ./sql_run script). It could also be a way of merging both testdb and buildfarmer db.

Pros:

• Using a library improves the code organization and flexibility to use
• This is a way we could merge both testdb (plain text files database) and buildfarmer db
• When these scripts are open sourced they will not be only in a good shape (developed with good practises), but everyone could make use of them easily
• Newcomers won't have to learn what each of the scripts does, just using the library would be fine.

Cons:

• It will take some effort to: Design the library, refactor all the scripts, creating a console tool that we could use