From 2c7e6965c70ce91726584e1b08795d880c2f9820 Mon Sep 17 00:00:00 2001 From: miguelgonrod Date: Mon, 9 Feb 2026 14:25:03 -0500 Subject: [PATCH 1/2] Added documentation to update_issue_spreadsheet Signed-off-by: miguelgonrod --- .../buildfarmer/update_issue_spreadsheet.md | 96 +++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 playbook/buildfarmer/update_issue_spreadsheet.md diff --git a/playbook/buildfarmer/update_issue_spreadsheet.md b/playbook/buildfarmer/update_issue_spreadsheet.md new file mode 100644 index 0000000..e48c50c --- /dev/null +++ b/playbook/buildfarmer/update_issue_spreadsheet.md @@ -0,0 +1,96 @@ +# Update Issue Spreadsheet + +This page explains how to sync active known issues from the buildfarm database into the Buildfarm Issues Spreadsheet. +The sync is performed by [scripts/update_spreadsheet.py](../../../scripts/update_spreadsheet.py). + +## When to run + +Run this after refreshing known issues or when you need the spreadsheet to reflect the current open issues in the database. + +## Prerequisites + +- Access to the Buildfarm Issues Spreadsheet and the service account in [credentials.json](../../../credentials.json). +- The buildfarm database is present and up to date in [buildfarm-tools/database](../../database/). +- Python deps installed (the script uses `gspread`). + +## Create spreadsheet credentials + +Steps to get the spreadsheet credentials: + +1. Go to https://console.cloud.google.com with your Google account. +2. Create a new project, then enable **Google Sheets API** and **Google Drive API**. +3. Go to https://console.cloud.google.com/apis/credentials and create a new Service account credential. + - Give the credential a name and a description. + - With no additional parameters, save the credential. +4. Click your credential address and go to **Keys**. + - Click **Add Key** > **Create new key**. + - Select **JSON**. + - Save it as [buildfarm-tools-private/credentials.json](../../../credentials.json). +5. Go to the spreadsheet you want to edit (in this case **Buildfarm Issue Triage**). + - Click **Share**. + - Share the spreadsheet with the `client_email` found in the downloaded credentials file. + +## Data source + +The script executes [get_active_known_issues.sql](../../database/scripts/get_active_known_issues.sql) via `sql_run.sh`. +The SQL query is: + +```sql +SELECT + github_issue, + status, + assignee, + priority, + created_at, + issue_last_activity +FROM test_fail_issues +WHERE status != "CLOSED" +GROUP BY github_issue; +``` + +## What the script does + +1. Runs the SQL query and parses each row as one GitHub issue. +2. Builds the issue key as `owner/repo#number` from the issue URL. +3. Routes each issue to a sheet tab: + - **Gazebo Issues** if the key contains `gazebo` or `gz-` (case-insensitive). + - Otherwise to **ROS Issues**. +4. Updates existing rows (match by issue key in column A) or appends new rows. +5. Paints newly appended rows green for visibility. + +The spreadsheet columns are populated as: + +- Issue key (`owner/repo#number`) +- Assignee (or `Not assigned`) +- Status (or `Not assigned`) +- Created at +- Last activity +- Priority +- Notes (empty) + +## Run the sync + +From the repository root: + +```bash +python3 scripts/update_spreadsheet.py +``` + +If you run it from another directory, the script still works because it resolves paths relative to its location. + +## Troubleshooting + +- **SQL error**: Make sure [buildfarm-tools/database/scripts/sql_run.sh](../../database/scripts/sql_run.sh) is executable and the database exists. +- **Auth error**: Verify the service account in [credentials.json](../../../credentials.json) has access to the spreadsheet. +- **Sheet not found**: Check `SHEET_NAME`, `ROS_SHEET`, and `GAZEBO_SHEET` in [scripts/update_spreadsheet.py](../../../scripts/update_spreadsheet.py). +- **Invalid URLs skipped**: The database must store full GitHub issue URLs (e.g., `https://github.com/org/repo/issues/123`). + +# Links to other pages + +## Guides + +* [Known Issues](./known_issues.md) +* [Reporting Problems](./reporting_problems.md) +* [Buildfarmer triage tools](./buildfarmer_triage_tools.md#database-scripts-usage) + +[Back :arrow_backward: ](../index.md) From 470664bc55b4b7dbf3f418132ec7daa161535e49 Mon Sep 17 00:00:00 2001 From: miguelgonrod Date: Mon, 9 Feb 2026 14:28:20 -0500 Subject: [PATCH 2/2] Added a reference in index to update spreadsheet Signed-off-by: miguelgonrod --- playbook/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/playbook/index.md b/playbook/index.md index ae9aefb..654ee2b 100644 --- a/playbook/index.md +++ b/playbook/index.md @@ -5,6 +5,7 @@ This is the index of playbooks for buildfarm. Intended to provide guidance when * [Is an error a known issue or a flaky problem?](./is-an-error-a-known-issue-or-a-flaky-problem.md) * [Check historical data of a test regression](./check-historical-data-of-a-test-regression.md) * [Look for build regression statistics](./look-for-build-regression-statistics.md) +* [Update automatically the buildfarmers issue spreadsheet](./buildfarmer/update_issue_spreadsheet.md) ## Buildfarmer Triage * [Buildfarm introduction :arrow_forward:](./buildfarmer/README.md)