Skip to content

Conversation

@jsnshrmn
Copy link
Member

Description

  • Add retry logic to base command
  • use tenacity library
  • log at INFO level
  • Remove some exception handling in linksearchtotal_collect command to enable retry

Rationale

We were having quiet cron failure with no retries for linksearchtotal_collect which should run weekly. this change allows for some retries and makes the logs noisier for easier diagnostics.

Phabricator Ticket

https://phabricator.wikimedia.org/T403209

How Has This Been Tested?

Tested by running locally

Screenshots of your changes (if appropriate):

[INFO] Executing linksearchtotal_collect.py
[INFO] reading wiki-list
[INFO] connecting to db en
[WARNING] linksearchtotal_collect.py attempt 1 failed
[INFO] reading wiki-list
[INFO] connecting to db en
[WARNING] linksearchtotal_collect.py attempt 2 failed
[INFO] reading wiki-list
[INFO] connecting to db en
[WARNING] linksearchtotal_collect.py attempt 3 failed
[INFO] reading wiki-list
[INFO] connecting to db en
[WARNING] linksearchtotal_collect.py attempt 4 failed
[INFO] reading wiki-list
[INFO] connecting to db en
[WARNING] linksearchtotal_collect.py attempt 5 failed
[WARNING] Retries exhausted for linksearchtotal_collect.py
[ERROR] (2002, "Can't connect to MySQL server on 'enwiki.analytics.db.svc.wikimedia.cloud' (101)")

Types of changes

What types of changes does your code introduce? Add an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

@jsnshrmn jsnshrmn changed the title Add retry logic for all management commands WIP Add retry logic for all management commands Sep 12, 2025
@katydidnot katydidnot changed the title WIP Add retry logic for all management commands Add retry logic for all management commands Sep 12, 2025
Bug: T403209
Change-Id: Idd17a703d4bf64a0aa46eef875938dad6ef681e7
@katydidnot katydidnot self-requested a review September 15, 2025 20:15
Copy link
Contributor

@suecarmol suecarmol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for implementing the retry for this command. This might be useful for other commands in the future.

@suecarmol suecarmol merged commit e53ff4c into master Sep 16, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants