Skip to content

Conversation

@Selta
Copy link

@Selta Selta commented Oct 2, 2022

Currently, when running with an influx DB that starts in another container, the Speedflux will complete its 3 retries before the influxDB is accepting connections (even with depends_on set in the compose yaml). According to logs, it often gets through these retries in roughly 1 second, as exampled here:

2022-10-02 05:38:56,504 [INFO] Speedtest CLI data logger to InfluxDB started...
2022-10-02 05:38:57,515 [ERROR] Connection to influx host was refused. This likely means that the DB is down or INFLUX_DB_ADDRESS is incorrect. It's currently 'influx-grafana'
2022-10-02 05:38:57,515 [ERROR] Full Error follows

2022-10-02 05:38:57,515 [ERROR] HTTPConnectionPool(host='influx-grafana', port=8086): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb36ff9c8e0>: Failed to establish a new connection: [Errno 111] Connection refused'))
2022-10-02 05:38:57,515 [ERROR] Retry 1: Initiliazing DB.
2022-10-02 05:38:57,521 [ERROR] Connection to influx host was refused. This likely means that the DB is down or INFLUX_DB_ADDRESS is incorrect. It's currently 'influx-grafana'
2022-10-02 05:38:57,522 [ERROR] Full Error follows

2022-10-02 05:38:57,522 [ERROR] HTTPConnectionPool(host='influx-grafana', port=8086): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb36ffbba00>: Failed to establish a new connection: [Errno 111] Connection refused'))
2022-10-02 05:38:57,522 [ERROR] Retry 2: Initiliazing DB.
2022-10-02 05:38:57,527 [ERROR] Connection to influx host was refused. This likely means that the DB is down or INFLUX_DB_ADDRESS is incorrect. It's currently 'influx-grafana'
2022-10-02 05:38:57,527 [ERROR] Full Error follows

2022-10-02 05:38:57,527 [ERROR] HTTPConnectionPool(host='influx-grafana', port=8086): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb36ff12670>: Failed to establish a new connection: [Errno 111] Connection refused'))
2022-10-02 05:38:57,527 [ERROR] Retry 3: Initiliazing DB.
2022-10-02 05:38:57,533 [ERROR] Database Init failed for 3rd time. Exiting

This PR adds a simple exponential backoff to those retries, allowing influxDB a little more time (~28seconds with the default 2s backoff_in_seconds param) to start and accept connections. Comments and criticisms welcome, of course.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant