Kafka connector fallback hosts need to be explicitly added to the connector properties #185
Description
When using the Kafka connector fallback hosts aren't tried and the batch publish fails and goes to dlq (if configured) if the request times out
2024-06-19 09:49:54,627] ERROR Error while sending batch (com.ably.kafka.connect.client.DefaultAblyBatchClient:108)
connector | io.ably.lib.types.AblyException$HostFailedException: java.lang.Exception: connect timed out
connector | at io.ably.lib.types.AblyException.fromErrorInfo(AblyException.java:32)
connector | at io.ably.lib.types.AblyException.fromErrorInfo(AblyException.java:25)
connector | at io.ably.lib.http.Http$Request.sync(Http.java:45)
connector | at io.ably.lib.http.HttpPaginatedQuery.exec(HttpPaginatedQuery.java:56)
connector | at io.ably.lib.http.HttpPaginatedQuery.exec(HttpPaginatedQuery.java:40)
connector | at io.ably.lib.rest.AblyBase.request(AblyBase.java:288)
connector | at com.ably.kafka.connect.client.DefaultAblyBatchClient.sendBatches(DefaultAblyBatchClient.java:188)
connector | at com.ably.kafka.connect.client.DefaultAblyBatchClient.publishBatch(DefaultAblyBatchClient.java:90)
connector | at com.ably.kafka.connect.batch.BatchProcessingThread.run(BatchProcessingThread.java:36)
connector | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
connector | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
connector | at java.base/java.lang.Thread.run(Thread.java:834)
connector | Caused by: java.lang.Exception: connect timed out
connector | ... 11 more
it appears in the connector config that the list of fallbacks is an empty list
connector | client.fallback.hosts = []
The fallback host client option is set to this which overwrites the default fallback hosts which would be set on a Java SDK client if the fallback host option was not set
Workaround: Add the fallbacks explicitly to the connector config