Add partitions and replication-factor#323
Conversation
Ran into failing tests due to a partition and replication-factor not being in place in the create if not exists clause.
|
Kafka 2.5.0, or maybe it was 2.4.0, introduced the option to get those from defaults. Helps a lot when reusing the tests on clusters of different sizes. We should check why that isn't happening. |
|
Ah.... Well apache/kafka@8e16158 indicates that it should be in 2.4.1 (I'm using the latest in this repo which is 2.4.1). Digging further it appears that https://github.com/apache/kafka/blob/c57222ae8cd7866b7191722656f3065afaee5b7a/core/src/main/scala/kafka/admin/TopicCommand.scala#L674 would indicate that when we are creating and don't have the replicatAssignment and do have the zookeeper option it checks the partitions and replicationFactor options. I guess beyond that is slightly out of my depth of understanding. https://stackoverflow.com/questions/48986780/kafka-create-topic-with-default-number-of-partitions indicates (without substantiation) that if you have auto.create.topics.enable set then the default partitions should also work... I may be adding that so this all might be a moot point for me, but thought overall it would be good for this test to work for the "defaults" set in this project. Hope that helps. |
|
I recommend against auto.create.topics.enable. It has caused us too many surprises over the years, because a typo or mistake in a client will create a new topic instead of alerting us through a failure. See 4c389e7. The CLI args were removed in 7ffbfdc but that commit was vague about version requirements. I started #324 to help us investigate. |
Ran into failing tests due to a partition and replication-factor not being in place in the create if not exists clause.