Skip to content

Conversation

@ShapovalovaIrina
Copy link

Hi,

Thanks for the great library.

In my projects, I encountered the need to send messages to different Kafka instances. Also there was an issue #155 for this functionality.

Configuration

Similar to multiple consumers #131, I added the ability to configure multiple producers:

config :kaffe,
  producers: %{
    "producer_1" => [
      endpoints: [kafka1: 9092], # [hostname: port]
      topics: ["kafka-topic"],
  
      # optional
      partition_strategy: :md5,
      ssl: true,
      sasl: %{
        mechanism: :plain,
        login: System.get_env("KAFFE_PRODUCER_USER"),
        password: System.get_env("KAFFE_PRODUCER_PASSWORD")
      }
    ],
    "producer_2" => [
      endpoints: [kafka2: 9092], # [hostname: port]
      topics: ["another-kafka-topic"]
    ]
  }

Single producer configuration with :kaffe, :producer is preserved.

Message producing

To ensure backward compatibility with a single client, the produce_sync_with_client function was added, where the first argument is the producer name. The remaining arguments are identical to the produce_sync function.

Examples:

Kaffe.Producer.produce_sync_with_client("producer_1", "topic", [{"key1", "value1"}, {"key2", "value2"}])
Kaffe.Producer.produce_sync_with_client("producer_2", "key", "value")

@ShapovalovaIrina ShapovalovaIrina marked this pull request as ready for review December 15, 2025 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant