inject_into_ssl breaks Python kafka

[MichaelJJ]

Our Kafka server uses SASL_SSL authentication. When we use the standard truststore method, kafka-python is throwing a NoBrokersAvailable. If we comment out the truststore.inject_into_ssl() line, Kafka is able to communicate.

It appears that Python Kafka and truststore are conflicting. We are using Python-Kafka 2.3

[MichaelJJ]

It appears that this fails when truststore is enabled:

self._sock = self._ssl_context.wrap_socket(
                self._sock,
                server_hostname=self.host.rstrip("."),
                do_handshake_on_connect=False)

https://github.com/dpkp/kafka-python/blob/2.3/kafka/conn.py#L522

[MichaelJJ]

More context:

When python kafka is establishing a connection, _get_unverified_chain_bytes is empty, which causes truststore to throw Peer sent no certificates to verify.

If I change _verify_peercerts to:

cert_bytes = _get_unverified_chain_bytes(sslobj)
if not cert_bytes:
    return

Kafka is able to connect. It appears that the native SSL wrap_socket does not call _verify_peercerts

[skloibhofer]

Can this be fixed by instead using the "finer-grained" method with truststore.SSLContext and passing it only to the calls where it is needed? (Examples in the User Guide)

[MichaelJJ]

We looked into that, but we are using the global method because we have multiple modules that import 3rd party libraries that need the updated SSL handling.

We want to avoid patching all of the libraries. So far, only the python Kafka library seems to be having trouble with truststore.