GcpBearerAuthCredentialProvider: eager GoogleCredentials initialization breaks multi-cloud deployments

[wavejumper]

Summary

GcpBearerAuthCredentialProvider eagerly initializes GoogleCredentials in its constructor, causing it to fail during classpath scanning even when not actively used. This breaks multi-cloud deployments where multiple auth providers coexist on the classpath.

Root cause

In com.google.cloud.hosted.kafka.auth.GcpBearerAuthCredentialProvider:

• Line 48 eagerly creates a GoogleCredentials instance in the constructor

In io.confluent.kafka.schemaregistry.client.security.bearerauth.BearerAuthCredentialProviderFactory:

• Line 28 instantiates all BearerAuthCredentialProvider implementations found on the classpath during service discovery

This means GoogleCredentials is instantiated even when the provider isn't configured for use, causing failures if Google credentials aren't available on the default provider chain.

Impact

This prevents multi-cloud deployments where:

• Multiple bearer auth providers (e.g., Confluent Cloud + Google Cloud) exist on the classpath
• The application needs to dynamically choose which provider to use based on runtime configuration
• Google credentials may not be available in all environments

Real-world example: Our product Kpow provides first-class support for Google Managed Service for Apache Kafka (see docs). Until this is fixed, we'll need to maintain a separate GCP-specific build, creating unnecessary friction for our customers using Google's managed services.

Steps to reproduce

1. Add thecom.google.cloud.hosted.kafka/managed-kafka-auth-login-handler dependency to your classpath alongside the io.confluent/kafka-avro-serializer dependency
2. Configure your SerDes instance (eg io.confluent.kafka.serializers.KafkaAvroDeserializer or similar) to use a non-Google provider - eg specifying bearer.auth.credentials.source=CUSTOM | OAUTHBEARER | etc
3. Deploy to an environment without Google Cloud credentials configured
4. Observe a runtime exception constructing the SerDes class:

java.util.ServiceConfigurationError: io.confluent.kafka.schemaregistry-client.security-bearerauth.BearerAuthCredentialProvider: Provider com-google.cloud.hosted.kafka.auth.GcpBearerAuthCredentialProvider...

Caused by: java.io.IOException: Your default credentials were not found. To set up Application Default Credentials for your environment, see https://cloud.google.com/docs/authentication/external/set-up-adc.

Expected behavior

The provider should only attempt to initialize credentials when explicitly configured and used.

Proposed solution

Defer GoogleCredentials initialization from the constructor to the configure(Map<String, ?> configs) method, which is called only when Confluent's Schema Registry library has positively identified this provider based on configuration.

Considerations:

• The overloaded constructor for injecting custom credentials (used in testing) may need adjustment
• Custom credentials could potentially be passed via the configuration map instead
• This change would align with the lazy initialization pattern used by other auth providers

Workaround

Currently, users must either:

• Ensure Google credentials are always available, even when unused
• Maintain separate builds with different auth providers