[feat][client] PIP-234: Support shared resources in AuthenticationOAuth2 to reduce thread usage

[3pacccccc]

fix: #24795

🔗 Related links

• issue: [feat] Authentication implementations such as AuthenticationOAuth2 should support sharing thread pools or DNS cache #24795
• comments: [feat][client] oauth2 trustcerts file and timeouts #24944 (comment)

Motivation

Currently, each instance of AuthenticationOAuth2 creates its own HTTP client with dedicated thread pools and DNS resolvers. In applications with multiple Pulsar clients or authentication providers, this leads to excessive thread creation and inefficient resource utilization. According to issues #24795 and #24944, there's a need for authentication implementations to share resources like thread pools to reduce system overhead.

With this PR, the OAuth2 authentication flow is enhanced as follows:
1.Resource Sharing at Client Initialization:

• When a PulsarClient is created, it builds an AuthenticationInitContext containing shared resources
• This context includes the client's main EventLoopGroup, Timer, and NameResolver

2.Authentication Provider Initialization:

• The Authentication.start(AuthenticationInitContext) method is called with the context
• Authentication providers can now retrieve shared services via context.getService(Class<T>)

3.Simplified HTTP Client Creation:

• Authentication implementations can now use AuthenticationHttpClientFactory.createHttpClient() to obtain pre-configured HTTP clients
• The factory automatically leverages shared resources from the AuthenticationInitContext when available
• Developers only need to provide custom configuration parameters (timeouts, SSL certificates) if needed

4.Backward-Compatible Factory Methods:

• When not provided, the factory creates optimized HTTP clients using shared resources

Modifications

1.New API Interface:

• Introduced AuthenticationInitContext interface providing access to shared services

• Added start(AuthenticationInitContext) method to Authentication interface

2.Implementation:

• Created AuthenticationInitContextImpl implementing the context interface

• Modified PulsarClientImpl to build and pass context during authentication initialization

• Updated AuthenticationOAuth2 to use shared HTTP client resources via the context

3.HTTP Client Factory:

• Created AuthenticationHttpClientFactory for building HTTP clients with shared resources

• Added AuthenticationHttpClientConfig for HTTP client configuration

• Modified OAuth2 flow classes to accept pre-configured HTTP clients and DNS resolvers

4.Factory Updates:

• Enhanced AuthenticationFactoryOAuth2.ClientCredentialsBuilder to support custom HTTP clients and DNS resolvers

• Updated factory methods to leverage shared resources when available

Verifying this change

• Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: 3pacccccc#35

[lhotari]

I think that there's a need to revisit the original design expressed in#24795. The comment in #24944 (comment) contains some details of the direction:

For authentication plugins, one possibility would be that there would be a way to directly get a HTTP client with given configuration (such as trustCertsFilePath) that is sufficiently abstracted so that there's no need to couple to AsyncHttpClient in authentication plugin implementations.

One of the problems in exposing Netty and AsyncHttpClient classes in the public interfaces is that the classes are different when using a shaded Pulsar client.
That is why there should be a sufficient abstraction for the HTTP client.

For Pulsar internal authentication plugins, it wouldn't be a problem to support the solution in this PR since the classes are also shaded in the shaded Pulsar client and there wouldn't be a problem with unshaded Netty and shaded Netty classes, for example. However, it might be work trying to find a more stable abstraction that expressed in the original design.