Potential feature risk check and protection

[hangc0276]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Pulsar introduced a lot of features, but only the most commonly used features have been tested in large-scale production environments. For those features that we focus on developing but only used by relatively few users, such as topic compaction, delayed messages, geo-replication, load balance, etc, we need to enhance overall control over the quality.

We need to check and test each feature to make sure it is mature. All the features can be found here

For each feature, we need to complete the following check

• List the limitation of the feature, for example, this feature won't work well in which case
• Check if we have protection logic in this feature. If users use this feature in an unexpected way, we need to ensure it won't impact the Pulsar service, including the Pulsar broker and BookKeeper
• Check if we configured alerts and dashboard
• Run the feature in the Chaos test cluster
• Run a stress test for the feature, and generate a report to show the performance limitation under specific resources.