-
Notifications
You must be signed in to change notification settings - Fork 11
Open
Labels
featurenew featuresnew features
Description
Describe the feature
At present, the concept of sharding is a logical concept, which only exists in the Client for parallel consumption. As a result, each Reader needs to pull the full binlog, and there is a visible performance upper limit for pulling and storing (cannot scale linearly)
- From logical sharding to physical sharding, the concept of partition is introduced, each Reader does not need to pull all binlogs, reducing storage and processing pressure, and at the same time supporting parallel processing of data by Reader (each Reader processes a part of the partition), improving ptubes Scalability
- The data of each partition can be subscribed separately to reduce the data backtracking when the service is duplicated
- Each partition has a separate writing thread, effectively breaking through the performance bottleneck of single-threaded sequential writing
Metadata
Metadata
Assignees
Labels
featurenew featuresnew features