Implement a system wide subscription system that allows more efficient subscribing of streams to avoid duplicate traffic in most cases

[daviddawson]

Currently every subscription is independent.

If 10 services are subscribed to the same stream, they will all be sent the data, leading to 10 times as much traffic.

Since most of them will remain in lockstep most of the time, a single subscription could be used, most of the time, distributing the events out as they are received.

The only time this will not be the case is when a service needs to perform a full or partial replay of a stream. Eg, If a recovery is required or a view is being upgraded.

In this case, a separate subscription should be created, then when the subscription approaches the current main subcription, it should be shut down and the service then start taking events from the main subscription.