Unify KartonServiceInfo objects, add 'instance_id' field to deduplicate connections from the same replica #313
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Karton uses
CLIENT NAMEto bind Karton service metadata with Redis connection. This information is used for counting active replicas and for removing inactive non-persistent queues.The problem is that in case of concurrent Redis calls, more than one connection may be used by the client. Initially I thought that this can be fixed by
single_connection_clientoption that sets a lock in connection pool instead of spawning new connections. Unfortunately, asyncio Redis client still keeps two connections opened, making replicas count value completely unreliable.This PR contains two changes:
instance_idin KartonServiceInfo object and generated during KartonBase construction (or in KartonBackend if someone uses it standalone). All connections coming from the same instance of service should have sameinstance_idinstance_idcan be passed as an argument to "hello" request in Karton Gateway, along with other KartonServiceInfo information.KartonServiceInfo.karton_versionis optional to be able to parse allCLIENT NAMEvalues to KartonServiceInfo, regardless of whether they have extended fields or not.The replica counting logic should be as follows:
KartonServiceInfo.instance_id=<uuid>: objects should be deduplicated byinstance_idand each object should be counted as a separate replica ofidentityKartonServiceInfo.instance_id=None(coming from <6.0.0): each object should be counted as a separate replica ofidentity