Skip to content

Fix FIFO group locking behaviour#294

Open
Thomasvdam wants to merge 2 commits intoAdmiral-Piett:masterfrom
Thomasvdam:fix/fifo-group-locking
Open

Fix FIFO group locking behaviour#294
Thomasvdam wants to merge 2 commits intoAdmiral-Piett:masterfrom
Thomasvdam:fix/fifo-group-locking

Conversation

@Thomasvdam
Copy link
Contributor

@Thomasvdam Thomasvdam commented Mar 15, 2024

First of all thank you for this amazing software, it's already been incredibly helpful! While using it I noticed that my FIFO batches were always only a single message and after some investigation I noticed it was due to a small bug in the implementation.

I added a (perhaps too verbose) test to verify that my suspicion was correct and that the fix actually worked but that uncovered a bunch of other bugs in the FIFO group locking behaviour so the PR became a little bigger than intended.

This is my first serious Golang contribution so my apologies if it's not idiomatic or performant Go.

Edit): I found a small bug where message attributes were not set when receiving a batch of messages. I hope you don't mind that I added it to this PR, but if that's a problem I can also make a separate PR for just those changes.

Explanation of changes

Previously a FIFO queue was only able to return 1 message per request as it locked the group right after encountering the first message. Group unlocking had a similar problem where any deleted message in a group would unlock the whole group, even if there were pending messages in that group.

In addition messages could receive a receipt/timeout despite not being served if they were part of a locked group. By moving the receipt assignment to come after the group lock check this should no longer occur.

SendMessageBatch did not extract message attributes. I added an expectation to an existing test to verify it indeed didn't work and updated the implementation to extract message attributes for all sendEntries.

Related issues

I think this should close #212

Previously a FIFO queue was only able to return 1 message per request as
it locked the group right after encountering the first message. Group
unlocking had a similar problem where any deleted message in a group
would unlock the whole group, even if there were pending messages in
that group.
@Thomasvdam Thomasvdam force-pushed the fix/fifo-group-locking branch from 7e36d97 to 5f2f8dd Compare March 19, 2024 19:35
@Admiral-Piett
Copy link
Owner

@Thomasvdam Thanks for taking a look at this. Overall the goal of the change looks good to me, but we're in the middle of a large scale feature. I can't let this in since it'll cause more deployment headaches than we can bare there right now. I'm sorry. Once we're finished with the JSON work, let's regroup on this and see how this fits in with whatever comes out of that.

Thanks again for supporting us here - I really appreciate it.

@Thomasvdam
Copy link
Contributor Author

No problem at all! I’ll keep an eye out for the JSON update 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQS batch not working.

2 participants