one queue for multiple tasks

[felipao-mx]

current implementation requires dedicated queue for each task. It could be a difficult to manage so many queues. So I propose to use a unique queue for many tasks definiton.

current implementation

TASK_1_QUEUE_URL = 'http://127.0.0.1:4000/123456789012/task1.fifo'
TASK_2_QUEUE_URL = 'http://127.0.0.1:4000/123456789012/task2.fifo'


@task(queue_url=TASK_1_QUEUE_URL, region_name='us-east-1')
async def task1(message) -> None:
    print(message)

@task(queue_url=TASK_2_QUEUE_URL, region_name='us-east-1')
async def task2(message) -> None:
    print(message)

proposed

QUEUE_URL = 'http://127.0.0.1:4000/123456789012/my_tasks.fifo'


@task(queue_url=QUEUE_URL, region_name='us-east-1')
async def task1(message) -> None:
    print(message)

@task(queue_url=QUEUE_URL, region_name='us-east-1')
async def task2(message) -> None:
    print(message)

we could distinguish each tasks by using message attributes
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Client.send_message

MessageAttributes={
        'string': {
            'StringValue': 'string',
            'BinaryValue': b'bytes',
            'StringListValues': [
                'string',
            ],
            'BinaryListValues': [
                b'bytes',
            ],
            'DataType': 'string'
        }
    },