Add an asynchronous API

[brettstack]

This is a very common and powerful Serverless pattern. You can see an example of it implemented here https://github.com/brettstack/civ6-play-by-cloud-turn-notifier.

Acceptance Criteria:

1. Add a Lambda Function connected to the POST /sqs endpoint that stores the body of the request on the SQS Queue (see the Civ repo for implementation, though it should be simplified as much as possible)
2. Render two buttons to the UI that invoke the API endpoint POST /sqs - one should send a valid message, and the other should send a malformed message (to experiment with DLQs)
3. Add a Lambda Function connected to the SQS Queue that increments a value against the current logged in user. The Lambda Function should use the https://www.npmjs.com/package/@middy/sqs-partial-batch-failure middleware similar to the Civ repo and should throw when it detects a malformed SQS message
4. Alarms should be configured for the ApproximateAgeOfOldestMessage and NumberOfMessagesSent metrics on the DLQ.
5. The ApproximateAgeOfOldestMessage alarm should be configured as a rollback trigger Add CloudFormation Rollback Alarms/Triggers #11
6. Software architecture diagram is updated

[marconi1992]

Hi @brettstack, I guess it's using an AWS service integration to push the request body directly to the SQS queue instead of a Lambda right?

Example: https://medium.com/@pranaysankpal/aws-api-gateway-proxy-for-sqs-simple-queue-service-5b08fe18ce50

[marconi1992]

Also, If we go for using a Lambda to push the request body it's a good opportunity to validate the body size to avoid reaching the SQS limits, or we can handle the error.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-quotas.html

Probably it's an edge case. However, it's common to use async APIs for ingesting a lot of data. For those cases, we can store the request payload in an S3 bucket and push the reference as the message in SQS. It requires pulling the payload from S3 in the SQS Lambda.

We followed a similar approach in Saga when we replaced Kinesis with SQS.

Pushing to SQS:
https://github.com/wizeline/academy-saga-be/blob/dev/app/ingestion/content-webhook/data-stream/index.js

In lambda triggered by SQS
https://github.com/wizeline/academy-saga-be/blob/dev/app/ingestion/processor/data-stream/index.js

[brettstack]

Using API Gateway -> Lambda -> SQS definitely gives you more control than direct API Gateway -> SQS and I'm open to the example implementation being done in this way.

The main benefit of the direct approach is that it's more resilient. For example, adding a Lambda dependency means you're vulnerable to hitting resource limits.

My general guidance is: use the direct approach if possible; use Lambda when necessary.