Cloudwatch ThrottlingExceptions when sharing resource between app instances

[oortlieb]

I have a server process using a lambdaws-ified module to do some file processing. The server needs to provide a response, so the server must wait for a response, which lambdaws seems to get by querying the cloudwatch logs.

When running multiple instances of the server, however, each instance must query the cloudwatch hose to look for responses -- this leads to ThrottlingExceptions, and as a result, responses are often not received before the timeout.

Is there a suggested way to work around this? Is lambdaws optimized for a lambda instance per server instance (ie, could I work around this by providing a separate lambda function to each server instance)? Or is the cloudwatch hose throttling a global restriction?

[slvnperron]

There's an issue open on cloudwatch-logs-hose repo about this. The cloudwatch hose is being throttled per account if I'm right, so we need to implement a back-off algorithm.

The hose is being used to capture Lambda limit errors such as timeouts. The result is get from SQS. Maybe you could try to disable the hose if you don't need such reporting?

https://github.com/mentum/lambdaws/blob/master/lib/LambdaHelper.js#L107

This will require a small change to the lib

[oortlieb]

Thanks for the clarification -- the issue with dropped responses must be happening somewhere else.