If aws-sdk is a dependency, lambda errors

[camilleanne]

I hit an odd bug when setting up a streambot lib that requires the aws-sdk for kms, but if aws-sdk is declared in the package.json I get a Cannot find module 'xmlbuilder' error bubbling up from the aws-sdk. This occurs even after investigating the bundle.zip and finding that aws-sdk with the xmlbuilder module is installed correctly.

If aws-sdk is removed from package.json, aws-sdk still is installed into the node_modules folder in bundle.zip because of https://github.com/mapbox/streambot/blob/master/bin/bundle#L24. I still see an error in the lambda logs, but it's changed to something like cannot find index (actual log has been lost. Seems that when you change the event source it clears the logs -- logs are stored by shardId and it looks like previous logs were overwritten).

If I make sure that there is no aws-sdk installed in bundle.zip, everything works fine.

Maybe this has something to do with:

the current version of the AWS SDK is pre-installed in Lambda

and bad interactions between versions? Perhaps this case only occurs when deploying a lambda function that requires aws-sdk directly.

cc @mick @rclark

[rclark]

Yeah if we can confirm that the issue goes away when aws-sdk is not included in the bundle, then we should patch the bundling function to explicitly exclude aws-sdk from the bundle.

Its a little confusing -- I've seen this work fine in libs that don't explicitly depend on aws-sdk but one of its dependencies does, so the bundle probably does include aws-sdk though not at the top level. I'll work on seeing if I can make a little test to isolate these cases.

[rclark]

I wasn't able to reproduce the problem -- I modified the streambot-example to bundle aws-sdk 2.1.25 and with the following streambot module code:

var AWS = require('aws-sdk');
var streambot = require('streambot');

module.exports.streambot = streambot(exampleService);

function exampleService(records, callback) {
  var kms = new AWS.KMS();

  kms.listAliases({}, function(err, data) {
    if (err) streambot.log.error(err);
    else streambot.log.info(data);
    callback();
  });
}

[camilleanne]

by updating npm @2.1.6 to npm @2.9.0 I was able to get around this issue. I diffed the bundle created on 2.1.6 vs the one created with 2.9.0: https://gist.github.com/camilleanne/b95e9dd24427497e6ef7

It still remains a mystery why this made a difference since the diffs don't look particularly telling.