expected credentials.json is different from Amazon's example

[jeroenvandijk]

Hi Marc,

Thanks a lot for Lemur. I've started using it today and it promises to be very useful. Although you have documented everything thoroughly, there was one thing that stopped me from getting lemur working.

In the readme you mention the expected structure of the credentials.json file. Unfortunately this isn't the same as the main example of Amazon mentioned here: http://aws.amazon.com/developertools/2264 . Amazon doesn't use underscores in their example.. So I had a credentials.json file with a dash that worked well with the elastic-mapreduce client, but one that didn't work with lemur. To makes things the error that I got was a non-descriptive NullPointerException somewhere in the signing process (com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68). It took me a while before I figured it was the difference in the credentials.json file.

I tested my updated credentials.json file with the elastic-mapreduce client and it seems the ruby client doesn't care about dashes or underscores. I suggest either updating the README or making lemur so that it doesn't care about the dashes either. If you agree, I could create a pull request a little later.

Cheers,
Jeroen

[mlimotte]

Good catch, Jeroen.

I suspected this would be one of the trickier aspects of making it all work, largely because Amazon has so many different formats for these credentials. That's why I added so many options for specifying them, including a fallback to explicit LEMUR_* environment variables. I'm not sure why I have underscores vs. Amazon's example... I don't think I just made that up... maybe an older version of their documentation used underscores.

Anyway, your suggestions make sense. The README should be adjusted and lemur should accept either underscore or dashes. Also, while I have a fair amount of error checking and debug info around most of this credential discovery path, I see that a there is no error checking for the necessary keys in the JSON file. I just read it in and assume it's ok. Hence, the not very helpful NPE you got. This should be fixed also.

If you'd like to take stab at it, I'd be happy to review your patch.