Help with Documentation

[bfleming-ciena]

This seems like it could be very useful, but I'm having trouble understanding how to setup the sueprvisor container. First, it is great to provide a docker image, to save on all the details a new user like me isn't interested in.

But I don't know what kind of setup I need once my container is up. I end up getting

Traceback (most recent call last):
File "/usr/local/bin/minotaur", line 98, in
Minotaur().deploy()
File "/usr/local/bin/minotaur", line 87, in deploy
exec("{0}.main()".format(sys.argv[3]))
File "", line 1, in
File "/deploy/labs/kafka/kafka.py", line 53, in main
str(args.num_nodes), args.instance_type, args.zk_version, args.kafka_url)
File "/deploy/labs/kafka/kafka.py", line 23, in init
vpc_id = self.get_vpc(environment).id

[okushchenko]

Hi! Thanks for trying out minotaur.
Looks like you have not deployed infrastructure components yet. So first of all you may want to deploy infrastructure(vpc, subnets, nat - these kind of stuff). You can type the following command to see some help:
minotaur infrastructure deploy all -h
Resulting command must be something like:
minotaur infrastructure deploy all -e test -r us-east-1 -z us-east-1a
After infrastructure is deployed you can deploy any lab you want.

[bfleming-ciena]

Definately making more sense now with needing the infrastructure components. I am getting an error on the public VPC CIDR address. Any ideas?

minotaur infrastructure deploy all -e test -r us-east-1 -z us-east-1a -i m1.small -c 10.0.0.0/24

The CIDR '10.0.2.0/27' is invalid.

thanks!

[okushchenko]

Recommended subnetting scheme is as follows: for each vpc of, for example, 10.0.0.0/21 there is 2 public 10.0.0.0/23, 10.0.0.4/23 subnets, 2 private 10.0.0.2/24, 10.0.0.6/24 subnets and 2 reserved 10.0.0.3/24, 10.0.0.7/24 subnets. So there is 3 subnets(public, private and reserved) per one availability zone.
tl;dr: just use /21 subnet mask for vpc deployment(10.0.0.0/21 or 10.0.8.0/21 and so on).

[bfleming-ciena]

Thanks. I'm trying to do this will the all option. The /21 subnet mask worked.

The nat creation failed next.

cancelled
08:30:16 UTC-0800 CREATE_FAILED AWS::IAM::InstanceProfile InstanceProfile The role with name None cannot be found.

[okushchenko]

Thanks for your patience, I will try to guide you further through deployment procedure.
Looks like you have not created iam roles, they must be created first of all.
Do the following:
minotaur infrastructure deploy iampolicies

[bfleming-ciena]

Thanks for the help. Sorry if I'm just not following the instructions correctly.

Anything else prior to that command?

minotaur infrastructure deploy iampolicies

errors with:

File "/deploy/infrastructure/aws/infrastructure.py", line 73, in deploy
notification_arns=self.notification_arns, disable_rollback=True, capabilities=["CAPABILITY_IAM"])
AttributeError: 'Iampolicies' object has no attribute 'notification_arns'

[okushchenko]

Ohh, it's a bug, thanks. I've done quick patch for it - please use my development fork(https://github.com/alexgear/minotaur/tree/hadoop) to get updated version. Make sure to use hadoop branch - it is where patch was dropped.

[okushchenko]

Also don't forget to populate this directory with json file where your credentials will be stored. You can read more about this procedure in bastion readme.

[bfleming-ciena]

I had to create a 'trusted' group and an 'administrators' group to get the iampolicies stack to work. I just made these two with full admin privileges to get past that part. Let's see how the deploy all goes this time.

I'm confused about this bastion part. It's just an ssh server in the public sub-domain. I have my pem file, but I don't have a public key. Where do I get that?

[okushchenko]

You can generate public key from your privatekey.pem using following command:
openssl rsa -in privkey.pem -pubout > key.pub
Bastion helps you to ssh into instances which don't have public IP address.

[bfleming-ciena]

Closer. The nat creation still partially completes.

11:47:49 UTC-0800 CREATE_FAILED AWS::CloudFormation::Stack nat-test-us-east-1-us-east-1a The following resource(s) failed to create: [WaitForInstance].
11:47:47 UTC-0800 CREATE_FAILED AWS::CloudFormation::WaitCondition WaitForInstance WaitCondition timed out. Received 0 conditions when expecting 1

[okushchenko]

May you write here the content of /var/log/user-data.log file located on nat instance. If it's too long - write me an email.

[bfleming-ciena]

Error indicated there wasn't a 0.0.0.0/0 route in my route table. Trying to work around it, I added it manually. I think I'm getting further now. Lets see!

[okushchenko]

Awesome! Route table entry with destination 0.0.0.0/0 must point to the network interface of the nat instance and must be automaticly created by user-data script. But to investigate your issue I really need the content of /var/log/user-data.log file located on nat instance, please send it to me.

[bfleming-ciena]

Ok, I wanted to start over to better test because I was still having some trouble.

Ran this:
minotaur infrastructure deploy all -e test -r us-east-1 -z us-east-1a -i m1.small -c 10.0.0.0/21

It hangs on the nat creation. Here is the /var/log/user-data.log file

+ export AWS_DEFAULT_REGION=us-east-1
+ AWS_DEFAULT_REGION=us-east-1
+ ROUTE_TABLE_ID=rtb-e8473a8d
+ INSTANCE_WAIT_HANDLE_URL='https://cloudfor...'
++ curl http://169.254.169.254/latest/meta-data/instance-id
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    10  100    10    0     0   1502      0 --:--:-- --:--:-- --:--:--  1666
+ INSTANCE_ID=i-163cc7e7
++ aws ec2 describe-network-interfaces --filters '{"Name":"attachment.instance-id", "Values":["i-163cc7e7"]}'
+ DESCRIBE_INTERFACES_RESPONSE='{
    "NetworkInterfaces": [
        {
            "Status": "in-use", 
            "MacAddress": "12:4b:e4:7f:a5:72", 
            "SourceDestCheck": true, 
            "VpcId": "vpc-4d224e28", 
            "Description": null, 
            "Association": {
                "PublicIp": "5...", 
                "IpOwnerId": "amazon"
            }, 
            "NetworkInterfaceId": "eni-04548229", 
            "PrivateIpAddresses": [
                {
                    "Association": {
                        "PublicIp": "5...", 
                        "IpOwnerId": "amazon"
                    }, 
                    "Primary": true, 
                    "PrivateIpAddress": "10.0.2.226"
                }
            ], 
            "RequesterManaged": false, 
            "AvailabilityZone": "us-east-1a", 
            "Attachment": {
                "Status": "attached", 
                "DeviceIndex": 0, 
                "AttachTime": "2015-01-16T00:09:50.000Z", 
                "InstanceId": "i-163cc7e7", 
                "DeleteOnTermination": true, 
                "AttachmentId": "eni-attach-cb2a29b6", 
                "InstanceOwnerId": "876410254869"
            }, 
            "Groups": [
                {
                    "GroupName": "nat-test-us-east-1-us-east-1a-SecurityGroup-1QGV5HGUTBZ71", 
                    "GroupId": "sg-a011c5c4"
                }
            ], 
            "SubnetId": "subnet-209e480b", 
            "OwnerId": "876410254869", 
            "TagSet": [], 
            "PrivateIpAddress": "10.0.2.226"
        }
    ]
}'
+ yum -y install jq
Loaded plugins: priorities, update-motd, upgrade-helper
Resolving Dependencies
--> Running transaction check
---> Package jq.x86_64 0:1.4-1.0.amzn1 will be installed
--> Processing Dependency: jq-libs(x86-64) = 1.4-1.0.amzn1 for package: jq-1.4-1.0.amzn1.x86_64
--> Processing Dependency: libjq.so.1()(64bit) for package: jq-1.4-1.0.amzn1.x86_64
--> Running transaction check
---> Package jq-libs.x86_64 0:1.4-1.0.amzn1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
 Package         Arch           Version                 Repository         Size
================================================================================
Installing:
 jq              x86_64         1.4-1.0.amzn1           amzn-main          28 k
Installing for dependencies:
 jq-libs         x86_64         1.4-1.0.amzn1           amzn-main          90 k
Transaction Summary
================================================================================
Install  1 Package (+1 Dependent package)
Total download size: 118 k
Installed size: 238 k
Downloading packages:
--------------------------------------------------------------------------------
Total                                           553 kB/s | 118 kB     00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : jq-libs-1.4-1.0.amzn1.x86_64                                 1/2 
  Installing : jq-1.4-1.0.amzn1.x86_64                                      2/2 
  Verifying  : jq-libs-1.4-1.0.amzn1.x86_64                                 1/2 
  Verifying  : jq-1.4-1.0.amzn1.x86_64                                      2/2 
Installed:
  jq.x86_64 0:1.4-1.0.amzn1                                                     
Dependency Installed:
  jq-libs.x86_64 0:1.4-1.0.amzn1                                                
Complete!
++ jq --raw-output '.["NetworkInterfaces"][0]["NetworkInterfaceId"]'
++ echo '{
    "NetworkInterfaces": [
        {
            "Status": "in-use", 
            "MacAddress": "12:4b:e4:7f:a5:72", 
            "SourceDestCheck": true, 
            "VpcId": "vpc-4d224e28", 
            "Description": null, 
            "Association": {
                "PublicIp": "5...", 
                "IpOwnerId": "amazon"
            }, 
            "NetworkInterfaceId": "eni-04548229", 
            "PrivateIpAddresses": [
                {
                    "Association": {
                        "PublicIp": "5...", 
                        "IpOwnerId": "amazon"
                    }, 
                    "Primary": true, 
                    "PrivateIpAddress": "10.0.2.226"
                }
            ], 
            "RequesterManaged": false, 
            "AvailabilityZone": "us-east-1a", 
            "Attachment": {
                "Status": "attached", 
                "DeviceIndex": 0, 
                "AttachTime": "2015-01-16T00:09:50.000Z", 
                "InstanceId": "i-163cc7e7", 
                "DeleteOnTermination": true, 
                "AttachmentId": "eni-attach-cb2a29b6", 
                "InstanceOwnerId": "876410254869"
            }, 
            "Groups": [
                {
                    "GroupName": "nat-test-us-east-1-us-east-1a-SecurityGroup-1QGV5HGUTBZ71", 
                    "GroupId": "sg-a011c5c4"
                }
            ], 
            "SubnetId": "subnet-209e480b", 
            "OwnerId": "876410254869", 
            "TagSet": [], 
            "PrivateIpAddress": "10.0.2.226"
        }
    ]
}'
+ NETWORK_INTERFACE_ID=eni-04548229
++ aws ec2 describe-route-tables --filters '{"Name":"route-table-id", "Values":["rtb-e8473a8d"]}'
+ DESCRIBE_ROUTE_TABLES_RESPONSE='{
    "RouteTables": [
        {
            "Associations": [
                {
                    "SubnetId": "subnet-d39147f8", 
                    "RouteTableAssociationId": "rtbassoc-924010f7", 
                    "Main": false, 
                    "RouteTableId": "rtb-e8473a8d"
                }
            ], 
            "RouteTableId": "rtb-e8473a8d", 
            "VpcId": "vpc-4d224e28", 
            "PropagatingVgws": [], 
            "Tags": [
                {
                    "Value": "RouteTable", 
                    "Key": "aws:cloudformation:logical-id"
                }, 
                {
                    "Value": "arn:aws:cloudformation:us-east-1:876410254869:stack/subnet-test-us-east-1-us-east-1a-private/6933a810-9d13-11e4-b6d3-5001b491380a", 
                    "Key": "aws:cloudformation:stack-id"
                }, 
                {
                    "Value": "subnet-test-us-east-1-us-east-1a-private", 
                    "Key": "aws:cloudformation:stack-name"
                }
            ], 
            "Routes": [
                {
                    "GatewayId": "local", 
                    "DestinationCidrBlock": "10.0.0.0/21", 
                    "State": "active"
                }
            ]
        }
    ]
}'
+ [[ {
    "RouteTables": [
        {
            "Associations": [
                {
                    "SubnetId": "subnet-d39147f8", 
                    "RouteTableAssociationId": "rtbassoc-924010f7", 
                    "Main": false, 
                    "RouteTableId": "rtb-e8473a8d"
                }
            ], 
            "RouteTableId": "rtb-e8473a8d", 
            "VpcId": "vpc-4d224e28", 
            "PropagatingVgws": [], 
            "Tags": [
                {
                    "Value": "RouteTable", 
                    "Key": "aws:cloudformation:logical-id"
                }, 
                {
                    "Value": "arn:aws:cloudformation:us-east-1:876410254869:stack/subnet-test-us-east-1-us-east-1a-private/6933a810-9d13-11e4-b6d3-5001b491380a", 
                    "Key": "aws:cloudformation:stack-id"
                }, 
                {
                    "Value": "subnet-test-us-east-1-us-east-1a-private", 
                    "Key": "aws:cloudformation:stack-name"
                }
            ], 
            "Routes": [
                {
                    "GatewayId": "local", 
                    "DestinationCidrBlock": "10.0.0.0/21", 
                    "State": "active"
                }
            ]
        }
    ]
} == *\0\.\0\.\0\.\0* ]]
+ aws ec2 delete-route --route-table-id rtb-e8473a8d --destination-cidr-block 0.0.0.0/0
A client error (InvalidRoute.NotFound) occurred: no route with destination-cidr-block 0.0.0.0/0 in route table rtb-e8473a8d