From 86dfa8d9f4142e4a6e442bc08a25ab3f94725908 Mon Sep 17 00:00:00 2001 From: Nishant Date: Thu, 9 Aug 2018 17:30:32 -0700 Subject: [PATCH 1/3] initial tutorial 11 commit --- tutorials/tutorial11_rllab_ec2.ipynb | 129 +++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 tutorials/tutorial11_rllab_ec2.ipynb diff --git a/tutorials/tutorial11_rllab_ec2.ipynb b/tutorials/tutorial11_rllab_ec2.ipynb new file mode 100644 index 00000000..9b29ca07 --- /dev/null +++ b/tutorials/tutorial11_rllab_ec2.ipynb @@ -0,0 +1,129 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Exercise 11: Running rllab experiments on EC2\n", + "\n", + "This tutorial, a complement to tutorial 10, covers the process of running rllab experients on an Amazon EC2 instance. This tutorial assumes rllab has been installed correctly ([instructions](https://rllab.readthedocs.io/en/latest/user/installation.html)). " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First, follow the [rllab cluster setup instructions](https://rllab.readthedocs.io/en/latest/user/cluster.html) with region `us-west-1`. Modify `rllab/config_personal.py` to reference the most currently Flow Docker image (at the time of this writing, `evinitsky/flow`). \n", + "\n", + "Navigate to your `flow-devel` directory and modify `Makefile.template` per the instructions in that file. \n", + "\n", + "- The variable `RLLABDIR` should be the relative path from your `flow-devel` directory to \n", + "- " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Setting up AWS\n", + "\n", + "\n", + " - Pull `openai/rllab-distributed` repository and switch to production `sharedPolicyUpdate_release` branch\n", + " ```bash\n", + " git clone https://github.com/openai/rllab-distributed.git\n", + " cd rllab-distributed\n", + " git checkout sharedPolicyUpdate_release\n", + " ```\n", + " - Follow [rllab local setup instructions](https://rllab.readthedocs.io/en/latest/user/installation.html)\n", + " - Follow [rllab cluster setup instructions](http://rllab.readthedocs.io/en/latest/user/cluster.html)\n", + " - If prompted, region = us-west-1\n", + " - Note: the current Docker image path is \"evinitsky/flow-distributed\". Your `rllab-distributed/rllab/config_personal.py` should reflect that.\n", + " - (Optional): As desired, add to `config_personal.py` files and \n", + " directories that you do not need uploaded to EC2 for every \n", + " experiment by modifying `FAST_CODE_SYNC_IGNORES`.\n", + " - Go to `Makefile.template` in `learning-traffic/flow_dev` and update\n", + " the path to your rllab root directory (no trailing slash)\n", + " - The `flow_dev` reference in the Makefile might need to be updated to `flow`\n", + " - (See note below); Run `make prepare` \n", + " - Try an example! Run any experiment from `flow_dev/examples`, change\n", + " mode to “ec2”\n", + " - You can run it locally by changing the mode to local_docker. If this isn't working, make sure to check that your local docker image is the most current image. \n", + " - Log into AWS via: https://cathywu.signin.aws.amazon.com/console\n", + " - If you don’t see the instance you just launched (give it a few \n", + " minutes) running on AWS, then make sure your region is correct (go to\n", + " top right and change to `US West (N. California)` \n", + "\n", + "## Notes\n", + "\n", + "- When we run experiments on AWS, we create a new instance for each\n", + "seed and use the Docker image I created as the VM. Built into the rllab \n", + "script for running experiments in EC2 mode, we upload the rllab root \n", + "directory to AWS. This way, the AWS instance has access to all files it\n", + " might need to successfully run the experiment. Editing that code is\n", + " pretty complicated, so Cathy and I have decided on the following \n", + " workflow:\n", + " - All code modification will happen in the learning-traffic directory\n", + " - Before each experiment, run the command `make prepare` , which will\n", + " remove the flow_dev directory in rllab root and copy\n", + " `learning-traffic/flow_dev-dev/flow_dev` into your rllab root directory\n", + " - This means if you make modifications to flow_dev in the rllab\n", + " directory, they may be lost\n", + " - Before each experiment, always make sure you have a commit to that \n", + " exact snapshot of the `flow_dev` directory. This is because you may\n", + " modify flow_dev later. When you want to run `visualizer.py` , which is\n", + " our modified version of `sim_policy.py` , AKA when you want to \n", + " create rollout plots, you need the files in `rllab/flow_dev` to match\n", + " the files that were there when you originally ran the experiment.\n", + " So when you want to create rollout plots, you will checkout the\n", + " commit that matches when you ran the experiment and run make \n", + " prepare, then you can create rollout plots in the rllab directory.\n", + " - Ping me if there are issues!\n", + "- I recommend cleaning up your rllab directory, especially if you \n", + "notice you’re uploading a large size to AWS (it will tell you how many\n", + " MB you are uploading, should be < 5 MB). The command `make clean` \n", + " removes the debug directory (since so far we hardcoded that our SUMO\n", + " files go into that directory, this should be changed in the future) \n", + " and also all XML files in rllab root directory.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 9ff877d4678b1fee442b7601ac37663882db1c75 Mon Sep 17 00:00:00 2001 From: Nishant Date: Mon, 20 Aug 2018 18:16:49 -0700 Subject: [PATCH 2/3] draft tutorial --- tutorials/tutorial11_rllab_ec2.ipynb | 98 ++++++++-------------------- 1 file changed, 28 insertions(+), 70 deletions(-) diff --git a/tutorials/tutorial11_rllab_ec2.ipynb b/tutorials/tutorial11_rllab_ec2.ipynb index 9b29ca07..3d12ec55 100644 --- a/tutorials/tutorial11_rllab_ec2.ipynb +++ b/tutorials/tutorial11_rllab_ec2.ipynb @@ -13,94 +13,48 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "First, follow the [rllab cluster setup instructions](https://rllab.readthedocs.io/en/latest/user/cluster.html) with region `us-west-1`. Modify `rllab/config_personal.py` to reference the most currently Flow Docker image (at the time of this writing, `evinitsky/flow`). \n", + "## Setting up rllab with AWS\n", + " \n", + "First, follow the [rllab cluster setup instructions](https://rllab.readthedocs.io/en/latest/user/cluster.html) with region `us-west-1`. Modify `rllab/config_personal.py` to reference the most current Flow Docker image (at the time of this writing, `evinitsky/flow`). [@fangyu is this true?]\n", "\n", - "Navigate to your `flow-devel` directory and modify `Makefile.template` per the instructions in that file. \n", - "\n", - "- The variable `RLLABDIR` should be the relative path from your `flow-devel` directory to \n", - "- " + "Navigate to your `flow` directory and modify `Makefile.template` per the instructions in that file. The variable `RLLABDIR` should be the relative path from your `flow` directory to `rllab` and should not have a backslash at the end. " ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": { "collapsed": true }, - "outputs": [], - "source": [] - }, - { - "cell_type": "markdown", - "metadata": {}, "source": [ - "# Setting up AWS\n", + "## Running an experiment\n", + "\n", + "When running AWS experiments, your entire `rllab` directory is uploaded to AWS so that the files necessary for your experiment are available to the EC2 instance. Thus, commands are included to copy over your `flow` directory to your `rllab` root directory (this is the reason for the `RLLABDIR` variable above). \n", "\n", + "- Before running an experiment, run `make prepare` from your `flow` directory.\n", + "- Ensure you have committed or otherwise tracked the state of your `flow` directory, because that instance is what will be used to run your experiment. Upon visualization, the same files will need to be used—for example, changes to environment's state-space would break the ability to run a trained policy using a different state space. Check out an old commit of your `flow` directory and run `make prepare` to use visualization tools included with rllab.\n", "\n", - " - Pull `openai/rllab-distributed` repository and switch to production `sharedPolicyUpdate_release` branch\n", - " ```bash\n", - " git clone https://github.com/openai/rllab-distributed.git\n", - " cd rllab-distributed\n", - " git checkout sharedPolicyUpdate_release\n", - " ```\n", - " - Follow [rllab local setup instructions](https://rllab.readthedocs.io/en/latest/user/installation.html)\n", - " - Follow [rllab cluster setup instructions](http://rllab.readthedocs.io/en/latest/user/cluster.html)\n", - " - If prompted, region = us-west-1\n", - " - Note: the current Docker image path is \"evinitsky/flow-distributed\". Your `rllab-distributed/rllab/config_personal.py` should reflect that.\n", - " - (Optional): As desired, add to `config_personal.py` files and \n", - " directories that you do not need uploaded to EC2 for every \n", - " experiment by modifying `FAST_CODE_SYNC_IGNORES`.\n", - " - Go to `Makefile.template` in `learning-traffic/flow_dev` and update\n", - " the path to your rllab root directory (no trailing slash)\n", - " - The `flow_dev` reference in the Makefile might need to be updated to `flow`\n", - " - (See note below); Run `make prepare` \n", - " - Try an example! Run any experiment from `flow_dev/examples`, change\n", - " mode to “ec2”\n", - " - You can run it locally by changing the mode to local_docker. If this isn't working, make sure to check that your local docker image is the most current image. \n", - " - Log into AWS via: https://cathywu.signin.aws.amazon.com/console\n", - " - If you don’t see the instance you just launched (give it a few \n", - " minutes) running on AWS, then make sure your region is correct (go to\n", - " top right and change to `US West (N. California)` \n", + "`make clean` removes the debug directory and also all XML files in rllab root directory to reduce the size of the data to upload to AWS. If you are using already-existing network files (from, say, OpenStreetMap), ensure they do not get deleted by `make clean` by storing such files elsewhere.\n", "\n", - "## Notes\n", + "Inside the experiment, change the `mode` to `ec2` (other options are `local` and `local_docker`). You should run the experiment in `local_docker` mode briefly before running the `ec2` version to ensure there are no errors, particularly with Docker image compatability\n", "\n", - "- When we run experiments on AWS, we create a new instance for each\n", - "seed and use the Docker image I created as the VM. Built into the rllab \n", - "script for running experiments in EC2 mode, we upload the rllab root \n", - "directory to AWS. This way, the AWS instance has access to all files it\n", - " might need to successfully run the experiment. Editing that code is\n", - " pretty complicated, so Cathy and I have decided on the following \n", - " workflow:\n", - " - All code modification will happen in the learning-traffic directory\n", - " - Before each experiment, run the command `make prepare` , which will\n", - " remove the flow_dev directory in rllab root and copy\n", - " `learning-traffic/flow_dev-dev/flow_dev` into your rllab root directory\n", - " - This means if you make modifications to flow_dev in the rllab\n", - " directory, they may be lost\n", - " - Before each experiment, always make sure you have a commit to that \n", - " exact snapshot of the `flow_dev` directory. This is because you may\n", - " modify flow_dev later. When you want to run `visualizer.py` , which is\n", - " our modified version of `sim_policy.py` , AKA when you want to \n", - " create rollout plots, you need the files in `rllab/flow_dev` to match\n", - " the files that were there when you originally ran the experiment.\n", - " So when you want to create rollout plots, you will checkout the\n", - " commit that matches when you ran the experiment and run make \n", - " prepare, then you can create rollout plots in the rllab directory.\n", - " - Ping me if there are issues!\n", - "- I recommend cleaning up your rllab directory, especially if you \n", - "notice you’re uploading a large size to AWS (it will tell you how many\n", - " MB you are uploading, should be < 5 MB). The command `make clean` \n", - " removes the debug directory (since so far we hardcoded that our SUMO\n", - " files go into that directory, this should be changed in the future) \n", - " and also all XML files in rllab root directory.\n" + "After running `python example.py` once the `mode` of `example.py` is `ec2`, you should see your experiment running on AWS." ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": { "collapsed": true }, + "source": [ + "## Fetching Results\n", + "\n", + "To get the results of your AWS experiments, navigate to your `rllab` directory and run `python scripts/sync_s3.py`. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, "outputs": [], "source": [] } @@ -122,6 +76,10 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" + }, + "widgets": { + "state": {}, + "version": "1.1.2" } }, "nbformat": 4, From e9f7ea00725f1a171f402720b51b8affa11fdf13 Mon Sep 17 00:00:00 2001 From: Nishant Date: Mon, 20 Aug 2018 18:20:10 -0700 Subject: [PATCH 3/3] result dir --- tutorials/tutorial11_rllab_ec2.ipynb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tutorials/tutorial11_rllab_ec2.ipynb b/tutorials/tutorial11_rllab_ec2.ipynb index 3d12ec55..86f61bd5 100644 --- a/tutorials/tutorial11_rllab_ec2.ipynb +++ b/tutorials/tutorial11_rllab_ec2.ipynb @@ -48,7 +48,8 @@ "source": [ "## Fetching Results\n", "\n", - "To get the results of your AWS experiments, navigate to your `rllab` directory and run `python scripts/sync_s3.py`. " + "- To get the results of your AWS experiments, navigate to your `rllab` directory and run `python scripts/sync_s3.py`. \n", + "- Your experiment results will be in `data/s3` in your `rllab` directory." ] }, {