intersection

Intersection

This example illustrates the training of an ego agent to make an unprotected left turn at an all-way-stop intersection with traffic, using the Stable Baselines3 reinforcement-learning library. Here, the PPO algorithm is used.

The ego agent earns reward for the distance travelled per-step and is penalised for colliding with other vehicles, for going off-road, for going off-route, or for driving on road-shoulder.

This example is only meant to demonstrate the use of SB3 library with SMARTS task environments. The trained agent does not solve the task environment.

Trained agent making an uprotected left turn

Observation space

• Topdown RGB image
  • size (width x height): 112 pixels x 112 pixels
  • resolution: 50 meter / 112 pixel

observation_space = gym.spaces.Box(low=0, high=255, shape=(3, 112, 112), dtype=np.uint8)

Action space

• Throttle: [0,1]
• Brake: [0,1]
• Steering: [-1,1]

action_space = gym.spaces.Box(low=-1.0, high=1.0, shape=(3,), dtype=np.float32)

Colab

1. Try it here:

Setup

$ git clone https://github.com/huawei-noah/SMARTS.git
$ cd <path>/SMARTS/examples/rl/intersection
$ python3.7 -m venv ./.venv
$ source ./.venv/bin/activate
$ pip install --upgrade pip
$ pip install -e .

Train

1. Train

   $ cd <path>/SMARTS/examples/rl/intersection
   $ python3.7 run.py 

2. Trained model is saved into <path>/SMARTS/examples/rl/intersection/logs/<folder_name> folder.
3. Monitor the RL agent during or after the training using tensorboard

   $ cd <path>/SMARTS/examples/rl/intersection
   $ tensorboard --logdir ./logs/

Evaluate

1. Start

   $ cd <path>/SMARTS/examples/rl/intersection
   $ scl envision start --scenarios ./.venv/lib/python3.7/site-packages/smarts/scenarios &

2. Run
   • Evaluate your own model

     $ python3.7 run.py --mode=evaluate --model="./logs/<folder_name>/<model>" --head

   • Evaluate pre-trained agent

     $ curl -o ./logs/pretrained/intersection.zip --create-dirs -L https://github.com/Adaickalavan/SMARTS-zoo/raw/main/intersection-v0/PPO_5800000_steps.zip        
     $ python3.7 run.py --mode=evaluate --model="./logs/pretrained/intersection" --head

3. Go to localhost:8081 to view the simulation in Envision.

Docker

1. Train a model inside docker

   $ cd <path>/SMARTS
   $ docker build --file=./examples/rl/intersection/Dockerfile --network=host --tag=intersection .
   $ docker run --rm -it --network=host --gpus=all intersection
   (container) $ cd /src/examples/rl/intersection
   (container) $ python3.7 run.py