This project uses SocialGAN to synthesize edge-case driving scenarios for training self-driving cars. We use Social GAN to jointly forecast trajectories of traffic participants (cars) capturing socially relevant group behavior in traffic. The original SocialGAN paper can be found here: Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
CarGAN has four components: i) the original SocialGAN component uses recurrent sequence-to-sequence model observes motion histories and predicts future behavior, using a novel pooling mechanism to aggregate information across people, ii) a CNN based feature extractor from LiDAR data and road marking and iii) attention models that decompose the car behaviors into social context and physical context and iv) a second discriminator that verifies that the synthesized anomalous trajectories are physically correct but socially inappropriate or even adversarial.