Major refactor to support new architectures

README.md

@@ -70,9 +70,10 @@ from open_flamingo import create_model_and_transforms
 model, image_processor, tokenizer = create_model_and_transforms(
     clip_vision_encoder_path="ViT-L-14",
     clip_vision_encoder_pretrained="openai",
-    lang_encoder_path="anas-awadalla/mpt-1b-redpajama-200b",
+    lang_model_path="anas-awadalla/mpt-1b-redpajama-200b",
     tokenizer_path="anas-awadalla/mpt-1b-redpajama-200b",
     cross_attn_every_n_layers=1,
+    model_family="flamingo",
     cache_dir="PATH/TO/CACHE/DIR"  # Defaults to ~/.cache
 )
 ```
@@ -226,7 +227,7 @@ OpenFlamingo is developed by:
 The team is primarily from the University of Washington, Stanford, AI2, UCSB, and Google.
 
 # Acknowledgments
-This code is based on Lucidrains' [flamingo implementation](https://github.com/lucidrains/flamingo-pytorch) and David Hansmair's [flamingo-mini repo](https://github.com/dhansmair/flamingo-mini). Thank you for making your code public! We also thank the [OpenCLIP](https://github.com/mlfoundations/open_clip) team as we use their data loading code and take inspiration from their library design.
+This code is based on Lucidrains' [flamingo implementation](https://github.com/lucidrains/flamingo-pytorch) and David Hansmair's [flamingo-mini repo](https://github.com/dhansmair/flamingo-mini). Thank you for making your code public! We also thank the [OpenCLIP](https://github.com/mlfoundations/open_clip) and [OpenLM](https://github.com/mlfoundations/open_lm) team as we use their data loading/z-loss code and take inspiration from their library design. 
 
 We would also like to thank [Jean-Baptiste Alayrac](https://www.jbalayrac.com) and [Antoine Miech](https://antoine77340.github.io) for their advice, [Rohan Taori](https://www.rohantaori.com/), [Nicholas Schiefer](https://nicholasschiefer.com/), [Deep Ganguli](https://hai.stanford.edu/people/deep-ganguli), [Thomas Liao](https://thomasliao.com/), [Tatsunori Hashimoto](https://thashim.github.io/), and [Nicholas Carlini](https://nicholas.carlini.com/) for their help with assessing the safety risks of our release, and to [Stability AI](https://stability.ai) for providing us with compute resources to train these models.