Improvements to Foundation Models lesson

[sandeep-ps]

[minump]

Changes suggested:

Chapter 16:

• 1. Section 16.4, we have mentioned several foundation models. It would be very helpful to provide a brief description on each, like CLIP, BERT, Llama, Sora, Gemini, GPT3, GPT4, GPT4o, DALL-E.

• 2. It would be helpful to introduce a bit more (in layman language) about auto-regressive models and non-regressive models and their differences.

• 3. “GPT-3 and CLIP models utilize transformers as the underlying architecture.” What about other models?

• 4. This chapter introduces many very useful models. While I am reading 16.5.2.3 Diffusion models and others, I think it will be beneficial to provide some examples on how these models can be used to support Arctic research (or geoscience research in general). Can you cite some papers and give some examples here?

• a. What can we use diffusion models for?

• b. How can Arctic research benefits from diffusion models?

• 5. “Figure: Fig : Training a diffusion model. Image source : Lilweng’s blog”

• a. Not sure how to read or understand its process. Can you add to the Figure title to briefly describe?

• 6. Format consistency: for some figures, we have numbers, and some are not. Some contains the title from the original post/paper, and our added titles.

• 7. Figure 16.1 SAM: I wonder if we can add a little more description to the figure

Chapter 17.

• 1. The example of SAM are very interesting. I wonder if we can add an example using RTS images to show their segmentation results (in addition to the glacier results used).

• 2. Section 17.4: Session hands-on code in github.com/ncsa/cyber2a-workshop

• a. Can we push this to the Cyber2A repository?

• 3. For the RAG section, can we add a brief description at the beginning of the section on what problem are we trying to solve here? I think it will give a big picture view about the problem/use case.

• 4. I almost feel that this RAG/LLM and SAM can be separated into two chapters, one focusing on computer vision applications, and the other on NLP.

Chapter 0.

• 1. Add all package requirements to chapter 0.