ModuleGoals.md

Module Goals

M01 : Motivating Applications, Machine Learning Pipeline (Data, Models, Loss, Optimization), Backpropagation

• Online Demo
• Micrograd code for backprop, engine.py
• Colab file IO examples

Goals

• Understand the key components to set up a classification task
• Relate business problems to machine learning methods
• Understand how chain rule works
• Understand why multiclass logistic regression may not work well even for 2D data

M02 : Feedforward Networks: Nonlinearities, Convolutional Neural Networks: Convolution, Pooling

• Notebook: Pytorch Basics
• Nonlinearities visualization
• CNN forward pass visualization
• NN architecture visualization

Goals

• Get acquainted with the basics of Python
• Understand the notion of hidden layers and nonlinearities
• Convolution layer as collection of filters applied to input tensors
• Why pooling helps in reducing parameters downstream

M03 : Jumpstarting Convolutional Neural Networks: Visualization, Transfer, Practical Models (VGG, ResNet)

• Overfitting and Dropout example
• Explaniability via Captum
• Additional Links:
  • Text to Video
  • Vision Models
  • Deepseek-V3, Deekseek-R1, Deepseek-Janus-Pro and related links: 1, 2, 3 and 4.
• Project Ideas from Meta ELI5 series: 1, 2 and 3.
• Ethics
  • Data Leak

Goals

• Understand how to transfer parameters previously learned for a new task
• Know the different ways to debug a deep network
• Be aware of the different engineering tricks such as dropout, batch normalization
• Learn why image datasets can be enhanced using data augmentation
• Understand parameter-efficient fine-tuning techniques (LoRA, adapters) for pretrained models

M04 : Text and Embeddings: Introduction to NLP, Word Embeddings, Word2Vec

• Spacy
• Additional Reading:
  • Latent Dirichlet Allocation
  • CNN for sentence classification tasks. link1 and link2
  • Pytorch tutorial on using CNN for sentence classification: notebook 4
• Additional Links:
  • Notebook LM by Google
  • Controllable character video synthesis
  • Small Vision Models
• Ethics
  • EU AI Act
  • OpenAI o1 System Card

Goals

• Understand how natural language elements (such as words) are processed in an analytics workflow
• Understand the shortcomings of methods such as Naive Bayes, Latent Dirichlet Allocation
• Realize that a CNN can also be used for a NLP task (sentence classification/sentiment analysis)
• What is word2vec and how does it help in NLP tasks?

M05 : Recurrent Neural Networks and Transformers: Sequence to Sequence Learning, RNNs and LSTMs

• RNN example in Pytorch
• RNN function implementation in Pytorch
• LSTM example in Pytorch

Goals

• Know when prediction tasks can have sequential dependencies
• The RNN architecture and unfolding
• Know how LSTMs work
• Applications of 'sequential to sequential' models

M06 : Advanced NLP: Attention, BERT and Transformers, LLMs, VLMs, MLLMs

• Attention
  • Attention Is All You Need and an annotated version
  • Attention
  • Transformer Tutorial in Pytorch
  • Transformer Visualization
• Models before GPT3
  • BERT and its repository
  • Illustrated BERT
  • GPT2 Paper
  • GPT3 Repository
• Ethics
  • Paper (2024): Decline of AI Data Commons
• Code
  • SOTA Transformer Implementations including BERT and DistillBERT (for example BERT-base-uncased)
  • Open Deep Research Model
  • PEFT: Parameter-Efficient Fine-Tuning
  • Unsloth: Fast LLM Fine-tuning
• Papers
  • Llama, Llama 2, Llama 3

Goals

• Be able to explain self-attention and how it differs from simpler attention mechanisms seen in sequence to sequence models
• Be able to reason about keys, values and queries in self-attention
• Be able to recall the key characteristics of BERT and how pre-trained models can be used for NLP tasks.
• Understand the architecture and training paradigm of Large Language Models (LLMs)
• Know the basics of LLM fine-tuning using parameter-efficient methods (LoRA, PEFT)
• Be aware of vision-language models (VLMs) and multimodal LLMs (MLLMs)

M07 : Unsupervised Deep Learning: Variational Autoencoders, Diffusion Models, Generative Adversarial Networks

• VAE in Pytorch from Pytorch examples repository
• Diffusion Models
  • Introduction
  • Slides
• Notebook: GAN example on CelebFaces Attributes (CelebA) Dataset (dataset)
• VAE in Pytorch from Pytorch examples repository
• GAN Demo by Google 2020
• GAN animation

Goals

• Meaning of generative modeling
• What are variational autoencoders (VAEs) and where can they be used?
• The intuition behind generative adversarial networks (GANs)
• Differences between GANs and VAEs

M08 : Online Learning: A/B Testing, Multi-armed Bandits, Contextual Bandits

• Bandit Implementations in Python: SMPyBandits
• A blog post on Bandits by Lilian Weng

Goals

• What is online learning? How is it different from supervised learning?
• Relation between forecasting and decision making
• The multi armed bandit problem and solutions
• Contextual bandits

M09 : Reinforcement Learning: Policies, State-Action Value Functions, Bellman Equations, Q Learning

• Openai Gym
• RL in Pytorch from Pytorch examples repository
• RL with Human Feedback (RLHF)
• RLHF for ChatGPT like Assistants
• Implementations of RL
• Flappy Bird with Q learning
• ML Katas: Cliffworld with Q learning

Goals

• What is reinforcement learning?
• Basics of Markov Decision Processes
• Policies, Value functions and how to think about these two objects
• Be able to understand the difference between Bellman Expectation Equation and Bellman Optimality Equation
• Intuitive reasoning for the Q-Learning update rule
• Be able to identify relationships between state value functions, state-action value functions and policies

M10 : Deep Reinforcement Learning: Function Approximation, DQN for Atari Games, DQN for Atari Games, MCTS for AlphaGo

• OpenAI Baselines for RL
• Cartpole environment with DQN

Goals

• Know the role of function approximation in Q-learning
• Be able to understand the key innovations in the DQN model
• Identify the differences between Monte Carlo tree search vs Monte Carlo rollouts
• Be able to identify key compoments of the AlphaGo (and variants such as AlphaZero) Go playing agent

M11 : AI Ethics, Fairness, Accountability, Transparency and Sustainability

• Textbooks (see Syllabus)
  • Responsible AI by Patrick Hall and Rumman Chowdhury (2022)
  • Practical Fairness by Aileen Nielsen (2020)
  • Fairness and Machine Learning: Limitations and Opportunities by Barocas, Hardt and Narayanan (2018)
  • The Framework for ML Governance by Kyle Gallatin (2021)
• Fairness Tools and Libraries
  • IBM AI Fairness 360
  • Google What-If Tool
  • Microsoft Fairlearn
• Explainability and Transparency
  • SHAP (SHapley Additive exPlanations)
  • LIME (Local Interpretable Model-agnostic Explanations)
• Sustainability
  • ML CO2 Impact Calculator
  • Green AI Paper
• Regulation and Guidelines
  • EU AI Act
  • NIST AI Risk Management Framework

Goals

• Understand the key principles of responsible AI: fairness, accountability, transparency, and ethics
• Be able to identify sources of bias in ML pipelines (data, model, deployment)
• Know how to use fairness metrics and tools to evaluate and mitigate bias in models
• Understand the importance of model interpretability and explainability for stakeholder trust
• Be aware of the environmental impact of training large models and strategies for sustainable AI
• Familiarize with regulatory frameworks (EU AI Act, NIST) governing AI deployment