Demystifying Generative AI

Refer to the following source: O'Reilly course.

Please note that many of these code examples need an OpenAI account to work. It is recommended to buy at least a dollar's worth of tokens. https://platform.openai.com/docs/overview

Linear Regression: GD vs. SGD

This repo includes two small linear regression examples that illustrate different optimization strategies:

• 10_linear_regression_gd.py: Batch Gradient Descent (uses all data each step)
• 11_linear_regression_sgd.py: Stochastic Gradient Descent (uses 1 sample per step)

When to use which

• Batch GD: More stable updates and smooth convergence; good for small/medium datasets that fit in memory. Each step costs O(n).
• SGD: Much cheaper per step (O(1) with single sample), introduces noise that can help escape shallow minima; ideal for large/streaming datasets. May require more steps and careful tuning.
• Mini-batch GD: Middle ground (sample_size > 1). Balances stability and speed; commonly used in practice.

Practical tips

• Learning rate (L): Too large can diverge; too small slows training.
• Epochs: Both scripts default to 1,000,000 iterations for demonstration. Reduce for quick runs.
• Reproducibility: Both scripts set a random seed (np.random.seed(42)).
• Outputs: Each script prints progress and saves a plot (linear_regression_gd.svg or linear_regression_sgd.svg).

Run the examples

python 10_linear_regression_gd.py
python 11_linear_regression_sgd.py

Tip: For faster feedback, try setting epochs = 50_000 (or even lower) and/or increasing L slightly, then adjust as needed to maintain stable convergence.

Script Overview

Quick purposes for the included Python scripts:

• 0_no_date.py: Minimal chat example that asks the model for today's date.
• 1_haiku.py: Generates a short haiku about AI via chat completions.
• 2_basic_chat_agent.py: Simple stateful chat agent with history trimming.
• 3_food_to_grams.py: Converts food measurements (e.g., tbsp) into grams; JSON-first parsing.
• 4_antiscammer.py: Educational multi-agent demo (scam detector + time-waster responder).
• 5_simple_addition_api.py: Function-calling demo to add two numbers via a Python function.
• 6_weather_lookup.py: Weather agent that calls OpenWeatherMap via function calling.
• 7_workout_buddy.py: Kettlebell workout tracker using function calls to track sets.
• 8_rag_demonstration.py: Minimal RAG pipeline with embedding cache and retrieval.
• 10_linear_regression_gd.py: Linear regression trained with batch gradient descent.
• 11_linear_regression_sgd.py: Linear regression trained with stochastic gradient descent.
• linear_regression_gd_plot.py: 3D loss surface + GD optimization path visualization.
• linear_regression_sgd_plot.py: 3D loss surface with GD and SGD path comparison.
• exercise.py: Template for a polite agent; placeholders to be customized.
• exercise_answer.py: Completed polite agent CLI with improved prompt and UX.