diff --git a/slides/01/01.md b/slides/01/01.md index 144d650..be40ed6 100644 --- a/slides/01/01.md +++ b/slides/01/01.md @@ -265,6 +265,23 @@ class: section - $∇_{→x} f(→x)$: gradient of $f$ with respect to $→x$, i.e., $\left(\frac{∂f(→x)}{∂x_1}, \frac{∂f(→x)}{∂x_2}, \ldots, \frac{∂f(→x)}{∂x_n}\right)$ +--- +# Linear Algebra Conventions & Broadcasting + +## Vector Addition + +Linear algebra distinguishes between column vectors ($N\times 1$ matrices) and row vectors ($1\times N$ matrices), often treating the addition of a row and a column as undefined. + +~~~ +However, in deep learning, vectors that are outside of multiplication are treated simply as 1D arrays. Therefore, vector + vector is valid if their lengths match. +~~~ + +## Matrix-Vector Broadcasting + +When adding a matrix and a vector, we adopt broadcasting conventions common in NumPy and PyTorch. The vector is implicitly copied to match the matrix shape. + +![w=40%,h=center](broadcasting.png) + --- section: Random Variables class: section diff --git a/slides/01/broadcasting.png b/slides/01/broadcasting.png new file mode 100644 index 0000000..be71afb Binary files /dev/null and b/slides/01/broadcasting.png differ diff --git a/slides/01/broadcasting.png.ref b/slides/01/broadcasting.png.ref new file mode 100644 index 0000000..18e7e2c --- /dev/null +++ b/slides/01/broadcasting.png.ref @@ -0,0 +1 @@ +Figure 2 of NumPy Broadcasting, https://numpy.org/doc/stable/user/basics.broadcasting.html