Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions slides/01/01.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,23 @@ class: section
- $∇_{→x} f(→x)$: gradient of $f$ with respect to $→x$, i.e.,
$\left(\frac{∂f(→x)}{∂x_1}, \frac{∂f(→x)}{∂x_2}, \ldots, \frac{∂f(→x)}{∂x_n}\right)$

---
# Linear Algebra Conventions & Broadcasting

## Vector Addition

Linear algebra distinguishes between column vectors ($N\times 1$ matrices) and row vectors ($1\times N$ matrices), often treating the addition of a row and a column as undefined.

~~~
However, in deep learning, vectors that are outside of multiplication are treated simply as 1D arrays. Therefore, vector + vector is valid if their lengths match.
~~~

## Matrix-Vector Broadcasting

When adding a matrix and a vector, we adopt broadcasting conventions common in NumPy and PyTorch. The vector is implicitly copied to match the matrix shape.

![w=40%,h=center](broadcasting.png)

---
section: Random Variables
class: section
Expand Down
Binary file added slides/01/broadcasting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions slides/01/broadcasting.png.ref
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Figure 2 of NumPy Broadcasting, https://numpy.org/doc/stable/user/basics.broadcasting.html