Skip to content

Latest commit

 

History

History
42 lines (22 loc) · 821 Bytes

File metadata and controls

42 lines (22 loc) · 821 Bytes

self-learning records.

numpy-only, from linear regression to transformer.

python -m mtorch.test.test_matmul

路径

1,linear regression

2,logistic regression

3,mlp - backward propagation - 手写识别 -优化器与训练 - 稳定器

4,autograd

5,实践例子 - resnet(搞懂) - cnn(了解) (深入理解与巩固神经网络)

6,字符级语言模型

7,RNN / LSTM (循环神经网络) (了解,不做深入)

8,Embeddings (词嵌入)

9,Positional Encoding (位置编码)

10,Attention Mechanism

11,transformer - Layer Normalization - Residual Connections (残差连接) - Transformer Block

12,实例:GPT (Decoder-only) vs BERT (Encoder-only) 手戳gpt2

性能

numpy实现 单cpu运行

Cupy gpu

c++ 实现 gpu

能不能rust?

要不要接触cuda