Skip to content

Commit d799c46

Browse files
Big-TRexBig-TRex
committed
Site updated: 2025-06-30 11:44:43
1 parent 8f606e8 commit d799c46

File tree

20 files changed

+1515
-734
lines changed

20 files changed

+1515
-734
lines changed

‎2025/06/03/hello-world/index.html‎

Lines changed: 2 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -292,7 +292,7 @@ <h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerl
292292
<div class="site-state-item site-state-posts">
293293
<a href="/LLM-Blog/archives/">
294294

295-
<span class="site-state-item-count">4</span>
295+
<span class="site-state-item-count">5</span>
296296
<span class="site-state-item-name">日志</span>
297297
</a>
298298
</div>
@@ -303,7 +303,7 @@ <h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerl
303303
<span class="site-state-item-name">分类</span></a>
304304
</div>
305305
<div class="site-state-item site-state-tags">
306-
<span class="site-state-item-count">5</span>
306+
<span class="site-state-item-count">7</span>
307307
<span class="site-state-item-name">标签</span>
308308
</div>
309309
</nav>
@@ -391,59 +391,6 @@ <h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerl
391391

392392

393393

394-
395-
396-
<script>
397-
if (typeof MathJax === 'undefined') {
398-
window.MathJax = {
399-
loader: {
400-
source: {
401-
'[tex]/amsCd': '[tex]/amscd',
402-
'[tex]/AMScd': '[tex]/amscd'
403-
}
404-
},
405-
tex: {
406-
inlineMath: {'[+]': [['$', '$']]},
407-
tags: 'ams'
408-
},
409-
options: {
410-
renderActions: {
411-
findScript: [10, doc => {
412-
document.querySelectorAll('script[type^="math/tex"]').forEach(node => {
413-
const display = !!node.type.match(/; *mode=display/);
414-
const math = new doc.options.MathItem(node.textContent, doc.inputJax[0], display);
415-
const text = document.createTextNode('');
416-
node.parentNode.replaceChild(text, node);
417-
math.start = {node: text, delim: '', n: 0};
418-
math.end = {node: text, delim: '', n: 0};
419-
doc.math.push(math);
420-
});
421-
}, '', false],
422-
insertedScript: [200, () => {
423-
document.querySelectorAll('mjx-container').forEach(node => {
424-
let target = node.parentNode;
425-
if (target.nodeName.toLowerCase() === 'li') {
426-
target.parentNode.classList.add('has-jax');
427-
}
428-
});
429-
}, '', false]
430-
}
431-
}
432-
};
433-
(function () {
434-
var script = document.createElement('script');
435-
script.src = '//cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js';
436-
script.defer = true;
437-
document.head.appendChild(script);
438-
})();
439-
} else {
440-
MathJax.startup.document.state(0);
441-
MathJax.texReset();
442-
MathJax.typeset();
443-
}
444-
</script>
445-
446-
447394

448395

449396

‎2025/06/10/A0-onboarding/index.html‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
<meta property="og:description" content="本作业旨在帮助你熟悉编程环境、提交流程以及基本的 PyTorch 编程。通过完成它，你将确保开发环境配置正确，理解如何提交未来的作业，并加强 PyTorch 编程技能。 Environment SetupOption 1: Local Setup Python: 3.10 或更高版本 Packages: 推荐通过以下命令安装所有必要的依赖项： 1pip install -r requirem">
2929
<meta property="og:locale" content="zh_CN">
3030
<meta property="article:published_time" content="2025-06-10T08:54:20.000Z">
31-
<meta property="article:modified_time" content="2025-06-27T06:39:15.524Z">
31+
<meta property="article:modified_time" content="2025-06-29T13:51:36.393Z">
3232
<meta property="article:author" content="DeepEngine">
3333
<meta property="article:tag" content="Environment">
3434
<meta name="twitter:card" content="summary">
@@ -186,7 +186,7 @@ <h1 class="post-title" itemprop="name headline">
186186
<i class="far fa-calendar-check"></i>
187187
</span>
188188
<span class="post-meta-item-text">更新于</span>
189-
<time title="修改时间：2025-06-27 14:39:15" itemprop="dateModified" datetime="2025-06-27T14:39:15+08:00">2025-06-27</time>
189+
<time title="修改时间：2025-06-29 21:51:36" itemprop="dateModified" datetime="2025-06-29T21:51:36+08:00">2025-06-29</time>
190190
</span>
191191
<span class="post-meta-item">
192192
<span class="post-meta-item-icon">
@@ -285,7 +285,7 @@ <h3 id="TODO-Task-A0"><a href="#TODO-Task-A0" class="headerlink" title="TODO: Ta
285285
\mathbf{Y} = \mathbf{X} \times \mathbf{W} \Rightarrow
286286
\frac{\partial \mathbf{Y}}{\partial \mathbf{X}} = \mathbf{W}^\top</script></li>
287287
<li><p>对于 $\frac{\partial \mathbf{Y}}{\partial \mathbf{W}}$：</p>
288-
<p>首先，考虑单个样本 $ \mathbf{X}_i $ 的 <code>forward</code> 计算，有：$\mathbf{y} = \mathbf{x} \times W$，如果把 b 理解成 batch，那 $\mathbf{W}$ 对不同样本共享权重。同样：</p>
288+
<p>首先，考虑单个样本 $ \mathbf{X}_i $ 的 <code>forward</code> 计算，有：$\mathbf{y} = \mathbf{x} \times \mathbf{W}$，如果把 b 理解成 batch，那 $\mathbf{W}$ 对不同样本共享权重。同样：</p>
289289
<script type="math/tex; mode=display">\mathbf{y}_{j:}=\mathbf{x}_j \times \mathbf{W} \Rightarrow \frac{\partial \mathbf{y}_{jk}}{\partial \mathbf{W}_{:k}} = \mathbf{x}_j^\top</script><script type="math/tex; mode=display">\frac{\partial \text{Loss}^{(i)}}{\partial \mathbf{W}} = \frac{\partial \text{Loss}^{(i)}}{\partial \mathbf{Y}_i}\frac{\partial \mathbf{Y}_i}{\partial \mathbf{W}} = \mathbf{X}_i^\top \times \frac{\partial \text{Loss}^{(i)}}{\partial \mathbf{Y}_i}</script><p>其中，$\text{Loss}^{(i)}$ 可以理解为第 $i$ 个样本对于最终损失的贡献。最后，考虑 batch 个样本，根据链式法则组合，有：</p>
290290
<script type="math/tex; mode=display">\frac{\partial \text{Loss}}{\partial \mathbf{W}} = \sum_{i=1}^{b} \mathbf{X}_i^\top \times \frac{\partial \text{Loss}}{\partial \mathbf{Y}_i}</script><p>由此，我们得到局部导数形式：</p>
291291
<script type="math/tex; mode=display">
@@ -525,7 +525,7 @@ <h1 id="Contact"><a href="#Contact" class="headerlink" title="Contact"></a>Conta
525525
<div class="site-state-item site-state-posts">
526526
<a href="/LLM-Blog/archives/">
527527

528-
<span class="site-state-item-count">4</span>
528+
<span class="site-state-item-count">5</span>
529529
<span class="site-state-item-name">日志</span>
530530
</a>
531531
</div>
@@ -536,7 +536,7 @@ <h1 id="Contact"><a href="#Contact" class="headerlink" title="Contact"></a>Conta
536536
<span class="site-state-item-name">分类</span></a>
537537
</div>
538538
<div class="site-state-item site-state-tags">
539-
<span class="site-state-item-count">5</span>
539+
<span class="site-state-item-count">7</span>
540540
<span class="site-state-item-name">标签</span>
541541
</div>
542542
</nav>

‎2025/06/14/A1-matmul/index.html‎

Lines changed: 2 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ <h5 id="TODO-2"><a href="#TODO-2" class="headerlink" title="TODO"></a>TODO</h5><
335335
<div class="site-state-item site-state-posts">
336336
<a href="/LLM-Blog/archives/">
337337

338-
<span class="site-state-item-count">4</span>
338+
<span class="site-state-item-count">5</span>
339339
<span class="site-state-item-name">日志</span>
340340
</a>
341341
</div>
@@ -346,7 +346,7 @@ <h5 id="TODO-2"><a href="#TODO-2" class="headerlink" title="TODO"></a>TODO</h5><
346346
<span class="site-state-item-name">分类</span></a>
347347
</div>
348348
<div class="site-state-item site-state-tags">
349-
<span class="site-state-item-count">5</span>
349+
<span class="site-state-item-count">7</span>
350350
<span class="site-state-item-name">标签</span>
351351
</div>
352352
</nav>
@@ -434,59 +434,6 @@ <h5 id="TODO-2"><a href="#TODO-2" class="headerlink" title="TODO"></a>TODO</h5><
434434

435435

436436

437-
438-
439-
<script>
440-
if (typeof MathJax === 'undefined') {
441-
window.MathJax = {
442-
loader: {
443-
source: {
444-
'[tex]/amsCd': '[tex]/amscd',
445-
'[tex]/AMScd': '[tex]/amscd'
446-
}
447-
},
448-
tex: {
449-
inlineMath: {'[+]': [['$', '$']]},
450-
tags: 'ams'
451-
},
452-
options: {
453-
renderActions: {
454-
findScript: [10, doc => {
455-
document.querySelectorAll('script[type^="math/tex"]').forEach(node => {
456-
const display = !!node.type.match(/; *mode=display/);
457-
const math = new doc.options.MathItem(node.textContent, doc.inputJax[0], display);
458-
const text = document.createTextNode('');
459-
node.parentNode.replaceChild(text, node);
460-
math.start = {node: text, delim: '', n: 0};
461-
math.end = {node: text, delim: '', n: 0};
462-
doc.math.push(math);
463-
});
464-
}, '', false],
465-
insertedScript: [200, () => {
466-
document.querySelectorAll('mjx-container').forEach(node => {
467-
let target = node.parentNode;
468-
if (target.nodeName.toLowerCase() === 'li') {
469-
target.parentNode.classList.add('has-jax');
470-
}
471-
});
472-
}, '', false]
473-
}
474-
}
475-
};
476-
(function () {
477-
var script = document.createElement('script');
478-
script.src = '//cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js';
479-
script.defer = true;
480-
document.head.appendChild(script);
481-
})();
482-
} else {
483-
MathJax.startup.document.state(0);
484-
MathJax.texReset();
485-
MathJax.typeset();
486-
}
487-
</script>
488-
489-
490437

491438

492439

‎2025/06/17/A2-norm-emb/index.html‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -295,7 +295,7 @@ <h4 id="Optional-Task4：旋转位置编码"><a href="#Optional-Task4：旋转
295295
<li>输出张量 <code>E</code> 的形状与 <code>X</code> 的形状相同，表示应用旋转编码后的结果。</li>
296296
</ul>
297297
</li>
298-
<li><p>由于 <strong>RoPE</strong> 矩阵的稀疏性，直接用矩阵乘法来实现会很浪费算力，推荐使用 $(8)$ 中的方法来实现标准 <strong>RoPE</strong>。其中，$\bigotimes$ 是逐位相乘，并且可以注意到 $(8)$ 中，$X$ 按 $(x<em>0,x_1),(x_2,x_3),\cdots$ 的顺序进行分组。而 <code>Llama</code> 和 <code>ChatGLM</code> 的 <strong>RoPE</strong> 模块对 $X$ 的顺序进行了重组，也即按 $(x_0,x</em>{\frac{d}{2}}),(x<em>1,x</em>{\frac{d}{2}+1}),\cdots$ 的顺序进行分组，请使用 $(9)$ 中的方法实现 <strong>Task4</strong> 中的 <strong>RoPE</strong>。</p>
298+
<li><p>由于 <strong>RoPE</strong> 矩阵的稀疏性，直接用矩阵乘法来实现会很浪费算力，推荐使用 $(8)$ 中的方法来实现标准 <strong>RoPE</strong>。其中，$\bigotimes$ 是逐位相乘，并且可以注意到 $(8)$ 中，$X$ 按 $(x_0,x_1),(x_2,x_3),\cdots$ 的顺序进行分组。而 <code>Llama</code> 和 <code>ChatGLM</code> 的 <strong>RoPE</strong> 模块对 $X$ 的顺序进行了重组，也即按 $(x_0,x_{\frac{d}{2}}),(x_1,x_{\frac{d}{2}+1}),\cdots$ 的顺序进行分组，请使用 $(9)$ 中的方法实现 <strong>Task4</strong> 中的 <strong>RoPE</strong>。</p>
299299
<div class="note info">
300300
<p>余弦\正弦参数对 <code>(C, S)</code> 也要做相应调整。</p>
301301
</div>
@@ -451,7 +451,10 @@ <h5 id="TODO-3"><a href="#TODO-3" class="headerlink" title="TODO"></a>TODO</h5><
451451
<a href="/LLM-Blog/2025/06/14/A1-matmul/" rel="prev" title="A1 matmul">
452452
<i class="fa fa-chevron-left"></i> A1 matmul
453453
</a></div>
454-
<div class="post-nav-item"></div>
454+
<div class="post-nav-item">
455+
<a href="/LLM-Blog/2025/06/29/A3-modeling-mlp/" rel="next" title="A3 Modeling MLP">
456+
A3 Modeling MLP <i class="fa fa-chevron-right"></i>
457+
</a></div>
455458
</div>
456459
</footer>
457460

@@ -524,7 +527,7 @@ <h5 id="TODO-3"><a href="#TODO-3" class="headerlink" title="TODO"></a>TODO</h5><
524527
<div class="site-state-item site-state-posts">
525528
<a href="/LLM-Blog/archives/">
526529

527-
<span class="site-state-item-count">4</span>
530+
<span class="site-state-item-count">5</span>
528531
<span class="site-state-item-name">日志</span>
529532
</a>
530533
</div>
@@ -535,7 +538,7 @@ <h5 id="TODO-3"><a href="#TODO-3" class="headerlink" title="TODO"></a>TODO</h5><
535538
<span class="site-state-item-name">分类</span></a>
536539
</div>
537540
<div class="site-state-item site-state-tags">
538-
<span class="site-state-item-count">5</span>
541+
<span class="site-state-item-count">7</span>
539542
<span class="site-state-item-name">标签</span>
540543
</div>
541544
</nav>

0 commit comments

Comments
 (0)