本文深入解析卷积神经网络CNN的反向传播算法，涵盖全连接层、池化层和卷积层的梯度计算，详细阐述在不同层中如何更新权重和偏置，并给出具体计算公式。
1. 全连接层
与深度神经网络DNN的反向传播算法一致，辅助变量：
(1) { δ L = ∂ J ∂ z L = ∂ J ∂ a L ⊙ σ ′ ( z L ) δ l = ( W l + 1 ) T δ l + 1 ⊙ σ ′ ( z l ) \left\{\begin{aligned} &\delta^L = \frac{\partial J}{\partial z^L} = \frac{\partial J}{\partial a^L} \odot \sigma'(z^L)\\ &\\ &\delta^l = (W^{l+1})^T\delta^{l+1}\odot \sigma'(z^l) \end{aligned}\right. \tag{1} ⎩⎪⎪⎪⎨⎪⎪⎪⎧δL=∂zL∂J=∂aL∂J⊙σ′(zL)δl=(Wl+1)Tδl+1⊙σ′(zl)(1)
进而求得参数 W W W， b b b的梯度：
{ ∂ J ∂ W l = ∂ J ∂ z l ∂ z l ∂ W l = δ l ( a l − 1 ) T ∂ J ∂ b l = ∂ J ∂ z l ∂ z l ∂ b l = δ l \left\{\begin{aligned} &\frac{\partial J}{\partial W^l} = \frac{\partial J}{\partial z^l} \frac{\partial z^l}{\partial W^l} = \delta^l(a^{l-1})^T\\ &\\ & \frac{\partial J}{\partial b^l} = \frac{\partial J}{\partial z^l} \frac{\partial z^l}{\partial b^l} = \delta^l \end{aligned}\right. ⎩⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎧∂Wl∂J=∂zl∂J∂Wl∂zl=δl(al−1)T∂bl∂J=∂zl∂J∂bl∂zl=δl
解释一下式(1)中为何使用点乘：
以平方误差损失函数为例：
J = 1 2 ∥ a L − y ∥ 2 2 = 1 2 ∑ i = 1 N ( a i L − y ) 2 = 1 2 ∑ i = 1 N ( σ ( z i L ) − y ) 2 J = \frac{1}{2}\left\Vert a^L - y\right\Vert_2^2= \frac{1}{2}\sum_{i=1}^N(a_i^L - y)^2= \frac{1}{2}\sum_{i=1}^N(\sigma(z_i^L) - y)^2 J=21∥∥aL−y∥∥22=21i=1∑N(aiL−y)2=21
扫一扫
2165
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
