神经常微分方程（NODE）介绍

NODE

Neural Ordinary differential equation是对ResNet或者RNN模块的一种连续化结果，二者每个block的计算公式如下：
$$h_{t+1}=h_t+f(h_t,{\theta }_t,t)$$
对其进行适当的变换¹可以得到：
h t + 1 = f ( h t , θ t , t ) + h t = Δ t Δ t f ( h t , θ t , t ) + h t = Δ t f ( h t , θ t , t ) Δ t + h t \begin{aligned} h_{t+1}&=f(h_t,\theta_t,t)+h_t\\ &=\frac{\Delta_t}{\Delta_t}f(h_t,\theta_t,t)+h_t\\ &=\Delta_t\frac{f(h_t,\theta_t,t)}{\Delta_t}+h_t \end{aligned} ht+1​​=f(ht​,θt​,t)+ht​=Δt​Δt​​f(h