nn.F.cross_entropy

用于计算交叉熵损失

实际上在 forward 时会经过 Softmax 和交叉熵两个过程。

Backward

\frac{\partial L}{\partial z _{i}} = j = 1 \sum C \frac{\partial L}{\partial p _{j}} \cdot \frac{\partial p _{j}}{\partial z _{i}} = (- \frac{y _{i}}{p _{i}}) \cdot p_{i} (1 - p_{i}) + j \neq = i \sum (- \frac{y _{j}}{p _{j}}) \cdot (- p_{i} p_{j})

化简得

\frac{\partial L}{\partial z _{i}} = p_{i} - y_{i}

向量形式：

\frac{\partial L}{\partial z} = p - y