Neural Networks · 2022 · Journal article
Anomalous diffusion dynamics of learning in deep neural networks
Stochastic gradient descent navigates fractal-like deep network loss landscapes through anomalous superdiffusion, enabling escape from sharp local minima.
The scientific question
How does stochastic gradient descent escape sharp local minima in deep networks?
Deep network training optimizes highly non-convex loss landscapes, yet standard Brownian diffusion approximations fail to explain how SGD reliably finds wide minima.

The core idea
Statistical Physics of Loss Navigation
Using energy-landscape methods from complex physical systems, we track weight trajectories and diffusion exponents during stochastic gradient descent across convolutional and residual architectures.
What we found
Fractal Landscapes Drive Superdiffusive Jumps
Interaction between SGD and fractal-like loss geometry produces early superdiffusive jumps that escape sharp local minima, followed by late-stage subdiffusion as weights settle.
Continue the question
Related reading
- A spatiotemporal mechanism of visual attention: Superdiffusive motion and theta oscillations of neural population activity patternsScience Advances · 2022
- Global Credit Assignment via Dynamical CriticalityICML 2026 · 2026
Cite this work
Guozhang Chen, Cheng Kevin Qu, Pulin Gong. Anomalous diffusion dynamics of learning in deep neural networks. Neural Networks, 2022. doi:10.1016/j.neunet.2022.01.019.