NeurIPS 2026 · Accepted

Aligning Inductive Bias for Data-Efficient Generalization in State Space Models

Qiyu Chen, Guozhang Chen

Formalizing state space model inductive bias through kernel frequency response enables task-dependent spectral initialization for data-efficient sequence generalization.

The scientific question

How can sequence models align their initial inductive bias with task structure?

State space models rely on fixed, task-agnostic priors; when this bias mismatches task structure, training consumes extra samples overcoming initialization.

Overview of state-space model inductive bias and task-dependent initialization.
Figure 1 from the preprint ↗

The core idea

Power-Spectrum Matching Initialization

We characterize linear time-invariant state space model bias via the spectrum of its induced kernel and introduce Task-Dependent Initialization (TDI) to match task power spectra.

What we found

Data-Efficient Spectral Generalization

Across synthetic and real-world sequence benchmarks, aligning initial model frequency response with task spectral structure substantially improves sample efficiency when default priors are mismatched.

Continue the question

Related reading

Explore our research

Cite this work

Qiyu Chen, Guozhang Chen. Aligning Inductive Bias for Data-Efficient Generalization in State Space Models. NeurIPS 2026.