Published in ICLR 2026, 2026
Introduces Free Energy Mixer (FEM), which interprets (q,k) attention scores as a prior and performs a log-sum-exp free-energy readout to reweight values at the channel level, enabling a smooth transition from mean aggregation to selective channel-wise retrieval without increasing asymptotic complexity.
Published in NeurIPS 2025 (Spotlight), 2025
Introduces Zero‑Sum Linear Attention (ZeroS), which removes the uniform zero‑order term and reweights residuals to enable stable positive/negative attention weights, allowing contrastive operations within a single layer while retaining O(N) complexity.
Published in ICML 2025 (Poster), PMLR 267: 40848–40867, 2025
Shows that a linear attention layer can be interpreted as a dynamic VAR; proposes SAMoVAR to realign multi‑layer Transformers with autoregressive forecasting for improved interpretability and accuracy.
Published in ICML 2025 (Poster), PMLR 267: 40464–40490, 2025
Adds ARMA structure to autoregressive attention via a weighted varying gate, decoupling long‑range and local effects and improving TSF quality without increasing asymptotic complexity.
Published in ICLR 2025 (Poster), 2025
Reformulates TSF as in‑context learning by constructing tokens of (lookback, future) task pairs, enabling Transformers to adapt predictors from context without parameter updates.
Published in ICML 2024 (Poster), PMLR 235: 32990–33006, 2024
Constructs Auxiliary Time Series (ATS) as exogenous inputs to capture inter‑series relations; identifies continuity, sparsity, and variability principles; improves multivariate TSF even with simple predictors.
Published in ICLR 2024 (Poster), 2024
Presents ARM with AUEL, Random Dropping, and multi‑kernel local smoothing to better capture series‑wise patterns and inter‑series dependencies for long‑term multivariate TSF.