Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting
PDF Proceedings arXiv Code Poster
Shows that a linear attention layer can be interpreted as a dynamic VAR; proposes SAMoVAR to realign multi‑layer Transformers with autoregressive forecasting for improved interpretability and accuracy.
Interprets linear attention through the lens of VAR and reorganizes the stack (attention/MLP/I‑O flow) to match autoregressive objectives, yielding more interpretable and competitive multivariate TSF models.