Transformers in Time-Series Analysis: A Tutorial

By Sabeen Ahmed, Ian E. Nielsen, Aakash Tripathi, Shamoon Siddiqui, Ravi Prakash Ramachandran, Ghulam Rasool. Circuits, systems, and signal processing, 2022

307 citations recorded by the source corpus.

Transformer architectures have widespread applications, particularly in Natural Language Processing and Computer Vision. Recently, Transformers have been employed in various aspects of time-series analysis. This tutorial provides an overview of the Transformer architecture, its applications, and a collection of examples from recent research in time-series analysis. We delve into an explanation of the core components of the Transformer, including the self-attention mechanism, positional encoding, multi-head, and encoder/decoder. Several enhancements to the initial Transformer architecture are highlighted to tackle time-series tasks. The tutorial also provides best practices and techniques to overcome the challenge of effectively training Transformers for time-series analysis.

Subfields: Artificial Intelligence, Economics and Econometrics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/bdacaef16d45e1e7826bf5191637d436d5a98451

Citations in the corpus

• Is Mamba Effective for Time Series Forecasting? (2024, unknown): “Ran-gapuram et al. [46] attempts to employ SSM for TSF, but the SSM architecture it used is unable to identify and filter content effectively, and the captured dependencies are solely based on distance, resulting in unsatisfactory performance.”