Time2Vec: Learning a Vector Representation of Time

By Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, J. Ramanan, Jaspreet Sahota, S. Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, Marcus A. Brubaker. 2019

497 citations recorded by the source corpus.

Time is an important feature in many applications involving events that occur synchronously and/or asynchronously. To effectively consume time information, recent studies have focused on designing new architectures. In this paper, we take an orthogonal but complementary approach by providing a model-agnostic vector representation for time, called Time2Vec, that can be easily imported into many existing and future architectures and improve their performances. We show on a range of models and problems that replacing the notion of time with its Time2Vec representation improves the performance of the final model.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Michael M. Bronstein
2. Benjamin Paul Chamberlain
3. Pan Li
4. Fabrizio Frasca
5. Pascal Poupart
6. Rishab Goel
7. Federico Monti
8. Karsten M. Borgwardt

In Representation & Self-Supervised Learning

1. Pan Li
2. Michael M. Bronstein
3. Yanbang Wang
4. Seyed Mehran Kazemi
5. Rishab Goel
6. Pascal Poupart
7. Benjamin Paul Chamberlain
8. Federico Monti

In Signal Processing

1. Federico Monti
2. Peter Forsyth
3. Bogdan Gabrys
4. Jure Leskovec
5. Karsten M. Borgwardt
6. Michael M. Bronstein
7. Katarzyna Musial
8. Benjamin M. Marlin

In Time-Series & Forecasting

1. Bogdan Gabrys
2. Jure Leskovec

Citations in the corpus

• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (2020, neutral): “Prior works in natural language processing and time series analysis [18, 39, 42] have used a similar positional encoding to represent time or 1D position.”
• Foundations and Modeling of Dynamic Networks Using Dynamic Graph Neural Networks: A Survey (2020, methodology): “TGAT [18] Temporal [113] and structural [88] attention Yes No Yes No Directed or undirected interaction”
• Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks (2021, methodology): “For f 2 ( t ) , we adopt random Fourier features to encode time (Xu et al., 2019; Kazemi et al., 2019) which may approach any positive definite kernels according to the Bochner’s theorem (Bochner, 1992). where ω i ’s are learnable parameters.”
• Representation Learning for Dynamic Graphs: A Survey (2019, background): “Wikidata: Leblay and Chekol (2018) created a temporal knowledge base19 using Wikidata.”
• Multi-Time Attention Networks for Irregularly Sampled Time Series (2020, methodology): “(2019) and Kazemi et al. (2019), our proposed method learns a time representation.”
• Temporal Graph Networks for Deep Learning on Dynamic Graphs (2020, methodology): “is a time encoding and z i(t) = emb(i;t) = h (L) i (t). In the experiment, both for the Temporal Graph Attention and for the Temporal Graph Sum modules we use the time encoding presented in Time2Vec (Kazemi et al., 2019) and used in TGAT (Xu et al., 2020). The graph embedding modules mitigate the st”