A General SIMD-Based Approach to Accelerating Compression Algorithms
By Wayne Xin Zhao, Xudong Zhang, Daniel Lemire, Dongdong Shan, Jian-Yun Nie, Hongfei Yan, Ji-rong Wen. 2015
45 citations recorded by the source corpus.
Compression algorithms are important for data-oriented tasks, especially in the era of “Big Data.” Modern processors equipped with powerful SIMD instruction sets provide us with an opportunity for achieving better compression performance. Previous research has shown that SIMD-based optimizations can multiply decoding speeds. Following these pioneering studies, we propose a general approach to accelerate compression algorithms. By instantiating the approach, we have developed several novel integer compression algorithms, called Group-Simple, Group-Scheme, Group-AFOR, and Group-PFD, and implemented their corresponding vectorized versions. We evaluate the proposed algorithms on two public TREC datasets, a Wikipedia dataset, and a Twitter dataset. With competitive compression ratios and encoding speeds, our SIMD-based algorithms outperform state-of-the-art nonvectorized algorithms with respect to decoding speeds.
Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jonathan Frankle
- John V. Guttag
- Davis W. Blalock
- Jose Javier Gonzalez Ortiz
- Michael Carbin
- Tianlong Chen
- Zhangyang Wang
- Michael Crawshaw
In Information Systems
Citations in the corpus
- What is the State of Neural Network Pruning? (2020, background): “…ratio,” which the compression literature has long used to mean original size compressed size (Siedelmann et al., 2015; Zukowski et al., 2006; Zhao et al., 2015; Lindstrom, 2014; Ratanaworabhan et al., 2006; Blalock et al., 2018), but many pruning authors define (usually without making the…”