A Machine Learning Perspective on Predictive Coding with PAQ
By Byron Knoll, Nando de Freitas. 2011
22 citations recorded by the source corpus.
PAQ8 is an open source lossless data compression algorithm that currently achieves the best compression rates on many benchmarks. This report presents a detailed description of PAQ8 from a statistical machine learning perspective. It shows that it is possible to understand some of the modules of PAQ8 and use this understanding to improve the method. However, intuitive statistical explanations of the behavior of other modules remain elusive. We hope the description in this report will be a starting point for discussions that will increase our understanding, lead to improvements to PAQ8, and facilitate a transfer of knowledge from PAQ8 to other machine learning methods, such a recurrent neural networks and stochastic memoizers. Finally, the report presents a broad range of new applications of PAQ to machine learning tasks including language modeling and adaptive text prediction, adaptive game playing, classification, and compression using features from the field of deep learning.
Subfields: Artificial Intelligence, Computational Theory and Mathematics, Optimization & Training, Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Alex Graves
- Yoshua Bengio
- J. Urgen Schmidhuber
- Quoc V. Le
- Kyunghyun Cho
- Sergey Levine
- Mohit Bansal
- Ruslan Salakhutdinov
In Computational Theory and Mathematics
In Optimization & Training
In Probabilistic & Bayesian Methods
- Alex Graves
- Yarin Gal
- Yoshua Bengio
- Richard E. Turner
- Zoubin Ghahramani
- Roger Baker Grosse
- Max Welling
- Christos Louizos
Citations in the corpus
- Generating Sequences With Recurrent Neural Networks (2013, methodology): “One advantage of adaptive weight is that early stopping is not needed The results are presented with two equivalent metrics: bits-per-character (BPC), which is the average value of − log 2 Pr( x t +1 | y t ) over the whole test set; and perplexity which is two to the power of the average number of b”