Online Learning with Gated Linear Networks
By Joel Veness, Tor Lattimore, Avishkar Bhoopchand, Agnieszka Grabska-Barwinska, Christopher Mattern, Peter Toth. 2017
26 citations recorded by the source corpus.
This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our method gains representational power by the use of data conditioning. We state under general conditions a learnable capacity theorem that shows this approach can in principle learn any bounded Borel-measurable function on a compact subset of euclidean space; the result is stronger than many universality results for connectionist architectures because we provide both the model and the learning procedure for which convergence is guaranteed.
Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Quoc V. Le
- Pieter Abbeel
- Mohit Bansal
- Ruslan Salakhutdinov
- Jianfeng Gao
- Kyunghyun Cho
In Computer Networks and Communications
In Management Science and Operations Research
- Yannis Assael
- Yutian Chen
- Scott E. Reed
- David Krueger
- Oriol Vinyals
- Heiga Zen
- Nando de Freitas
- Caglar Gulcehre
In Optimization & Training
- Yutian Chen
- Nando de Freitas
- Yannis Assael
- Caglar Gulcehre
- Oriol Vinyals
- Scott E. Reed
- David Krueger
- Heiga Zen
Citations in the corpus
- Scalable agent alignment via reward modeling: a research direction (2018, background): “…even more ambitious would be the development of theoretically well-founded scalable learning algorithms that come with probably approximately correct (Dziugaite & Roy, 2017) or sample complexity guarantees, capacity statements, well-calibrated uncertainty estimates, etc. (Veness et al., 2017).”
- Sample Efficient Adaptive Text-to-Speech (2018, background): “Veness et al. (2017) presented a gated linear model able to model complex densities from a single pass of a limited dataset.”