Evolino for recurrent support vector machines

By J. Urgen Schmidhuber, Matteo Gagliolo, Daan Wierstra, Faustino J. Gomez. 2005

31 citations recorded by the source corpus.

Traditional Support Vector Machines (SVMs) need pre-wired finite time windows to predict and classify time series. They do not have an internal state necessar y to deal with sequences involving arbitrary long-term dependencies. Here we introduce a new class of recurrent, truly sequential SVM-like devices with internal adaptive states, trained by a novel method called EVOlution of systems with KErnel-based outputs (Evoke), an instance of the recent Evolino class of methods [1, 2]. Evoke evolves recurrent neural networks to detect and represent temporal dependencies while using quadratic programming/support vector regression to produce precise outputs, in contrast t o our recent work [1, 2] which instead uses pseudoinverse regression. Evoke is the first SVM-based mechanis m learning to classify a context-sensitive language. It also outperforms recent state-of-the-art gra dient-based recurrent neural networks (RNNs) on various time series prediction tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Reservoir computing approaches to recurrent neural network training (2009, methodology): “A standard SVM (having its own kernel) can also be used as a readout from a continuous-value reservoir [133].”
Training Recurrent Networks by Evolino (2007, methodology): “We call this Evolino va riant EVOlution of systems with KErnel-based outputs (Evoke; [37]).”
Investigation of financial market prediction by recurrent neural network (2011, background): “Investigated network reached good learning and still makes very accurate predictions [1], [10-12].”