Deep reservoir computing: A critical experimental analysis

By C. Gallicchio, Alessio Micheli, L. Pedrelli. Neurocomputing, 2017

485 citations recorded by the source corpus.

Abstract In this paper, we propose an empirical analysis of deep recurrent neural network (RNN) architectures with stacked layers. The main aim is to address some fundamental open research issues on the significance of creating deep layered architectures in RNN and to characterize the inherent hierarchical representation of time in such models, especially for efficient implementations. In particular, the analysis aims at the study and proposal of approaches to develop and enhance hierarchical dynamics in deep architectures within the efficient Reservoir Computing (RC) framework for RNN modeling. The effect of a deep layered organization of RC models is investigated in terms of both occurrence of multiple time-scale and increasing of richness of the dynamics. It turns out that a deep layering of recurrent models allows an effective diversification of temporal representations in the layers of the hierarchy, by amplifying the effects of the factors influencing the time-scales and the richness of the dynamics, measured as the entropy of recurrent units activations. The advantages of the proposed approach are also highlighted by measuring the increment of the short-term memory capacity of the RC models.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/bc6b1bc042dacc2cba5242c32cdb8f94f896d4d3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

  1. Wei Lin
  2. Peter Bienstman
  3. Jie Sun
  4. Peter Tino
  5. Can Huang
  6. Min Yan

Citations in the corpus