IEEE Transactions on Neural Networks and Learning Systems

By Jane M. Zakovorotnaya, B. Zhanturina. 2019

Reservoir computing (RC) is a machine learning framework for temporal (sequential) pattern recognition, which originates from specific types of recurrent neural network models including echo state networks and liquid state machines. An RC system consists of a dynamical reservoir for mapping inputs into a high-dimensional temporal representation space and a readout for pattern analysis from the high-dimensional signal. Since only the readout part is trained typically with a simple linear algorithm, the learning cost of RC is extremely low compared with that of other recurrent neural networks. Such machine learning methods capable of high-speed learning are particularly demanded for online and real-time information processing systems. Further potential capabilities arise while addressing problems characterized by limited amount of supervised data. Due to these advantages, application fields of RC have been rapidly expanding. Theoretical aspects of RC have also been intensively studied in this decade. In addition, physical RC which exploits physical phenomena to realize the function of a reservoir has attracted much attention toward development of energy efficient machine learning hardware. This special issue focuses on new challenges for fully exploiting the potential of RC in practical applications and realizing extremely efficient information processing hardware. We would like to solicit high-quality works which pursue extremely efficient RC systems and devices, as well as breakthrough approaches to the model design and analysis. The topics include, but are not limited to:

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Comprehensive Survey of Neural Architecture Search (2020, neutral): “Artificially designed networks have made breakthroughs in many fields, including image recognition [4, 7, 112], machine translation [1, 3, 141, 149], semantic segmentation [142, 143, 148], object detection [8, 146, 147], and video understanding [144, 145] and so on.”
Evolving Robust Neural Architectures to Defend from Adversarial Attacks (2019, methodology): “This niching scheme uses the idea of Spectrum-based niching from [Vargas and Murata, 2017] but explores a different approach to it.”
Measuring Catastrophic Forgetting in Neural Networks (2017, neutral): “The fixed expansion layer (FEL) model uses a large hidden layer that is sparsely populated with excitatory and inhibitory weights [4].”
All-but-the-Top: Simple and Effective Postprocessing for Word Representations (2017, background): “Table 6: Before-After results (x100) on the text classiﬁcation task using CNN (Kim, 2014) and vanilla RNN, GRU-RNN and LSTM-RNN.”
Multi‐Omics Factor Analysis—a framework for unsupervised integration of multi‐omics data sets (2018, methodology): “Technically, MOFA builds upon the statistical framework of group Factor Analysis (Virtanen et al , 2012; Khan et al , 2014; Klami et al , 2015; Bunte et al , 2016; Zhao et al , 2016; Leppa¨aho & Kaski, 2017), which we have adapted to the requirements of multi-omics studies (Materials and Methods):…”
Recent Advances in Physical Reservoir Computing: A Review (2018, methodology): “, 2018), incorporating new learning algorithms such as the FORCE learning (Sussillo and Abbott, 2009) and its variants (Pyle and Rosenbaum, 2018), and/or employing various types of reservoir elements (Xia et al., 2015; Inubushi and Yoshimura, 2017).”
Understanding the One Pixel Attack: Propagation Maps and Locality Analysis (2019, background): “, 2013] and neuroevolution with unified neuron models [Vargas and Murata, 2017] would make for a promising investigation.”
A survey on modern trainable activation functions (2020, background): “Based on NIN, authors of (Pang et al., 2017) proposed Convolution in Convolution which uses a sparse MLP instead of the classic full-connected MLP as activation function.”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, background): “in 2015 and the final conclusion was all the LSTM variants were all about the same [140].”
Federated Learning for Intrusion Detection System: Concepts, Challenges and Future Directions (2021, methodology): “[Zhu and Jin(2019)] used an elitist non-dominated sorting genetic algorithm as the multi-objective evolutionary algorithm.”
Universal quantum control through deep reinforcement learning (2018, methodology): “While a classic RL technique, Q-learning, has been applied to quantum control problems recently [17, 18], these studies have not yet included practical leakage or control errors.”
Residual Mixture of Experts (2022, background): “Mixture of Experts (MoE) has a long history through these decades [6, 29, 31, 61].”
On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning (2018, background): “According to the symmetric distribution property of random variables, the following theorem [37] holds:”