Low-Cost Recurrent Neural Network Expected Performance Evaluation

By Andrés Camero, Jamal Toutouh, Enrique Alba. 2018

20 citations recorded by the source corpus.

Recurrent neural networks are strong dynamic systems, but they are very sensitive to their hyper-parameter configuration. Moreover, training properly a recurrent neural network is a tough task, therefore selecting an appropriate configuration is critical. There have been proposed varied strategies to tackle this issue, however most of them are still impractical because of the time/resources needed. In this study, we propose a low computational cost model to evaluate the expected performance of a given architecture based on the distribution of the error of random samples. We validate empirically our proposal using three use case.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Bayesian Neural Architecture Search using A Training-Free Performance Metric (2020, methodology): “reduces the evaluation time. In this study, we propose to tackle the architecture optimization problem with a hybrid approach. Speciﬁcally, we combine BO [30, 21] for optimizing the architecture, MRS [7] for evaluating the performance of candidate architectures, and ADAM [23] (a gradient descent-bas”
Waste generation prediction under uncertainty in smart cities through deep neuroevolution (2019, methodology): “Specifically, we propose to study the use of the MAE random sampling [47, 48] to compare RNN architectures, aiming to reduce the computational power and the time needed to find an appropriate architecture.”
A Specialized Evolutionary Strategy Using Mean Absolute Error Random Sampling to Design Recurrent Neural Networks (2019, methodology): “[10], [12] studied the problem so we can compare our”
DLOPT: Deep Learning Optimization Library (2018, methodology): “Most of the hyper-parameter optimization techniques included in DLOPT are originals and are publicly available, i.e. the source code and several scientiﬁc publications 11,15,16 .”
Random error sampling-based recurrent neural network architecture optimization (2019, methodology): “sults of this methodology. Some authors have explored different approaches to speed up the evaluation of DNN architectures to improve the efﬁciency of automatic hyperparameter optimization algorithms [10], [11]. A promising approach to evaluate stacked RNN architectures is the MAE (mean absolute err”