Using ant colony optimization to optimize long short-term memory recurrent neural networks

By A. Elsaid, Fatima El Jamiy, James Higgins, Brandon Wild, Travis Desell. 2018

30 citations recorded by the source corpus.

This work examines the use of ant colony optimization (ACO) to improve long short-term memory (LSTM) recurrent neural networks (RNNs) by refining their cellular structure. The evolved networks were trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. Results were obtained using MPI (Message Passing Interface) on a high performance computing (HPC) cluster, which evolved 1000 different LSTM cell structures using 208 cores over 5 days. The new evolved LSTM cells showed an improvement in prediction accuracy of 1.37%, reducing the mean prediction error from 6.38% to 5.01% when predicting excessive engine vibrations 10 seconds in the future, while at the same time dramatically reducing the number of trainable weights from 21,170 to 11,650. The ACO optimized LSTM also performed significantly better than traditional Nonlinear Output Error (NOE), Nonlinear AutoRegression with eXogenous (NARX) inputs, and Nonlinear Box-Jenkins (NBJ) models, which only reached error rates of 11.45%, 8.47% and 9.77%, respectively. The ACO algorithm employed could be utilized to optimize LSTM RNNs for any time series data prediction task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Bayesian Neural Architecture Search using A Training-Free Performance Metric (2020, methodology): “Besides NEAT, there are several evolutionary-based approaches to evolve an RNN, such as EXALT [14], EXAMM [32], or a method using ant colony optimization (ACO) to improve LSTM RNNs by refining their cellular structures [15].”
A Survey on Evolutionary Neural Architecture Search (2020, unknown): “SI ACO [105] [135], [137], [158], [159] PSO [73], [74], [96], [101], [102], [109], [130], [160] [37], [38] [24], [46] [55], [84], [161]”
A Specialized Evolutionary Strategy Using Mean Absolute Error Random Sampling to Design Recurrent Neural Networks (2019, methodology): “[37] proposed the use of ant colony optimization (ACO) to improve LSTM RNNs by refining their cellular structure.”
Investigating recurrent neural network memory structures using neuro-evolution (2019, methodology): “Ant colony optimization (ACO) has also been investigated as a way to select which connections should be utilized in RNNs and LSTM RNNs by Desell and ElSaid [23]–[25].”
Random error sampling-based recurrent neural network architecture optimization (2019, methodology): “In [36] proposed the use of ant colony optimization (ACO) to improve LSTM RNNs by reﬁning their cellular structure.”