Optimized Particle Swarm Optimization (OPSO) and its application to artificial neural network training

By Michael Meissner, Michael Schmuker, Gisbert Schneider. BMC Bioinformatics, 2006

320 citations recorded by the source corpus.

BackgroundParticle Swarm Optimization (PSO) is an established method for parameter optimization. It represents a population-based adaptive optimization technique that is influenced by several "strategy parameters". Choosing reasonable parameter values for the PSO is crucial for its convergence behavior, and depends on the optimization task. We present a method for parameter meta-optimization based on PSO and its application to neural network training. The concept of the Optimized Particle Swarm Optimization (OPSO) is to optimize the free parameters of the PSO by having swarms within a swarm. We assessed the performance of the OPSO method on a set of five artificial fitness functions and compared it to the performance of two popular PSO implementations.ResultsOur results indicate that PSO performance can be improved if meta-optimized parameter sets are applied. In addition, we could improve optimization speed and quality on the other PSO methods in the majority of our experiments. We applied the OPSO method to neural network training with the aim to build a quantitative model for predicting blood-brain barrier permeation of small organic molecules. On average, training time decreased by a factor of four and two in comparison to the other PSO methods, respectively. By applying the OPSO method, a prediction model showing good correlation with training-, test- and validation data was obtained.ConclusionOptimizing the free parameters of the PSO method can result in performance gain. The OPSO approach yields parameter combinations improving overall optimization performance. Its conceptual simplicity makes implementing the method a straightforward task.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Neurology

View Paper: https://www.semanticscholar.org/paper/bbbfbc678cf3c557d2e3551ccb2acc787b00399d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Neurology

Citations in the corpus

A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data (2013, methodology): “It has been shown by test results that this method performs better than other methods (Meissner et al. 2006).”
Simplifying Particle Swarm Optimization (2010, methodology): “[20] who attempt to tune the PSO parameters by using another instance of the PSO method as overlaid meta-optimizer as well.”
Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks (2007, methodology): “In order to overcome the disadvantages of gradient based algorithms, many global optimization methods have been proposed for training feed-forward neural networks such as Genetic Algorithms [4,5,6,7,8,9,10,11,12,13,14,15,16,17], The Particle Swarm Optimization algorithm [18, 19, 20, 22, 21, 23], Dif”
Parameter tuning for configuring and analyzing evolutionary algorithms (2011, unknown): “OPSO [67] Multi stage A”
Hyperparameter Search in Machine Learning (2015, methodology): “A wide variety of optimization methods have been used for hyp erparameter search, including particle swarm optimization [26, 27], genetic algorithms [32], coup led simulated annealing [33] and racing algorithms [6].”