The irace package: Iterated racing for automatic algorithm configuration

By Manuel López-Ibáñez, Jérémie Dubois-Lacoste, Leslie Pérez Cáceres, Mauro Birattari, Thomas Stützle. 2016

1,797 citations recorded by the source corpus.

Modern optimization algorithms typically require the setting of a large number of parameters to optimize their performance. The immediate goal of automatic algorithm configuration is to find, automatically, the best parameter settings of an optimizer. Ultimately, automatic algorithm configuration has the potential to lead to new design paradigms for optimization software. The irace package is a software package that implements a number of automatic configuration procedures. In particular, it offers iterated racing procedures, which have been used successfully to automatically configure various state-of-the-art algorithms. The iterated racing procedures implemented in irace include the iterated F-race algorithm and several extensions and improvements over it. In this paper, we describe the rationale underlying the iterated racing procedures and introduce a number of recent extensions. Among these, we introduce a restart mechanism to avoid premature convergence, the use of truncated sampling distributions to handle correctly parameter bounds, and an elitist racing procedure for ensuring that the best configurations returned are also those evaluated in the highest number of training instances. We experimentally evaluate the most recent version of irace and demonstrate with a number of example applications the use and potential of irace, in particular, and automatic algorithm configuration, in general.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Optimal Decision Trees for the Algorithm Selection Problem: Integer Programming Based Approaches (2019, methodology): “Popular software packages such as irace (López-Ibáñez et al., 2016) and ParamILS (Hutter et al.”
SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization (2021, background): “A more general view on the problem of HPO is called algorithm conﬁguration (AC) (Hutter et al., 2009; Ans´otegui et al., 2009; L´opez-Ib´a˜nez et al., 2016), where the goal is to determine a well-performing robust conﬁguration across a set of problem instances i from a ﬁnite set I : The origin of…”
Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model (2024, methodology): “With the development of various effective methodoliges (Blot et al., 2016; L´opez-Ib´a˜nez et al., 2016; Akiba et al., 2019) and frameworks (Burke et al., 2019), one can tune heuristics or combine different algorithmic components in an automatic manner.”
Efficient Parameter Importance Analysis via Ablation with Surrogates (2017, methodology): “…racing-ablation analysis, reduces the computational burden by applying the statistical techniques introduced in F-race (Birattari et al. 2002; López-Ibáñez et al. 2016) to eliminate some candidate configurations as soon as there is enough statistical evidence on an iteratively increased…”
Bio-inspired computation: Where we stand and what's next (2019, methodology): “The mechanism of I-RACE is simple: from an executable algorithm and a list of parameters (indicating, for each one, the type and range of possible values), the software samples a distribution for each parameter to be tuned, and updates it with the best configurations (as per a racing mechanism) to b”
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges (2021, methodology): “Particularly well-suited for HPO are iterated races (L (cid:2) opez-Ib (cid:2) añez et al., 2016), and we will use the terminology of that implementation to explain the main control parameters of IR.”
Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates (2017, methodology): “…local search (Hutter et al 2009), GGA, which leverages genetic algorithms (Ansótegui et al 2009, 2015), irace, a generalization of racing methods (López-Ibáñez et al 2011; Lang et al 2015), OpenTuner, an approach which appeals to bandit solvers on top of a set of search heuristics (Ansel et…”
A Tutorial On the design, experimentation and application of metaheuristic algorithms to real-World optimization problems (2021, methodology): “The second alternative implies the use of tools for automatic parameter tuning of metaheuristics [126], such as irace [127] and ParamILS [128], although it must be taken into account that the tuning with these kinds of tools can be computationally unaﬀordable in real-world problems.”
Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP (2018, methodology): “We calibrate the SA’s initial temperature T , cooling rate α and stopping temperature Tmin with the irace automatic algorithm configuration package (López-Ibáñez et al. 2016).”
LEAPSANDBOUNDS: A Method for Approximately Optimal Algorithm Configuration (2018, methodology): “, 2011; 2013), irace (Birattari et al., 2002; López-Ibánez et al., 2011), and GGA (Ansótegui et al.”
Recent advances in differential evolution - An updated survey (2016, methodology): “This strategy has been recently applied to devise a hierarchical binary DE for solving the minimal cut-sets identification problem in (Maio et al., 2015) In most of the occasions when DE needs to operate on discrete valued decision variables, a usual practice is to convert a real-valued solution int”
A Survey of Automatic Parameter Tuning Methods for Metaheuristics (2020, methodology): “[30] provided a software package implemented in programming language R, called irace, that includes the iterated F-Race algorithm as well”
Automated Algorithm Selection: Survey and Perspectives (2018, background): “Therefore, per-set algorithm selection techniques are typically not directly applicable to algorithm con-ﬁguration, although racing techniques can be extended to work well in this case (see, e.g., L´opez-Ib´a˜nez et al., 2016; P´erez C´aceres et al., 2017).”