The 3rd International Planning Competition: Results and Analysis

By Maria Fox, Derek Long. Journal of Artificial Intelligence Research, 2003

402 citations recorded by the source corpus.

This paper reports the outcome of the third in the series of biennial international planning competitions, held in association with the International Conference on AI Planning and Scheduling (AIPS) in 2002. In addition to describing the domains, the planners and the objectives of the competition, the paper includes analysis of the results. The results are analysed from several perspectives, in order to address the questions of comparative performance between planners, comparative difficulty of domains, the degree of agreement between planners about the relative difficulty of individual problem instances and the question of how well planners scale relative to one another over increasingly difficult problems. The paper addresses these questions through statistical analysis of the raw results of the competition, in order to determine which results can be considered to be adequately supported by the data. The paper concludes with a discussion of some challenges for the future of the competition series.

Subfields: Artificial Intelligence, Building and Construction, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/a80c555da5b60a3422b0acfc6beb30d46ad652a8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Building and Construction

In Computer Networks and Communications

Citations in the corpus

Anytime Heuristic Search (2011, background): “Influential examples of this approach are the HSP and HSPr planners of Bonet and Geffner (2001), which have performed well in the biennial planning competitions sponsored by the International Conference on Automated Planning and Scheduling (Long & Fox, 2003).”
Deterministic planning in the fifth international planning competition: PDDL3 and experimental evaluation of the planners (2009, methodology): “For the purposes of IPC3, pddl2.”
A review of generalized planning (2019, neutral): “This was evidenced at IPC-2002 where planners exploiting Domain-speciﬁc Control Knowledge (DCK) performed orders of magnitude faster than state-of-the-art planners [54].”
Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates (2017, methodology): “In this case we study satellite instances: planning problems arising in the control and observation scheduling of orbital satellites (Long and Fox 2003).”
VAL: automatic plan validation, continuous effects and mixed initiative planning using PDDL (2004, methodology): “The tool played an important role in the 3rd International Planning Competition [9], allowing reliable and automatic validation of the several thousand plans produced by the competitors.”
PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains (2003, background): “In particular, constraints dictated by the relative placement of actions with durations on a timeline can be approached in this way (Long & Fox, 2003a).”
The Metric-FF Planning System: Translating ''Ignoring Delete Lists'' to Numeric State Variables (2003, background): “A discussion of the STRIPS results can be found in the competition overview article by Long and Fox (2003). There were six numeric domains used in the competition.”
PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning (2018, background): “Domain 3 (figure 2) extends the classic PDDL domain rovers (Long and Fox 2003) by incorporating 3D visibility, distance, reachability, and collision constraints.”
Evaluation in artificial intelligence: from task-oriented to ability-oriented measurement (2017, neutral): “The planning competitionj Planning (Long and Fox 2003; Vallati et al. 2015)”
Automated Planning and Acting (2016, methodology): “The expressive power of HTN methods can be useful for developing practical applications [603, 444, 382], and a good set of methods can enable an HTN planner to perform well on benchmark problems [394].”