Challenges in coevolutionary learning: arms-race dynamics, open-endedness, and medicocre stable states

By Sevan Gregory Ficici, Jordan B. Pollack. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Multiagent Learning: Basics, Challenges, and Prospects (2012, background): “These paradigms are coevolutionary learning (Paredis 1995, Ficici and Pollack 1998), that is the field that investigates and develops learning algorithms inspired by natural evolution, using operators like selection, mutation, crossover, and others; multiview learning (Christoudias, Urtasun, and…”
Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions (2019, background): “This recognition is reﬂected in systems based on self-play (which is related to coevolution [29–31]), such as competitive two-player RL competitions [14, 32] wherein the task is a function of the competition, which is itself changing over time.”
Genetic and Evolutionary Computation – GECCO 2004 (2004, methodology): “This problem arises in various areas such as facility layout [9], graph partitioning [10], quorum cast routing [5], telecommunications [11], and matrix decomposition [4].”
POET: open-ended coevolution of environments and their optimized solutions (2019, background): “From an evolutionary perspective, one way to think about learning environments evolving along with their solutions is through the coevolution [10, 14, 26, 44, 62] of the two.”
Abandoning Objectives: Evolution Through the Search for Novelty Alone (2011, methodology): “…and Fogel, 1999; Hillis, 1991; Sims, 1994), but a common problem is that in practice such arms races may converge to mediocre stable states (Ficici and Pollack, 1998; Pollack et al., 1996; Watson and Pollack, 2001), cycle among various behaviors without further progress (Cartlidge and…”
Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions (2020, background): “…et al., 2019; Samuel, 1967; Silver et al., 2018) and Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) (both related to coevolution (Ficici & Pollack, 1998; Popovici et al., 2012; Wiegand et al., 2001)) have shown the bene-ﬁt of optimizing against constantly changing, increasingly…”
Revising the evolutionary computation abstraction: minimal criteria novelty search (2010, background): “…evolutionary algorithms (EAs), although based on abstractions of natural evolution, do not exhibit the same drive towards complexity as natural evolution; EAs will fail to solve a problem if the complexity required to solve is prohibitively high (e.g. an intelligent autonomous robot) [11] [38].”
Evolving Soccer Keepaway Players Through Task Decomposition (2005, background): “At one extreme of this dimension lies the paradigm known as coevolution (Rosin and Belew, 1995; Potter and Jong, 2000; Ficici and Pollack, 1998; Stanley and Miikkulainen, 2004), in which no additional information is given to the learner beyond the task decomposition itself.”
On the deleterious effects of a priori objectives on evolution and representation (2011, background): “A fundamental dilemma with traditional approaches is that crafting an effective fitness function is akin to understanding the fitness landscape or knowing the stepping stones a priori [5, 28].”
A Game-Theoretic Memory Mechanism for Coevolution (2003, background): “Some domains, however, permit the use of an external objective metric of behavior [ 4 ], or allow easy comparison to a known optimal solution [8], as alternative ways to detect forgetting.)”
Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents (2019, unknown): “Artiﬁcial Life and Multiagent Reinforcement Learning Research in “Artiﬁcial life” aims to model evolution and natural selection in biological life; (Langton, 1997; Ficici & Pollack, 1998).”