Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples

By Gail Weiss, Yoav Goldberg, Eran Yahav. 2017

209 citations recorded by the source corpus.

We present a novel algorithm that uses exact learning and abstraction to extract a deterministic finite automaton describing the state dynamics of a given trained RNN. We do this using Angluin's L* algorithm as a learner and the trained RNN as an oracle. Our technique efficiently extracts accurate automata from trained RNNs, even when the state vectors are large and require fine differentiation.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Model Architectures

In Safety, Robustness & Interpretability

Citations in the corpus

Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold (2022, methodology): “Studies proposing more interpretable methods typically build on state-of-the-art methods (Weiss et al., 2018) and much work focuses on leveraging components such as attention for interpretability, which have not been designed with that goal in mind (Serrano and Smith, 2019; Wiegreffe and Pinter,…”
Learning Causal State Representations of Partially Observable Environments (2019, background): “The most recent examples [41, 39, 40] are focused on the ability of character level RNNs to implicitly learn grammars.”
Learning Finite State Representations of Recurrent Policy Networks (2018, methodology): “A more recent approach is to use classic query-based learning algorithms to extract FSMs by asking membership and equivalence queries (Weiss et al., 2017).”