Knows what it knows: a framework for self-aware learning

By Lihong Li, Michael L. Littman, Thomas J. Walsh, Alexander L. Strehl. 2008

259 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics (2016, background): “The framework of self-aware learning [16, 2, 31] considers the case where the learning algorithm has an abstain option of saying “I don’t know”, instead of always making an actual prediction.”
Safe Policy Improvement with Baseline Bootstrapping (2017, background): “One can see it as a knows-what-it-knows algorithm, asking for help from the baseline when it does not know whether it knows (Li et al., 2008).”
Fairness in Learning: Classic and Contextual Bandits (2016, methodology): “We then move on to the general contextual bandit setting in Section 4 and prove a broad characterization result, relating fair contextual bandit learning to KWIK (“Knows What It Knows") learning [22].”
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems (2016, background): “While provably efficient exploration strategies are known for problems with finite states/actions or problems with nice structures (Kakade 2003; Asmuth et al. 2009; Jaksch, Ortner, and Auer 2010; Li et al. 2011; Osband, Russo, and Roy 2013), less is known for the general case, especially when genera”
Reinforcement learning with multi-fidelity simulators (2014, methodology): “In addition, our theoretical results hold for a large class of representations such as linear and Gaussian-noise dynamics covered by the KWIK learning framework [7].”
Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning (2017, background): “The less widely used TCE (total cost of exploration) bounds [13] and KWIK guarantees [14] suffer from the same issueand for conciseness are not discussed in detail.”
Deep Exploration via Randomized Value Functions (2017, background): “A relevant line of work establishes that efficient reinforcement learning with value function generalization reduces to efficient “knows what it knows” (KWIK) online regression (Li and Littman, 2010; Li et al., 2008).”
Efficient Exploration for Dialog Policy Learning with Deep BBQ Networks \& Replay Buffer Spiking (2016, background): “…exploration strategies are known for problems with ﬁnite states/actions or problems with “nice” structures [Kakade, 2003, Asmuth et al., 2009, Jaksch et al., 2010, Li et al., 2011, Osband et al., 2013], not much is known for the general case, especially when function approximation is involved.”
Strategic Object Oriented Reinforcement Learning (2018, unknown): “Leveraging KWIK, at each step of planning, if we encounter an unseen state or an unseen interaction, we consider the reward of that state to be Rmax.”
A unifying framework for computational reinforcement learning theory (2009, methodology): “After formulating the optimization of Dialer as an RL problem, we may apply various RL algorithms to solve it [Williams 2008; Li et al. 2009c].”
Privacy-Preserving Stochastic Gradual Learning (2018, methodology): “To provide the formal termination criteria for early stopping, we leverage the learning framework of KWIK (knows what it knows) [33] to explore the maximum complexity ( T max ) of PRESTIGE.”
Sample Complexity of Multi-task Reinforcement Learning (2013, methodology): “Following similar steps as in [Li et al., 2011], 3 we can see the squared difference on average is at least 3 The noisy union algorithm of [Li et al., 2011] is based on Θ( 2 (1 − γ ) 2 / ( V N )) , and after O ( 2 (1 − γ ) 2 ln ) visits to such state–actions, all models but the true one will be…”
Sample Efficient Reinforcement Learning with Gaussian Processes (2014, methodology): “The " Knows What it Knows " (KWIK) framework (Li et al., 2011) is a supervised learning framework for measuring the number of times a learner will admit uncertainty.”
Structure Learning for Safe Policy Improvement (2019, background): “The Knows What It Knows (KWIK) framework was developed to help RL agents stay aware of under-explored parts of the environment [Li et al., 2011].”
Efﬁcient Exploration for Dialogue Policy Learning with BBQ Networks & Replay Buffer Spiking (2016, background): “…provably efﬁcient exploration strategies are known for problems with ﬁnite states/actions or problems with nice structures [Kakade, 2003, Asmuth et al., 2009, Jaksch et al., 2010, Li et al., 2011], not much is known for the general case, especially when nonlinear function approximation is involved.”
Introspective perception: Learning to predict failures in vision systems (2016, background): “KWIK ‘Knows What It Knows’ frameworks [7] allow for active exploration which is beneficial in reinforcement learning problems.”