DISTILL: Low-Overhead Data-Driven Techniques for Filtering and Costing Indexes for Scalable Index Tuning

By Tarique Siddiqui, Wentao Wu, Vivek R. Narasayya, Surajit Chaudhuri. 2022

36 citations recorded by the source corpus.

Many database systems offer index tuning tools that help automatically select appropriate indexes for improving the performance of an input workload. Index tuning is a resource-intensive and time-consuming task requiring expensive optimizer calls for estimating the cost of queries over potential index configurations. In this work, we develop low-overhead techniques that can be leveraged by index tuning tools for reducing a large number of optimizer calls without making changes to the tuning algorithm or to the query optimizer. First, index tuning tools use rule-based techniques to generate a large number of syntactically-relevant indexes; however, a large proportion of such indexes are spurious and do not lead to a significant improvement in the performance of queries. We eliminate such indexes much earlier in the search by leveraging patterns in the workload, without making optimizer calls. Second, we learn cost models that exploit the similarity between query and index configuration pairs in the workload to efficiently estimate the cost of queries over a large number of index configurations using fewer optimizer calls. We perform an extensive evaluation over both real-world and synthetic benchmarks, and show that given the same set of input queries, indexes, and the search algorithm for exploration, our proposed techniques can lead to a median reduction in tuning time of 3X and a maximum of 12X compared to state-of-the-art tuning tools with similar quality of recommended indexes.

Subfields: Artificial Intelligence, Computer Networks and Communications, ML Systems & Hardware, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Jan Leike

Citations in the corpus

Self-critiquing models for assisting human evaluators (2022, methodology): “Overall, we believe there is potential to do empirical experiments for scalable oversight with today’s models, using schemes similar to reward modeling [LKE18] or debate [IA19].”
This is Going to Sound Crazy, But What If We Used Large Language Models to Boost Automatic Database Tuning Algorithms By Leveraging Prior History? We Will Find Better Configurations More Quickly Than Retraining From Scratch! (2025, unknown): “7 and 8. design [6, 22, 36, 70, 85]), holistic tuners that reason across multiple DBMS aspects [91], and large language model (LLM)-based tuners [27, 31, 88].”