Solving Linear Algebra by Program Synthesis

By Iddo Drori, Nakul Verma. 2021

20 citations recorded by the source corpus.

We solve MIT's Linear Algebra 18.06 course and Columbia University's Computational Linear Algebra COMS3251 courses with perfect accuracy by interactive program synthesis. This surprisingly strong result is achieved by turning the course questions into programming tasks and then running the programs to produce the correct answers. We use OpenAI Codex with zero-shot learning, without providing any examples in the prompts, to synthesize code from questions. We quantify the difference between the original question text and the transformed question text that yields a correct answer. Since all COMS3251 questions are not available online the model is not overfitting. We go beyond just generating code for questions with numerical answers by interactively generating code that also results visually pleasing plots as output. Finally, we automatically generate new questions given a few sample questions which may be used as new course content. This work is a significant step forward in solving quantitative math problems and opens the door for solving many university level STEM courses by machine.

Subfields: Artificial Intelligence, Computer Science Applications, Information Systems, Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Information Systems

In Large Language Models

In Natural Language Processing

Citations in the corpus

Competition-level code generation with AlphaCode (2022, methodology): “Codex was used to build interactive program synthesis systems that are capable of solving university-level linear algebra and probability and statistics questions in (Drori and Verma, 2021; Tang et al., 2021), and further used to create an advanced autocomplete system in GitHub Copilot.”
GitHub Copilot AI pair programmer: Asset or Liability? (2022, methodology): “Drori and Verma [16] studied Copilot’s capability in solving linear algebra problems for the MIT linear algebra course.”
A Survey on Pretrained Language Models for Neural Code Intelligence (2022, methodology): “Notably, Codex (Chen et al., 2021) has made a signiﬁcant milestone in program synthesis and has empowered programming intelligence tools like Copilot 2 , which have even been applied to solving linear algebra and math word problems (Tang et al., 2021; Drori and Verma, 2021).”