Triangulating Python Performance Issues with Scalene
By Emery D. Berger, Sam Stern, Juan Altmayer Pizzorno. 2022
26 citations recorded by the source corpus.
This paper proposes Scalene, a profiler specialized for Python. Scalene combines a suite of innovations to precisely and simultaneously profile CPU, memory, and GPU usage, all with low overhead. Scalene's CPU and memory profilers help Python programmers direct their optimization efforts by distinguishing between inefficient Python and efficient native execution time and memory usage. Scalene's memory profiler employs a novel sampling algorithm that lets it operate with low overhead yet high precision. It also incorporates a novel algorithm that automatically pinpoints memory leaks, whether within Python or across the Python-native boundary. Scalene tracks a new metric called copy volume, which highlights costly copying operations that can occur when Python silently converts between C and Python data representations, or between CPU and GPU. Since its introduction, Scalene has been widely adopted, with over 500,000 downloads to date. We present experience reports from developers who used Scalene to achieve significant performance improvements and memory savings.
Subfields: Artificial Intelligence, Program Analysis & Verification, ML Systems & Hardware
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Graham Neubig
- Uri Alon
- Yiming Yang
- Aman Madaan
- Parthasarathy Ranganathan
- Amir Yazdanbakhsh
- Alexander Shypula
- Milad Hashemi
In Program Analysis & Verification
In ML Systems & Hardware
Citations in the corpus
- Learning Performance-Improving Code Edits (2023, methodology): “SCALENE (Berger et al., 2022) designs a low-overhead Python-profiler that helps the developers to identify inefficiencies in their Python code and apply relevant optimization.”