Array programming with NumPy
By Charles R. Harris, K. Jarrod Millman, S. Van Der Walt, R. Gommers, Pauli Virtanen, D. Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith and 17 more. 2020
21,287 citations recorded by the source corpus.
Array programming provides a powerful, compact and expressive syntax for accessing, manipulating and operating on data in vectors, matrices and higher-dimensional arrays. NumPy is the primary array programming library for the Python language. It has an essential role in research analysis pipelines in fields as diverse as physics, chemistry, astronomy, geoscience, biology, psychology, materials science, engineering, finance and economics. For example, in astronomy, NumPy was an important part of the software stack used in the discovery of gravitational waves1 and in the first imaging of a black hole2. Here we review how a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring and analysing scientific data. NumPy is the foundation upon which the scientific Python ecosystem is constructed. It is so pervasive that several projects, targeting audiences with specialized needs, have developed their own NumPy-like interfaces and array objects. Owing to its central position in the ecosystem, NumPy increasingly acts as an interoperability layer between such array computation libraries and, together with its application programming interface (API), provides a flexible framework to support the next decade of scientific and industrial analysis. NumPy is the primary array programming library for Python; here its fundamental concepts are reviewed and its evolution into a flexible interoperability layer between increasingly specialized computational libraries is discussed.
Subfields: Artificial Intelligence, Information Systems and Management, Materials Chemistry, Numerical Methods & Scientific Computing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Quoc V. Le
- Percy Liang
- Luke S. Zettlemoyer
- Jianfeng Gao
- Yoshua Bengio
- Graham Neubig
- Kyunghyun Cho
In Information Systems and Management
In Materials Chemistry
- Frank Hutter
- Marius Thomas Lindauer
- Stefan Falkner
- Matthias Feurer
- Xuanyi Dong
- Katharina Eggensperger
- Michael Heinzinger
- Jure Leskovec
In Numerical Methods & Scientific Computing
- S. Nissanke
- Sebastian M. Gaebel
- Simon Stevenson
- Daniel Foreman-Mackey
- James Lee-Thorp
- Ali Farhadi
- Frank Hutter
- Jamie Callan
Citations in the corpus
- Scaling Up Models and Data with t5x and seqio (2022, background): “JAX (Bradbury et al., 2018; Frostig et al., 2018) is uniquely positioned to provide such benefits; its NumPy-like (Harris et al., 2020) API makes it easy to understand and develop, while the jax.pjit API backed by XLA GSPMD (Xu et al., 2021) provides a powerful and efficient compiler-based programming…”
- Fairseq S2T: Fast Speech-to-Text Modeling with Fairseq (2020, methodology): “Speech features can also be pre-computed and stored in NumPy (Harris et al., 2020) format.”
- High-Performance Large-Scale Image Recognition Without Normalization (2021, methodology): “Our training code is available at https://github.com/deepmind/ deepmind-research/tree/master/nfnets, and is written using numpy (Harris et al., 2020), JAX (Bradbury et al., 2018), Haiku (Hennigan et al., 2020), and the DeepMind JAX Ecosystem (Babuschkin et al., 2020).”
- PyCIL: a Python toolbox for class-incremental learning (2021, methodology): “Dependencies: PyCIL relies on open-source libraries such as NumPy and SciPy for linear algebra and optimization problems.”
- General-purpose, long-context autoregressive modeling with Perceiver AR (2022, neutral): “, using NumPy-style indexing notation (Harris et al., 2020).”
- Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022, methodology): “Robustness Gym [23] and Gensim [58] are primarily used to handle data inputs and outputs between the cards visual components, and are supported by other packages (nltk [9], numpy [27], pandas [42], sklearn [53]).”
- The Astropy Project: Sustaining and Growing a Community-oriented Open-source Project and the Latest Major Release (v5.0) of the Core Package (2022, methodology): “…and bulk of the maintenance of the Astropy Project components has primarily been sustained by a small number of heavily engaged or explicitly employed contributors (e.g., Astropy Collaboration et al. 2018), as is the case in many large, open-source software communities (e.g., Harris et al. 2020).”
- ClimaX: A foundation model for weather and climate (2023, positive): “Software and hardware stack We use PyTorch (Paszke et al., 2019), timm (Wightman, 2019), numpy (Harris et al., 2020) and xarray (Hoyer & Hamman, 2017) to manage our data and model training.”
- JCVI: A versatile toolkit for comparative genomics analysis (2024, background): “…also owes its success to the versatile array programming interface provided by libraries like NumPy, SciPy, and PyTorch that facilitate writing performant code [9] and enable a natural way of handling large data sets that are even non ‐ numeric and of variable length, such as DNA sequences [10].”