Data Structures for Statistical Computing in Python
By Wes McKinney. 2010
9,140 citations recorded by the source corpus.
In this paper we are concerned with the practical issues of working with data sets common to finance, statistics, and other related fields. pandas is a new library which aims to facilitate working with these data sets and to provide a set of fundamental building blocks for implementing statistical models. We will discuss specific design issues encountered in the course of developing pandas with relevant examples and some comparisons with the R language. We conclude by discussing possible future directions for statistical computing and data analysis using Python.
Subfields: Artificial Intelligence, ML Systems & Hardware
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Yoshua Bengio
- Percy Liang
- Jianfeng Gao
- Quoc V. Le
- Kyunghyun Cho
- Yonatan Belinkov
In ML Systems & Hardware
- Ali Ayub
- Alan R. Wagner
- Yoshua Bengio
- Andrew S. Rosen
- Misha Denil
- Ross D. Markello
- Bratislav Misic
- Danfei Xu
Citations in the corpus
- Array programming with NumPy (2020, background): “Now, fifteen years later, NumPy underpins almost every Python library that does scientific or numerical computation including SciPy [13], Matplotlib [14], pandas [15], scikit-learn [16], and scikit-image [17].”
- PyTorch: An Imperative Style, High-Performance Deep Learning Library (2019, methodology): “Third, with the advent of the free software movement, the scientific community moved away from closed proprietary software such as Matlab[9], and towards the open-source Python ecosystem with packages like NumPy [12], SciPy [19], and Pandas [20].”
- Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face (2023, methodology): “We include wrappers to load database tables, pandas DataFrames [26, 32], and text datasets on disk, as well as the ability to load any dataset either in memory-mapped mode or in streaming mode: the former makes it possible to handle larger-thanmemory datasets, and the latter larger-than-disk dataset”
- Statistical Arbitrage in Cryptocurrency Markets (2019, background): “Data preparation mostly relies on the packages numpy (van der Walt et al. 2011) and pandas (McKinney 2010), which are powerful tools for handling large amounts of data.”
- Garbage in, garbage out?: do machine learning application papers in social computing report where human-labeled training data comes from? (2019, positive): “All computational analysis and scripting was conducted in Python 3.7 [65], using the following libraries: Pandas dataframes [36] for data parsing and transformation; SciPy [28] and NumPy [64] for quantitative computations; and Matplotlib [25] and Seaborn [67] for visualization.”
- SCANPY: large-scale single-cell gene expression data analysis (2018, methodology): “SCANPY’s technological foundations SCANPY’s core relies on NUMPY [33], SCIPY [34], MATPLOTLIB [35], PANDAS [36], and H5PY [37].”
- Applications of machine learning methods for engineering risk assessment – A review (2020, methodology): “For example, Mojaddadi et al. (2017) propose an ensemble machine-learning approach to determine the level of risk of flood for a given geographical area.”
- Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches (2021, background): “, 2020), pandas (McKinney, 2010; pandas development team, 2020), and SciPy (Virtanen et al.”
- PyPSA: Python for Power System Analysis (2017, background): “PyPSA stores all data about network components in the DataFrame objects of the Python library pandas [50].”
- Darts: User-Friendly Modern Machine Learning for Time Series (2021, methodology): “TimeSeries can be indexed either with Pandas DatetimeIndex or RangeIndex (Wes McKinney, 2010).”