Does Coding in Pythonic Zen Peak Performance? Preliminary Experiments of Nine Pythonic Idioms at Scale

Link to pre-print: https://arxiv.org/abs/2203.14484

How to run

1. Extract pythonnic_performance.zip and move into the extracted directory.
2. Execute install.sh to install required dependencies.
3. Execute run.sh to start the experiment.
4. Execute python statistic.py to perform the statistical test and show data statistics.

Note that files in submitted_output are our experimental results shown in the paper.

Abstract

In field of data science, and for academics in general, the Python programming language is a popular choice,  mainly because of its libraries for storing, manipulating, and gaining insight from data.
Evidence includes the versatile set of machine learning, data visualization, and manipulation packages used for the ever-growing size of available data.
The Zen of Python is a set of guiding design principles that developers use to write acceptable and elegant Python code.
Most principles revolve around simplicity.
However, as the need to compute on large amounts of data, performance has become a necessity for the Python programmer.

The new idea in this paper is to empirically confirm whether writing the Pythonic way peaks performance at scale.
As a starting point, we conduct a set of preliminary experiments to evaluate nine Pythonic code examples by comparing the performance of both Pythonic and Non-Pythonic code snippets.
Our results reveal that writing in Pythonic idioms may save memory and time.
We show that incorporating list comprehension, generator expression, zip, and itertools.zip\_longest can save up to 7,000 MB and 32.25 seconds.
The results open more questions on how they could be utilized in a real-world setting.