A large-scale empirical study on mobile performance: Energy, Run-time and Memory

Software performance concerns have been attracting research interests at an increasing rate, especially regarding energy performance in non-wired computing devices. In the context of mobile devices, several research works have been devoted to assessing the performance of software and its underlying code. One important contribution of such research are sets of programming guidelines aiming at identifying efficient and inefficient programming practices, and consequently to steer software developers to write performance-friendly code. The most common programming guidelines resulting from these studies are the labeling of APIs and coding practices as efficient or inefficient in terms of performance.

Despite recent efforts in this direction, it is still almost unfeasible to obtain universal and up-to-date knowledge regarding a software’s performance. Namely regarding energy performance, where there has been growing interest in optimizing software energy consumption due to the power restrictions of such devices. There are still many difficulties reported by the community on measuring performance, namely in large-scale validation and replication.

In this paper,  we analyze the execution of a diversified corpus of applications of significant magnitude. We analyze the source-code performance of 1322 versions of 215 different Android applications, dynamically executed with over than 27900 tested scenarios. Our empirical analysis allowed to observe that certain coding practices previously identified as energy-greedy only replicate such behavior in specific contexts and have different impacts across different performance indicators. We also demonstrate that there are significant differences in terms of performance between the most used libraries suited for implementing common programming tasks, such as HTTP communication, JSON manipulation, image loading/rendering, among others, providing a set of recommendations to select the most efficient library for each performance indicator. Finally, we present a set of guidelines that can be used by practitioners to replicate energy studies and build more efficient mobile software.