MemProf - Memory Allocation Profiling Tool for Real-World Applications

Project Specification

This project aims to build a memory allocation profiling on top of the popular dynamic analysis

framework Valgrind, able to detect heap memory waste or misusage. The project is divided in

several components:

1. Track allocations / deallocations of the client program – create a Valgrind plug-in tool able

to detect when the target program performs a heap memory operation (allocation / deallocation)

2. Collect and store relevant information about the allocations / deallocations – for each

operation keep the call parameters and other meta-information about its context.

3. Implement compression on the output analysis data – the tool should provide an option to

output compressed analysis data instead of plain text.

The result of this project is a plug-in tool for Valgrind which creates memory usage statistics for

any real-world application. This project aims to be part of Valgrind official repository.

Abstract

The Large Hadron Collider (LHC) experiments produce a vast amount of data and its throughput is even increasing in time. Scalability is the main keyword and adding more hardware often solves the problem but is never the most cost efficient solution. Technology permitted us to build increasingly stronger CPUs and there are several tools that aid in understanding their utilization and suggest possible optimizations. The memory represents the main bottleneck since its bandwidth is limited and there are only few tools focusing on profiling memory efficiency. MemProf is a memory allocation profiling tool built on top of Valgrind, a very popular open source dynamic analysis framework, thereby taking benefit both from already existing components and high maintenance from its large number of contributors. Moreover, Valgrind’s architecture permits the tool to access more fine-grained features such as support for multi-threaded programs and memory access tracking. The tool has been run over several standard Linux programs such as unzip, telnet, netstat, evince and is currently optimized for analysing larger applications. This paper presents the development process of this project, the current status and potential future extensions.