818592
doi
10.5281/zenodo.818592
oai:zenodo.org:818592
user-empirical-software-engineering
Moseler, Oliver
University of Trier
Beck, Fabian
VISUS, University of Stuttgart
Diehl, Stephan
University of Trier
How Developers Locate Performance Bugs — Supplementary Material
Baltes, Sebastian
University of Trier
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
performance bugs
debugging
empirical study
<p><strong>Abstract:</strong></p>
<p><em>Background:</em> Performance bugs can lead to severe issues regarding computation efficiency, power consumption, and user experience. Locating these bugs is a difficult task because developers have to judge for every costly operation whether runtime is consumed necessarily or unnecessarily. Objective: We wanted to investigate how developers, when locating performance bugs, navigate through the code, understand the program, and communicate the detected issues.</p>
<p><em>Method:</em> We performed a qualitative user study observing twelve developers trying to fix documented performance bugs in two open source projects. The developers worked with a profiling and analysis tool that visually depicts runtime information in a list representation and embedded into the source code view.</p>
<p><em>Results:</em> We identified typical navigation strategies developers used for pinpointing the bug, for instance, following method calls based on runtime consumption. The integration of visualization and code helped developers to understand the bug. Sketches visualizing data structures and algorithms turned out to be valuable for externalizing and communicating the comprehension process for complex bugs.</p>
<p><em>Conclusion:</em> Fixing a performance bug is a code comprehension and navigation problem. Flexible navigation features based on executed methods and a close integration of source code and performance information support the process.</p>
<p><strong>Dataset:</strong></p>
<ol>
<li>
<p><strong>Tutorial:</strong> We provide the slides (PDF) and the video (MP4) we used in the tutorial phase of our study.</p>
</li>
<li>
<p><strong>Locating Bugs:</strong> We also provide supplementary material for each research question. We provide the advices we prepared for each bug in case a team got stuck (PDF); the questions we asked after each bug fixing session can be found on the introduction slides (PDF).</p>
<ul>
<li>
<p><strong>RQ1:</strong> Navigating and Understanding</p>
<ul>
<li>
<p><strong>RQ1.1:</strong> <em>How was information from the profiling tool or other parts of the IDE used to locate the performance bug?</em> Cross-case analysis (in German) (XLSX+ODS)</p>
</li>
<li>
<p><strong>RQ1.2:</strong> <em>Is the in-situ visualization of the profiling data beneficial compared to a traditional list representation?</em> Cross-case analysis (in German) (XLSX+ODS)</p>
</li>
<li>
<p><strong>RQ1.3:</strong> <em>What navigation strategies do developers pursue to locate a specific performance bug?</em> Interaction logs (TXT), Navigation visualizations (SVG), Screen recordings for Bug 3 (MP4, without audio because of confidentiality)</p>
</li>
</ul>
</li>
<li>
<p><strong>RQ2:</strong> Understanding and Communicating</p>
<ul>
<li>
<p><strong>RQ2.1:</strong> <em>How do developers communicate with each other when locating a performance bug?</em> Coding (XLSX+ODS), Sketches (PDF), Screen recordings for Bug 3 (MP4, without audio because of confidentiality)</p>
</li>
<li>
<p><strong>RQ2.2:</strong> <em>Could sketches help to understand and communicate a performance bug?</em> Coding (XLSX+ODS), Sketches (PDF), Cross-case analysis (in German) (XLSX+ODS), Sketching videos for Bug 3 (MP4, without audio because of confidentiality)</p>
</li>
</ul>
</li>
</ul>
</li>
<li>
<p><strong>Questionnaire:</strong> The questionnaire that the participants filled out at the end of the study can be found here (PDF).</p>
</li>
</ol>
Zenodo
2017-06-26
info:eu-repo/semantics/other
818591
user-empirical-software-engineering
1579893974.576465
18062
md5:cee28c7fd109a55380157b05afb352bd
https://zenodo.org/records/818592/files/cross_case_analysis.xlsx
50579
md5:a63da567f5f9b98c1389d0d3ab1e4d1c
https://zenodo.org/records/818592/files/coding_bug3.xlsx
34407
md5:3c54b63b8d7374f76adcbae75d7681e9
https://zenodo.org/records/818592/files/coding_bug3.ods
10631
md5:57ce930e8122155a12dbeb540a0f3fed
https://zenodo.org/records/818592/files/cross_case_analysis.ods
34022
md5:7cfcc232610d4705561e13f779bfcebe
https://zenodo.org/records/818592/files/interaction-logs_group4.log
2798814
md5:e4d00e8efc7fefd9831adc0594dbd830
https://zenodo.org/records/818592/files/sketches_group6.pdf
1529140
md5:adadf3983ce27074a445394ff1d08e0f
https://zenodo.org/records/818592/files/tutorial.pdf
130869220
md5:46eea210dc9511b0ae763d2ba46d4a47
https://zenodo.org/records/818592/files/group1_bug3_sketching_video_only.mp4
38254
md5:506b0dab31be7da95d1c3a01086262cb
https://zenodo.org/records/818592/files/interaction-logs_group1-2.log
9449
md5:e8b79428f2402436fbb337cab10a01d3
https://zenodo.org/records/818592/files/interaction-logs_group1-1.log
12320
md5:13f9f72de5c86737f166a42253ec2cd9
https://zenodo.org/records/818592/files/interaction-logs_group1-3.log
54603
md5:ec02d087f3664c3dc35e188747cb65ea
https://zenodo.org/records/818592/files/hints_performance_test_4.pdf
40934
md5:6d6a5d16309222d736bdd89e58c80835
https://zenodo.org/records/818592/files/interaction-logs_group2.log
77086
md5:e1640e62e6e848e0726bc7d63d90855e
https://zenodo.org/records/818592/files/hints_performance_test_3.pdf
52514
md5:45a1644e52bb1402061ac079eef98c29
https://zenodo.org/records/818592/files/interaction-logs_group3.log
52708
md5:a63993646cfb6450025ea7a0c16f4155
https://zenodo.org/records/818592/files/interaction-logs_group5.log
26809
md5:8c5d0f633f536c0b156be837e32b38fd
https://zenodo.org/records/818592/files/interaction-logs_group6.log
78083
md5:64fabbbdfabd924a72175fc2e8e3d866
https://zenodo.org/records/818592/files/hints_performance_test_2.pdf
77188
md5:7dce095553efc643f3d197a5716ac6b9
https://zenodo.org/records/818592/files/hints_performance_test_1.pdf
244018183
md5:d833faa49a785db059ee26db5cd0cbad
https://zenodo.org/records/818592/files/group6_bug3_sketching_video_only.mp4
148936455
md5:fb7284bb485f7bfd8da876df9f7857f0
https://zenodo.org/records/818592/files/group6_bug3_screencapture_video_only.mp4
112712095
md5:c951398286adca689c3957e477b4506d
https://zenodo.org/records/818592/files/group5_bug3_sketching_video_only.mp4
125852372
md5:6cb648bb93f2e065af03c297d59fd090
https://zenodo.org/records/818592/files/group5_bug3_screencapture_video_only.mp4
199976623
md5:28a366cf402310daf78240710114d059
https://zenodo.org/records/818592/files/group4_bug3_sketching_video_only.mp4
125789356
md5:c561e6f5c35c14186d330290f51bc402
https://zenodo.org/records/818592/files/group4_bug3_screencapture_video_only.mp4
5807701
md5:ab4d77ef1eb8fd4d288af94c9aab60c9
https://zenodo.org/records/818592/files/introduction_to_sampling_approach.mp4
21066
md5:4c42fb02142d5074985a0fc7869eb987
https://zenodo.org/records/818592/files/navigation-vis_group1_perf_bug_3.svg
17840
md5:4ecb6f2e277b6435786a9b03122ed7e0
https://zenodo.org/records/818592/files/navigation-vis_group2_perf_bug_3.svg
89895836
md5:42a687c902b65ebbab90bbdafbf1e1ee
https://zenodo.org/records/818592/files/group3_bug3_sketching_video_only.mp4
76411871
md5:0ac484c284cccb3555a0b5fe4b3c5398
https://zenodo.org/records/818592/files/group3_bug3_screencapture_video_only.mp4
72716899
md5:9fc83d92a60ee0a8ec2125c7bc2c7fb3
https://zenodo.org/records/818592/files/group2_bug3_sketching_video_only.mp4
22598
md5:9d3b594ae6de362b047752419938da66
https://zenodo.org/records/818592/files/navigation-vis_group3_perf_bug_3.svg
71568942
md5:a8df4dcdc0d8254323699a56f931b3e8
https://zenodo.org/records/818592/files/group2_bug3_screencapture_video_only.mp4
17189
md5:02af9fb4fe8da0f694034f1017f29d41
https://zenodo.org/records/818592/files/navigation-vis_group4_perf_bug_3.svg
22579
md5:2e221bb90a7f336b484891bf7f97678c
https://zenodo.org/records/818592/files/navigation-vis_group5_perf_bug_3.svg
10459
md5:5176c70fc067988789fa2cceadf04f03
https://zenodo.org/records/818592/files/navigation-vis_group6_perf_bug_3.svg
65002
md5:db302be7ea40f36d23967e5345ad7ec5
https://zenodo.org/records/818592/files/questionnaire.pdf
3559870
md5:1d7f7e4b1ae304e05f72c9584d1492f1
https://zenodo.org/records/818592/files/sketches_group1.pdf
2502344
md5:e46fe0e8e51594d81704de4500cc8289
https://zenodo.org/records/818592/files/sketches_group2.pdf
2530912
md5:526a2cba11283fd740fec334790aada9
https://zenodo.org/records/818592/files/sketches_group3.pdf
1087740
md5:41cc7fe20b1c6a60d431c766c7ac9c86
https://zenodo.org/records/818592/files/sketches_group4.pdf
1794502
md5:aeec6bb9b198676e0f51a2879d5e61ea
https://zenodo.org/records/818592/files/sketches_group5.pdf
97735556
md5:8b2eb6439a1b6ea16fb053ab5da772af
https://zenodo.org/records/818592/files/group1_bug3_screencapture_video_only.mp4
public
10.5281/zenodo.818591
isVersionOf
doi