Question,Group,Codes,Name,Summary,Extra comments
Q17,,,,,
Q20,1,"as much as possible, the higher the better + everything should be covered + high number + never lower the coverage",High number,Goal is to have high coverage,
Q20,2,cover all code changes + cover important paths + coverage in hot spots + test complex code + test core code + test edge cases + test user facing code,Important code only,Goal is to cover important parts of the code. Trivial code can be uncovered,
Q20,3,depends on input/output space + depends on the project + depends on type of testing,It depends per project,Coverage goal depends on multiple factors. Lot of different variables,
Q20,4,good reason needed for leaving code uncovered,Cover sensibly,"Goal is to have everything covered in a sensible matter, where uncovered items are justified properly",
Q20,5,100% not worth it + maintaining should take reasonable effort + no point in reaching specific coverage + quality of tests more important + too much coverage has no value,Quality more important,"Not a lot of focus on coverage, since it distracts from actually writing high quality code/tests",
Q20,,,Misc.,,
Q27,1,asked maintainers about tests before writing them + asked to add unhappy paths + changed code was already known to cause bugs + difficult CI set up + don't always write tests + lack of interest in testing + strong testing culture + tests added only after implementation was approved to reduce workload + tests requested using a different method + tests were requested by maintainer,Maintainer interaction,Clear interaction with maintainers,
Q27,3,continuously raising coverage until it passes + coverage can be misleading + coverage changes unexpectedly + coverage tools are annoying + coverage tools provide incentive + depends on the issue + mutation tests are very important + refactored code + tests were modified + tool informed uncovered lines + visibly uncovered lines give incentive,Tool interaction,Interaction with coverage tool,
Q27,5,coverage was needed on tests themselves + each PR comes with its own tests + tests were added,Unclear interaction,Unclear interaction but coverage got fixed,
Q27,6,missing tests due to lack of understanding + not easily testable,Difficulty testing,Difficult to test,
Q27,,,Misc.,,
Q29,1,CI blocking merge + PR gets closed or ignored + blocking merge without tests + no review until CI passing],Blocked merge,Blocking merge,
Q29,2,asked for coverage and it was added + asking for tests works better than asking for coverage + contributors add tests + coverage added in a later PR + maintainer writes the tests + resulting tests found a bug,Problem fixed,Problem got fixed,
Q29,3,code coverage tools not working properly + coverage tool isnt everything,Coverage bad,Coverage bad,
Q29,4,contributors admit to not testing beforehand + contributors give up + contributors try to circumvent testing + new contributors dont add tests + no tests + noticeable when contributors dont add tests,Contributors lacking,Contributors lacking (for lack of a better word),
Q29,5,contributors ask for help + contributors can ask for exceptions + coverage not enforced + depends on developer experience whether they understand coverage + giving examples of similar tests as a guide + guidelines were written,Interaction,Interaction between contributors and maintainers,
Q29,6,coverage exposed untested paths + coverage tool shows you take coverage seriously,Coverage good,Coverage good,
Q29,,,Misc.,,
Q31,1,ask to test + comment + conversations + giving reminders + maintainer interaction,Interaction w/ comments,Interaction through PR comments,
Q31,2,blocking merge + negative feedback + past incidents caused by coverage issues + the cost of adding it later + user reputation,Negative impacts,Negative impacts,
Q31,4,coverage guidelines + development culture + expecting coverage + lead by example + make it a goal + policy + review process + starting at 100% + take it seriously,Clear expectations,Setting expectations,
Q31,5,coverage report + coverage tools + document and track coverage reductions + visual indications,Coverage tool,General (coverage) tools usage,
Q31,6,email + notification,Notifications,Notifications,
Q31,7,avoid incorrect coverage data + focus coverage efforts only on areas where they are needed + focus on the area where code changes were made + keep PRs small + make easily testable code,Focused testing,Clear/Focussed testing efforts,
Q31,8,gamification + provide assistance + thank people,Positive reinforcement,Positive reinforcement,
Q31,9,having confidence code is correct with higher coverage + looks good to outsiders + more tests means less bugs,Good feeling,Feeling better about the code (quality),
Q31,10,teaching/studying software testing + understanding the importance of well tested code,Deeper understanding of necessity,Understanding of the necessity of well-tested code,
Q31,,,Misc.,"",
Q35,1,coverage irrelevant + coverage unrelated to diff + depends on coverage change + depends on how well project was covered before + failure insignificant + irrelevant tests + minor changes + minor coverage change + simple code + unimportant lines uncovered,Trivial change,Trivial/negligible coverage change,
Q35,2,CI provider failure + coverage incorrect + coverage unclear + library/dependency failure + refactoring breaks coverage + tool failure,Failure elsewhere,Failure elsewhere,
Q35,3,future tests promise + improve coverage at the end + maintainer writes tests themselves + until the PR is ready for review,Future fix,Ensurance tests are added/coverage will be fixed later,
Q35,4,expensive + improving coverage lacks value + laziness + legacy project,Not worth fixing,Unwillingless to expend effort,
Q35,5,desirable feature > tests + time pressure,Priority,Priorities,
Q35,6,complex PR + difficulty testing + education,Complexity,Complexity,
Q35,7,"check is a reminder, not a rule + code coverage =/= test quality + manual review good enough + not mandatory + other form of quality control",Other means,Ensuring quality is done through other means than coverage,
Q35,8,private code + project is bound to change,Not production code,Code not meant for production (yet),
Q35,9,not scare new contributors,Not scaring contributors,Not scaring contributors,
Q35,10,coverage change justified in PR/documentation,Change justified,Change justified,
Q35,,,Misc.,,
Q42,1,accountability + keeps people honest + user reputation,Honesty,Honesty of the developer,
Q42,2,automated + integrated into workflow + reports back to repo,Automation,Automation,
Q42,4,awareness of other peoples code,Awareness,Awareness of other people's contributions,
Q42,5,avoids backlogs + catch faulty tests + determine safe and dangerous areas of code + find potential improvements + find uncovered code + find uncovered lines + helps finding bugs + helps finding dead code + helps finding untested code + helps refactoring + helps writing tests + indicated uncovered lines,Finding improvements,Finding improvements/weaknesses,
Q42,6,community support,Community,Community,
Q42,9,easier contributing + easier reviewing + easier triage + faster development + quick feedback,Easier work,Easier/faster work,
Q42,10,easy set-up and/or use + flexible configuration + free to use,Low entry barrier,Low entry barrier,
Q42,11,encourages testing + encourages writing better code,Encourages testing,Encourages testing,
Q42,12,ensure code is tested + faith in project + feeling of safety + feeling of trust + gives confidence + guarantee code is executed + indicates reliability + indicates testing + sense of security + test quality indicator + visual coverage indicator,Guarantees,Safety guarantees?,
Q42,16,gamification,Gamification,Gamification,
Q42,17,graphics + nice design,UI/UX Design,Design,
Q42,18,coverage in one place + historical data + information + insight + insight into specific changes + measures quality + metrics + multiple forms of coverage + overview of coverage situation,Collected information,Collection of information,
Q42,19,impossible to lower coverage,Prevents coverage decrease,Prevents coverage from going down,
Q42,20,makes it look better + public + shows project cares,Looks good to public,Looks good to outsiders,
Q42,21,provides guidelines/protocol,Sets guidelines,Guidelines,
Q42,22,handy check + quality control + smaller chance to introduce bugs + software reliability,Quality control,Quality control,
Q42,23,teaching contributors about testing,Educational,Educational,
Q42,24,supports multiple languages + useful functions,Useful functions,Useful functions,
Q42,,,Misc.,,
Q43,1,can be ignored + once goal is reached people become lazy + you become numb/blind to it,Complacency,Complacency,
Q43,2,complex set-up + inflexible + poor documentation + requires overhead,Set-up,Set-up,
Q43,3,coverage treated as quality + false sense of security + focus on line coverage + overfitting the metric + writing/reviewing (unnecessary) tests takes time/effort,Mistaken for quality,Coverage treated as quality,
Q43,4,difficult to enforce + not everyone wants to use it,Reluctance to use,Reluctance to use,
Q43,5,false positives + flaky + lacks semantics + not robust to small changes,Unclear/wrong results,"Unclear, wrong or insignificant results",
Q43,6,frustrating + workflow interruptions,Annoying,Annoying,
Q43,8,hard to introduce later into a project + not useful in every stage,Not always applicable,Not always applicable,
Q43,9,hosted by a third-party + no direct integration + no explicit privacy settings + not free to use + potentially insecure + server can go offline,Third-party host,Third-party host concerns,
Q43,10,lack of support for language + lacking feature + lacking features,Lacking features,Lacking features,
Q43,12,hard to interpret results + no guidance + poor feedback + says nothing about quality of tests + unclear bigger picture,Poor feedback,Poor feedback,
Q43,13,arbitrary thresholds + noisy + redundant + verbose,Noisy,Noisy,
Q43,14,takes time,Takes time,Takes time,
Q43,15,too strict + unwelcoming,Strict,Strict,
Q43,16,crutch,Crutch,Crutch,
Q43,,,Misc.,,