Artifacts supplementing the ACM DTRAP 2020 article "Will You Trust This TLS Certificate? Perceptions of People Working in IT (extended version)"

These research artifacts supplement the following two publications:

• Will You Trust This TLS Certificate? Perceptions of People Working in IT [ACSAC 2019], DOI 10.1145/3359789.3359800, more details at https://crocs.fi.muni.cz/public/papers/acsac2019
• Will You Trust This TLS Certificate? Perceptions of People Working in IT (extended version) [ACM DTRAP 2020], DOI 10.1145/3419472, more details at https://crocs.fi.muni.cz/public/papers/dtrap2020

The artifacts contain the full experimental setup (as described in Section 2.1 of the paper) and the complete anonymized dataset underlying the evaluation presented in Sections 3 and 4.

The experimental setup contains the documents accompanying the task: the informed consent, pre-task questionnaire, task description, trust scales, and the list of questions posed during the post-task interview (all in PDFs). We further include the custom website with certificate validation documentation for the “redesigned” condition (static HTML). While working on the task, participants in the “redesigned” condition could access this website via a link that was in the redesigned error messages. Furthermore, we provide the software with which the participants interacted.  It contains the displayed error messages and validated certificates. These things are available both individually and incorporated in a snapshot of a virtual machine used at the experiment (importable directly into VirtualBox).

The collected data is presented in a single dataset (SPSS format; you can use PSPP as a free alternative). It includes the analysis syntax files to obtain the numerical results presented in the paper. For each participant, the dataset contains: 1) pre-task questionnaire answers, 2) reported trust ratings, 3) sub-task timing, 4) information on whether they browsed the Internet and 5) the interview codes assigned. Note that we do not publish the interview transcripts to preserve participant privacy.