Artifact for the PETS 2026 paper: "WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks"

Published July 10, 2026 | Version v2

Software Open

Artifact for the PETS 2026 paper "WebSP-Eval: Evaluating Web Agents on Website

Security and Privacy Tasks" (Guruprasad Viswanathan Ramesh, Asmit Nayak,

Basieem Siddique, and Kassem Fawaz; University of Wisconsin–Madison).

WebSP-Eval measures how well LLM-powered web agents perform website security

and privacy tasks (managing cookies, configuring privacy settings, revoking

sessions, controlling newsletter/marketing preferences, etc.) on live websites

tied to user accounts.

Contents:

- dataset/ — 200 task instances (138 unique tasks across 28 websites; WithNav

and W/oNav prompt variants), ground-truth action sequences and UI-element

annotations, and PII-templatized initial-state (S0) traces.

- src/ — the web agent (adapted from WebVoyager) and the Selenium

record-and-replay state-reset engine.

- extension/ — Chrome MV3 record-and-replay extension for login/state traces.

- judge/ — automated MLLM-as-a-judge.

- docs/ — installation, account-setup, and recording guides;

ARTIFACT-APPENDIX.md documents the evaluated functional test.

This archive is a byte-identical snapshot of commit

83026e5a30bcacbacefd600953f9acb9b6c5a21c of

https://github.com/wi-pi/webspeval_code — the version that passed PETS 2026

artifact evaluation with the Available and Functional badges.

No accounts, credentials, or PII are included: the shipped state traces are

templatized with {{WEBSP_ACCOUNT_*}} placeholders, and users create their own

sock-puppet accounts and login traces (see docs/ACCOUNT_SETUP.md).

Licensing: the code (src/, extension/, judge/, tools/) is released under the

Apache License 2.0 (LICENSE-code); the dataset (dataset/) is released under CC

BY-NC 4.0 (LICENSE-dataset).

Project website: https://wiscprivacy.com/webspeval

Files

Name	Size	Download all
webspeval_code-1.0.0-83026e5.zip md5:0634df56d59e142d0097f87da4e773d5	2.5 MB	Preview Download