Verification of Data Claims in the Paper

This document explains and verifies every quantitative claim in the manuscript "Computational Operationalization Choices for Spatial Data Quality in Screen-Based Eye Tracking" (accuracy-precision.tex).

All computations are collected in a single self-contained script. To run it, install the dependencies and execute from this directory:

unzip data.zip
pip install -r requirements.txt
python verify_paper_claims.py

The data files are distributed as data.zip. Unzip it first to create the data/ directory (including the data/microsaccade/ subdirectory).

Data Sources

All data files are in the data/ subdirectory:

The script loads the same gaze data referenced in the paper, reproduces each calculation, and compares the result with the number stated in the manuscript. Below we walk through every claim, explain what it means, and show the verification outcome.

Section 5.2 — Small-Angle Approximation

Paper (line 124):

"relative errors for the small-angle approximation grow with eccentricity but stay in the single-digit percentages below 25°"

What it means:
Three small-angle approximations are commonly used to simplify angular-to-Cartesian conversion: sin(theta) ~ theta, tan(theta) ~ theta, and cos(theta) ~ 1. The paper claims all three stay below 10 % relative error for eccentricities below 25 degrees.

Verification:

Result: CONFIRMED. Below 25 degrees, all three approximations stay in the single-digit percentages. At 20 degrees the largest error is cos ~ 1 at 6.4 %. At exactly 25 degrees, cos ~ 1 reaches 10.3 %.

Section 5.3 — Viewing-Distance Mismatch

Paper (line 158):

"a systematic shift in the low double digit percentages is conceivable"

What it means:
When the assumed viewing distance differs from the actual one, every angle computed from screen coordinates is systematically biased. A 60 cm assumption vs 66 cm actual maps to a ~10 % ratio.

Verification:

Result: CONFIRMED. The error is in the low double-digit range (~10 %) and grows with larger viewing-distance mismatches (e.g. 60->70 cm gives ~16 %).

Section 5.3 — Precision Doubling from Gaze-Origin Outliers

Paper (line 158):

"the SMI RED occasionally reports switched gaze origins for individual samples whose screen coordinates remain stable [...] Including these samples roughly doubled the computed precision value"

What it means:
The paper describes that per-sample 3D eye position (EPOS) data from the SMI RED can contain problematic values. When these are used to convert screen coordinates to angular values (instead of a fixed viewing distance), the outlier EPOS values inject additional variance into the angular data, inflating precision measures.

Verification (from smi_data.tsv):

The script identifies a concrete EPOS outlier (e.g., at index 705 in the recording, where L EPOS X jumps by +18.4 mm while the screen gaze POR X stays stable), then computes RMS-S2S precision for a 250-sample fixation interval containing that outlier, comparing fixed VD vs per-sample EPOS.

The single switched EPOS sample causes two large consecutive angular jumps in the RMS-S2S computation, inflating precision by a factor of ~2x.

Result: CONFIRMED. Precision ratio is 2.07x, consistent with "roughly doubled."

Section 5.4 — Constant Conversion Factor Overestimation

Paper (line 172):

"The relative error is at least that of the cos(theta) ~ 1 approximation ... and can reach slightly more than double that value"

What it means:
A constant pixel-to-degree factor (arctan(1 mm / VD)) converts Euclidean distances on the screen surface into degrees. At eccentric positions, this overestimates the true angular error. The minimum overestimation (tangential displacement at eccentricity theta) equals the cos(theta) ~ 1 error; the maximum (radial displacement from center) can approach twice that.

Verification:

Result: CONFIRMED. The tangential error matches cos ~ 1 (the floor), and the radial error approaches ~2x that value (the ceiling).

Section 5.5 — Coordinate-Wise vs Fréchet Median

Paper (line 193):

"In one specific case this yielded a trueness difference of 36 % (0.215 deg vs 0.158 deg)"

What it means:
When computing trueness, the centroid can be the coordinate-wise median (applied independently to azimuth and elevation) or the Fréchet median on the sphere (minimizing the sum of geodesic distances). These can give noticeably different results because the coordinate-wise median is not geometrically correct in angular space.

Verification (SMI RED 500, left eye, target 2, VD = 60 cm):

Result: CONFIRMED. Absolute values match and the percentage (35.7 % ~ 36 %) is consistent.

Section 5.6 — Fick vs Helmholtz Differences

Paper (line 220):

"the two parameterizations therefore yield different azimuth and elevation values ..., with the difference approximately following the cos(theta) ~ 1 curve"

What it means:
The same gaze direction, expressed in Fick vs Helmholtz angles, produces different numeric values for azimuth and elevation. The discrepancy grows with eccentricity, roughly tracking the cos(theta) ~ 1 error.

Verification:

The |Delta phi|/phi column shows the relative elevation difference, which closely tracks the theoretical cos(theta) ~ 1 error curve.

Result: CONFIRMED.

Section 5.6 — Coordinate-Axis Tilt

Paper (line 220):

"the coordinate-axis tilt is approximately phi * theta (e.g., approx 3.5 deg at 20 deg azimuth and 10 deg elevation)"

What it means:
In Fick coordinates, lines of constant elevation on the screen are not perfectly horizontal. They tilt by an amount proportional to the product of azimuth and elevation (both in radians). At (20 deg, 10 deg):

phi * theta  =  0.1745 rad * 0.3491 rad  =  0.0609 rad  =  3.49 deg

Verification:

Result: CONFIRMED.

Section 6 — Illustrative Example

Paper (lines 255-261):
Two pipelines process the same gaze data (485 samples, target at 16 cm right, 9 cm above screen center) with different operationalization choices.

Pipeline A

60 cm assumed VD, coordinate-wise median in screen mm, Euclidean distance with a constant deg/mm factor, target-centered decomposition.

Pipeline B

66 cm actual VD, Fréchet median on the sphere, geodesic distance, Fick decomposition (no recentering).

Results

Factor Decomposition

The 61 % overall difference can be decomposed multiplicatively:

The coordinate-wise median is the largest single factor (63 % of the total difference), confirming "more than half."

Section 6 — ETDQualitizer Reference Values

Paper (line 261):

"ETDQualitizer reports a total trueness (what ETDQualitizer calls accuracy) of 0.185 deg and Delta phi = 0.083 deg at the actual viewing distance of 66 cm."

What it means:
This is a reference comparison showing what ETDQualitizer produces for the same data.

Verification:

Result: MATCH. Using nanmedian as central tendency, ETDQualitizer reproduces the paper values.

Why ETDQualitizer computes trueness (not accuracy in the ISO sense)

Paper claims:

1. ETDQualitizer reports trueness (Section 6), despite the function being named accuracy().
2. ETDQualitizer effectively computes a coordinate-wise central tendency (Section 5.5).

Why "trueness":
ETDQualitizer's accuracy() function (__init__.py) works as follows:

1. Compute per-sample geodesic distance from each gaze point to the target (always positive).
2. Compute per-sample direction from target to gaze point on the tangent screen.
3. Decompose each offset into signed azimuth and elevation components: offset * [cos(direction), sin(direction)].
4. Apply the central tendency function (e.g. median) to the signed azimuth offsets and signed elevation offsets independently.
5. Return hypot(ct(azi_offsets), ct(ele_offsets)).

The key insight is that the offsets in step 3 are signed: gaze samples to the left of the target produce negative azimuth offsets, samples to the right produce positive ones. Taking the mean or median of signed offsets yields the centroid's displacement from the target — not the average absolute distance. By the ISO 5725-1 definitions used in the paper:

• Trueness = d(centr(g), t) — distance from centroid to target
• Accuracy = centr(d(g, t)) — central tendency of per-sample distances

mean(signed_offsets) gives the centroid offset (trueness), whereas mean(|offsets|) would give the average distance (accuracy). ETDQualitizer computes the former, so it reports trueness.

Why "coordinate-wise":
The central tendency function is applied independently to each coordinate direction (step 4): median(azi_offsets) and median(ele_offsets). This is equivalent to computing the coordinate-wise median of the gaze positions on the tangent screen and then measuring the offset from that centroid to the target. It is not a Fréchet median on the sphere (which would minimize the sum of geodesic distances jointly).

Section 4 — Microsaccade Analysis

Paper (line 107):

"microsaccades were infrequent during the selected time intervals and had negligible impact on accuracy and trueness"

"their per-sample displacement was small relative to the tracker's noise floor"

Method:
Microsaccades are detected using the Engbert & Kliegl (2003) algorithm with parameters from Nystrom, Niehorster et al. (2021):

• Velocity threshold: lambda = 6 (median-based SD)
• Minimum duration: 5 ms
• Minimum inter-saccade gap: 10 ms
• Bartlett (triangular) lowpass filter: 20 ms window
• 5-point central difference velocity computation

Data:
Six data files with sampling rates >= 200 Hz from the ETDQualitizer example data (the 120 Hz Tobii file is excluded due to insufficient temporal resolution for reliable microsaccade detection):

These are the same validation recordings used by ETDQualitizer's example scripts, recorded with a common screen configuration (528 x 297 mm, 1920 x 1080 px, viewing distance 650 mm) using 9 validation targets.

Verification:

Three sub-claims are verified:

Claim 1 — "infrequent"

Only 3 % of all samples fall within detected microsaccade intervals. The rate of 2-3 Hz is consistent with typical microsaccade rates during fixation (1-3 Hz in the literature).

Result: CONFIRMED. Microsaccades affect a small fraction of samples.

Claim 2 — "negligible impact on accuracy and trueness"

Both trueness (centroid offset from target, via ETDQualitizer.accuracy()) and ISO-accuracy (mean per-sample geodesic distance to target) are computed with all samples and again with microsaccade samples masked to NaN:

Result: CONFIRMED. Both mean changes are < 1 % when microsaccade samples are excluded.

Claim 3 (precision) — "per-sample displacement small relative to noise floor"

The tex explicitly scopes this claim to precision: "For precision, ... their per-sample displacement was small relative to the tracker's noise floor." The median per-sample displacement during microsaccades is compared to the RMS-S2S precision (noise floor) of the full interval:

Result: CONFIRMED. The median per-sample displacement during microsaccades is smaller than the tracker's RMS-S2S noise floor (ratio 0.84), meaning individual microsaccade steps are largely indistinguishable from normal sample-to-sample noise.

Summary Table