Seerie: An Offline 1.7B-Parameter Assistant for Scam Prevention in India, and a Manual Audit of Its Failures

Seerie is a scam-prevention assistant for India that runs entirely on an Android phone, with no internet connection after the one-time model download and no conversation data leaving the device. It is a QLoRA fine-tune of Qwen3-1.7B on 36,669 conversational records in English, Hinglish and nine Indic scripts, deployed as a 1.11 GB 4-bit GGUF model through llama.cpp.

On 70 evaluation prompts in 14 categories, the automatic harness passes 63 answers (90.0%). A manual reading of every answer finds 58 safe (82.9%; 95% CI 72.4-89.9) and 12 wrong (17.1%), and the harness agrees with the human judgement barely above chance (Cohen's kappa = 0.10). Errors are concentrated in fabricated legal sections, websites and phone numbers, and in Hinglish prompts (11 of 47 wrong, against 1 of 23 in English). The model largely learned the protective behaviour in its training data (do not pay, do not share an OTP, report to the national helpline) but not the facts, and the automatic harness could not tell the difference.

This record contains the research paper (PDF and LaTeX source), the on-device model (seerie-q4_k_m.gguf, 4-bit GGUF, 1.11 GB - the same file the Seerie Android app runs), the complete per-answer manual audit, the raw evaluation outputs, the audit script and figures. See README.md for details, intended use and limitations.

Seerie is a defensive tool. Its answers are not legal, financial or medical advice; in India, report cyber fraud to the national helpline 1930 or cybercrime.gov.in.