Multilingual Voice AI for Pilgrim Assistance: STT Reliability, Bias by Invisibility, and Ethical Deployment in Sacred Environments
Description
The deployment of voice AI assistants in multilingual, safety-critical environments presents challenges that extend beyond prompt engineering. Building on prior work that proposed the SACRED Framework through two proof-of-concept voice assistants deployed at Masjid al-Haram (Fareed, 2026), this study addresses two previously identified gaps: multilingual speech recognition reliability and deployment oversight. Three commercial speech-to-text providers were evaluated within a production voice orchestration platform. Structured language routing achieved reliable transcription for English, Urdu, and Bahasa Indonesia, while Arabic remained unresolvable across four configurations and eight test calls. A structured field level evaluation across 41 calls in Urdu and Bahasa Indonesia found that identification numbers, the field most directly tied to a pilgrim's identity, were correctly relayed to staff in only 15 percent of Urdu calls and 52 percent of Bahasa Indonesia calls at final confirmation, with many of these errors traced specifically to the assistant's response generation rather than to speech recognition itself. A supplementary check using a second speaker produced results in a similar range, offering limited but direct evidence that this unreliability is not solely an artifact of one individual's accent or pronunciation. Cross-referencing language priorities against Hajj demographic data revealed that Bahasa Indonesia, representing the largest single national pilgrim quota of 221,000 pilgrims, was entirely omitted from the original design. This paper conceptualizes this as bias by invisibility, distinct from technical exclusion, and proposes empirical demographic verification as its remedy. Concrete ethical mechanisms, including a data retention policy, safety-critical escalation paths, and a spiritual distress acknowledgment mechanism, are proposed and tested within Shifa. A three-layer governance framework is introduced. Together these findings motivate four revisions to the SACRED Framework and the introduction of Oversight as a seventh principle, extending the framework to SACREDO.
Files
Multilingual Voice AI for Pilgrim Assistance_ STT Reliability, Bias by Invisibility, and Ethical Deployment in Sacred Environments (1).pdf
Additional details
Additional titles
- Alternative title (English)
- Multilingual Voice AI for Pilgrim Assistance
Related works
- Continues
- Preprint: 10.5281/zenodo.21726075 (DOI)
Dates
- Accepted
-
2026-08-10Date the manuscript was finalized
References
- Bian, W., Lin, X., & Cheng, G. (2025). Bridging the Language Gap: Synthetic Voice Diversity via Latent Mixup for Equitable Speech Recognition. arXiv:2511.20534.
- Chen, J., Gao, X., Tatsuya, K., & Chen, N. F. (2026). LoASR-Bench: Evaluating Large Speech Language Models on Low-Resource ASR Across Language Families. arXiv:2603.20042.
- Nakatumba-Nabende, J., Kagumire, S., Kantono, C., & Nabende, P. (2025). A Systematic Literature Review on Bias Evaluation and Mitigation in Automatic Speech Recognition Models for Low-Resource African Languages. ACM Computing Surveys, 58(4), Article 105. DOI: 10.1145/3769089.
- Kuhlman, C., Jackson, L., & Chunara, R. (2020). No Computation Without Representation: Avoiding Data and Algorithm Biases Through Diversity. arXiv:2002.11836.
- Ceccon, M., Cornacchia, G., Dalle Pezze, D., Fabris, A., & Susto, G. A. (2025). Underrepresentation, Label Bias, and Proxies: Towards Data Bias Profiles for the EU AI Act and Beyond. Expert Systems with Applications, 292, 128266. arXiv:2507.08866.
- Alsayed, K. A. (2026). Operational AI Deployment Assurance: Governance-State Orchestration Under Threshold-Sensitive Deployment Conditions, A Governance Framework for High-Stakes AI Systems. arXiv:2605.27827.
- Htin, N. M. (2026). Redefining the Standard of Human Oversight for AI Negligence. Harvard Journal of Law & Technology Digest. Retrieved from https://jolt.law.harvard.edu/digest/redefining-the-standard-of-human-oversight-for-ai-negligence
- Counsel Health Editorial Team. (2026). What CHAI's Guidelines Reveal About Where Medical AI Governance Is Heading. Counsel Health. Retrieved from https://www.counselhealth.com/blog/coalition-for-health-ai
- Pakistan Point. (2025, May 31). Top 10 Countries Sending Most Hajj Pilgrims In 2025. Retrieved from https://www.pakistanpoint.com/en/story/1989105/top-10-countries-sending-most-hajj-pilgrims-in-2025-sa.html
- General Authority for Statistics (GASTAT). (2025). Hajj Statistics Publication 2025. Kingdom of Saudi Arabia. Retrieved from https://www.stats.gov.sa/documents/20117/2435281/Hajj+Statistics+Publication+2025EN+(1).pdf
- Fareed, A. (2026). Designing Culturally and Spiritually Sensitive AI Assistants for High-Density Religious Environments: A Prompt Engineering Framework Applied at Masjid al-Haram. Zenodo. https://doi.org/10.5281/zenodo.21726075