Conference paper Open Access

Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning

Liu, Hao; Ye, Hanting; Yang, Jie; Wang, Qing

Citation Style Language JSON Export

  "DOI": "10.1145/3485730.3493454", 
  "title": "Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning", 
  "issued": {
    "date-parts": [
  "abstract": "<p>Motivated by the trend of realizing full screens on devices such as smartphones, in this work we propose through-screen sensing with visible light for the application of fingertip air-writing. The system can recognize handwritten digits with under-screen photodiodes as the receiver. The key idea is to recognize the weak light reflected by the finger when the finger writes the digits on top of a screen. The proposed air-writing system has immunity to scene changes because it has a fixed screen light source. However, the screen is a double-edged sword as both a signal source and a noise source. We propose a data preprocessing method to reduce the interference of the screen as a noise source. We design an embedded deep learning model, a customized model ConvRNN, to model the spatial and temporal patterns in the dynamic and weak reflected signal for air-writing digits recognition. The evaluation results show that our through-screen fingertip air-writing system with visible light can achieve accuracy up to 91%. Results further show that the size of the customized ConvRNN model can be reduced by 94% with less<br>\nthan a 10% drop in performance.</p>", 
  "author": [
      "family": "Liu, Hao"
      "family": "Ye, Hanting"
      "family": "Yang, Jie"
      "family": "Wang, Qing"
  "id": "5646942", 
  "event-place": "Coimbra, Portugal", 
  "type": "paper-conference", 
  "event": "Workshop on Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things in conjuction with ACM SenSys 2021 (AIChallengeIoT)"
Views 24
Downloads 11
Data volume 14.3 MB
Unique views 24
Unique downloads 11


Cite as