Conference paper Open Access

Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning

Liu, Hao; Ye, Hanting; Yang, Jie; Wang, Qing


JSON-LD (schema.org) Export

{
  "description": "<p>Motivated by the trend of realizing full screens on devices such as smartphones, in this work we propose through-screen sensing with visible light for the application of fingertip air-writing. The system can recognize handwritten digits with under-screen photodiodes as the receiver. The key idea is to recognize the weak light reflected by the finger when the finger writes the digits on top of a screen. The proposed air-writing system has immunity to scene changes because it has a fixed screen light source. However, the screen is a double-edged sword as both a signal source and a noise source. We propose a data preprocessing method to reduce the interference of the screen as a noise source. We design an embedded deep learning model, a customized model ConvRNN, to model the spatial and temporal patterns in the dynamic and weak reflected signal for air-writing digits recognition. The evaluation results show that our through-screen fingertip air-writing system with visible light can achieve accuracy up to 91%. Results further show that the size of the customized ConvRNN model can be reduced by 94% with less<br>\nthan a 10% drop in performance.</p>", 
  "license": "https://creativecommons.org/licenses/by/4.0/legalcode", 
  "creator": [
    {
      "affiliation": "TU Delft", 
      "@type": "Person", 
      "name": "Liu, Hao"
    }, 
    {
      "affiliation": "TU Delft", 
      "@type": "Person", 
      "name": "Ye, Hanting"
    }, 
    {
      "affiliation": "TU Delft", 
      "@type": "Person", 
      "name": "Yang, Jie"
    }, 
    {
      "affiliation": "TU Delft", 
      "@type": "Person", 
      "name": "Wang, Qing"
    }
  ], 
  "headline": "Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning", 
  "image": "https://zenodo.org/static/img/logos/zenodo-gradient-round.svg", 
  "datePublished": "2021-11-05", 
  "url": "https://zenodo.org/record/5646942", 
  "@type": "ScholarlyArticle", 
  "@context": "https://schema.org/", 
  "identifier": "https://doi.org/10.1145/3485730.3493454", 
  "@id": "https://doi.org/10.1145/3485730.3493454", 
  "workFeatured": {
    "url": "https://aichallengeiot.github.io/", 
    "alternateName": "AIChallengeIoT", 
    "location": "Coimbra, Portugal", 
    "@type": "Event", 
    "name": "Workshop on Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things in conjuction with ACM SenSys 2021"
  }, 
  "name": "Through-Screen Visible Light Sensing Empowered by Embedded Deep Learning"
}
24
11
views
downloads
Views 24
Downloads 11
Data volume 14.3 MB
Unique views 24
Unique downloads 11

Share

Cite as