Look Behind You! – Using a Face Camera for Mobile Augmented Reality Odometry

Augmented reality applications provide new ways of presenting
cultural heritage assets thanks to the recent advancement in the eld
of smart devices. Unfortunately, the construction of the hardware and
lower computational power of mobile processors limit the potential of
these applications. Namely, almost all current visual-inertial odometry
libraries employed in smartphones require the real tracked objects to be
close and contain distinguishable features, which is an issue when observing
large virtual structures outdoors like historical buildings or objects
on plain walls of halls or museums. This paper exploits the possibility of
using the face cameras available in mobile devices for augmented reality
tracking. It designs a prototype composed of iPhone and iPad devices
and evaluates its contribution in two scenarios that current systems cannot
handle. The results reveal the clear benet of this approach for the
cultural heritage, allowing it to operate in situations when users look up
in the sky to see the roof of virtual buildings, or when they move closer
to a white wall to perceive details of a virtual painting. In the end, the
paper discusses the system's limitations and proposes solutions to them.