HSICityV2: Urban Scene Understanding via Hyperspectral Images

Light in all spectrum travel in the physical world. The trichromatism (RGB) human vision captures and understands it. Machine vision makes an analogy which use RGB camera for semantic segmentation and scene understanding. We argue that such machine vision suffers from metamerism, that different objects may appear in same RGB color while actually distinctive in spectrum. While learning based solutions, especially deep learning, have been heavily explored, they do not solve the fundamental physical limitation. In this paper, we propose to use Hyperspectral images (HSIs), which capture hundreds of consecutive narrow bands from the real visible world and therefore metamerism no longer exists. In short, we aim to 'see beyond human vision'. In practice, we introduce a novel large scale high quality HSI dataset for semantic segmentation in cityscapes. Namely, Hyperspectral City dataset. The dataset contains 1330 HSIs which are captured in typical urban driving scenes. Each HSI has 1889×1422 spatial resolution and 128 spectral channels ranged from 450nm to 950nm. The dataset provides semantic annotation at pixel level which is done manually by professional annotators. We believe this dataset enables a new direction for scene understanding.