Kernel Manifold Alignment for Domain Adaptation

The wealth of sensory data coming from different modalities has opened numerous opportu-
nities for data analysis. The data are of increasing volume, complexity and dimensionality,
thus calling for new methodological innovations towards multimodal data processing. How-
ever, multimodal architectures must rely on models able to adapt to changes in the data dis-
tribution. Differences in the density functions can be due to changes in acquisition
conditions (pose, illumination), sensors characteristics (number of channels, resolution) or
different views (e.g. street level vs. aerial views of a same building). We call these different
acquisition modes domains, and refer to the adaptation problem as domain adaptation. In
this paper, instead of adapting the trained models themselves, we alternatively focus on
finding mappings of the data sources into a common, semantically meaningful, representa-
tion domain. This field of manifold alignment extends traditional techniques in statistics
such as canonical correlation analysis (CCA) to deal with nonlinear adaptation and possibly
non-corresponding data pairs between the domains. We introduce a kernel method for man-
ifold alignment (KEMA) that can match an arbitrary number of data sources without needing
corresponding pairs, just few labeled examples in all domains. KEMA has interesting prop-
erties: 1) it generalizes other manifold alignment methods, 2) it can align manifolds of very
different complexities, performing a discriminative alignment preserving each manifold
inner structure, 3) it can define a domain-specific metric to cope with multimodal specifici-
ties, 4) it can align data spaces of different dimensionality, 5) it is robust to strong nonlinear
feature deformations, and 6) it is closed-form invertible, which allows transfer across-
domains and data synthesis. To authors’ knowledge this is the first method addressing all
these important issues at once. We also present a reduced-rank version of KEMA for
computational efficiency, and discuss the generalization performance of KEMA under
Rademacher principles of stability. Aligning multimodal data with KEMA reports outstanding
benefits when used as a data pre-conditioner step in the standard data analysis processing
chain. KEMA exhibits very good performance over competing methods in synthetic con-
trolled examples, visual object recognition and recognition of facial expressions tasks.
KEMA is especially well-suited to deal with high-dimensional problems, such as images
and videos, and under complicated distortions, twists and warpings of the data manifolds. A
fully functional toolbox is available at https://github.com/dtuia/KEMA.git.