Project deliverable Open Access
Viro, Vladimir; Schmidt, Julian
The WhoLoDancE search and similarity engine exists in order to support data mining and real-time querying of the motion captured data collected within the project. In order to reduce the computational burden while retaining most of the semantically relevant content, the search and similarity algorithms are applied not to the raw video or the point cloud data collected during the recording sessions, but first of all to the algorithmically reconstructed skeletal configurations of the dancers, with the further addition of low-, mid- and high-level features (HLFs) derived from these data.
We refer to each skeletal coordinate (for example, a right elbow angle) as a dimension. Thus the recordings are for our purposes multi-dimensional time series of floating-point numbers and we are solving a problem of providing an adequate infrastructure for search and similarity measurements over high-dimensional time series. The dimensionality of recordings collected within WhoLoDancE is not constant and ranges between 72 and 168, depending on the amount of detail collected during the motion capture sessions. For example, the Flamenco recordings feature finger motions, while recordings of other genres, for which finger motions are less relevant and haven’t been captured, do not.
We approach the problem of search in multidimensional time series with a variable number of dimensions by first performing the similarity computations on the one-dimensional time series constituting the data in our database and then aggregating these results across multiple dimensions in a suitable manner. This allows for the parallelization of most of the similarity computations across different dimensions, which is a useful feature given the large number of dimensions and the availability of parallel computing infrastructure (multi-core servers, GPUs, and compute clusters). The computation of similarity on a single dimension can also be parallelized naturally by partitioning the data, albeit at a slight loss in efficiency. These properties make our approach scalable and capable of supporting orders of magnitude more data than have been collected within the project.
D4.1 Data Integration Algorithm and System Analysis and Framework Description.pdf