Cyclist Actions: Motion History Images and Trajectories

The dataset consists of over 1.1 million samples of labeled cyclists actions. Every sample consists of two motion history images (MHI), consisting of motion over the past second, from two different cameras, the past trajectory of the cyclist of the last second (50 past positions), and a label of the currently performed action.

The samples were extracted from 1,639 video sequences of cyclists moving across an urban intersection at the University of Applied sciences in Aschaffenburg: https://www.th-ab.de/ueber-uns/organisation/labor/kooperative-automatisierte-verkehrssysteme/ausstattung/

The uploaded files consist of a numpy file with MHI, trajectories and labels,  and a json file containing 5-fold cross validation/test split.

Every MHI has a size 192x192x2. The two channels contain the separate motion contours of the bike and the cyclist.

The manually created labels are:

• straight: cyclists is moving and not turning
• tr/tl: cyclist is turning left/right
• move: cyclist is moving with nearly constant velocity and not turning
• start: cyclist was standing and starts moving
• starting_movement: first movement of cyclist before starting
• stop: cyclist was moving/starting and slows down to a halt
• wait: cyclist is standing
• hand_signal_left/right: cyclist indicates a turn by hand signal
• shoulder_check_left/right: cyclist looks over left/right shoulder
• out_of_saddle: cyclist is standing

The numpy file consists of python dictionaries with scenes of the form:

{SCENE_NAME:
    'mhi_hk1/2': [...], # list of zip compressed, python pickled mhi from cameras 1 and 2
    'x/y/z_tracked': [...], # tracked cyclists positions in x/y/z directions,
    'x/y/z_smoothed': [...], # smoothed (by rts smoother) cyclists positions in x/y/z directions,
    'orientation': [...], # orientation of the cyclists estimated by kalman filters
    'ts': [...], # utc timestamps in micro seconds
    LABEL_NAME: [...], # labels of different actions (0 or 1)}

The zipped/pickled mhi can be decompressed using:

import numpy as np
import pickle
import zlib

d = np.load('mhi_dataset_0,1,2,3,4,6,9,13,18,26,33,41,49.npy', allow_pickle=True, encoding='latin1').item()
for scene_name in d:
    for i in range(len(d[scene_name]['mhi_hk1']) - 50): # ommit 50 first frames as they have no mhi
        mhi1 = pickle.loads(zlib.decompress(d[scene_name]['mhi_hk1'][49 + i].encode('latin1')), encoding='latin1')
        mhi2 = pickle.loads(zlib.decompress(d[scene_name]['mhi_hk2'][49 + i].encode('latin1')), encoding='latin1')

This work results from the project DeCoInt 2, supported by the German Research Foundation (DFG) within the priority program SPP 1835: "Kooperativ interagierende Automobile", grant numbers DO 1186/1-2, FU 1005/1-2, and SI 674/11-2. Additionally, the work is supported by "Zentrum Digitalisierung Bayern".

Due to privacy laws in germany, we are not permitted to publish raw image data.