Official implementation in python. https://arxiv.org/abs/1904.04189

If you use the code, please cite

@inproceedings{kukleva2019unsupervised,
  title={Unsupervised learning of action classes with continuous temporal embedding},
  author={Kukleva, Anna and Kuehne, Hilde and Sener, Fadime and Gall, Jurgen},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR'19)},
  year={2019}
}

To train or reproduce numbers: HowTo

Pipeline for one activity class. Figure 1 in the paper.

Proposed pipeline for unsupervised learning with unknown activity classes. Figure 2 in the paper.

Each frame is color coded a) with the corresponding ground truth subaction label, b) with K assigned subaction labels after clustering as the second step in Fig.1 in our main paper, c)with the predicted labels after the decoding stage. The optimization of our network is performed with respect to relative timestep of each frame. In d) we show the respective relative time label in the continuous temporal embedding assigned to each frame feature. The color bar depicts that bright blue corresponds to 0 (startof the video) and pink to 1 (end of the video).

Breakfast dataset. The order of subactions: SIL, take bowl, pour cereals, pour milk, stir cereals, SIL

Breakfast dataset. The order of subactions: SIL, take cup, add teabag, pour water, SIL

Breakfast dataset. The order of subactions: SIL, spoon powder, pour milk, stir milk, SIL

Breakfast dataset. The order of subactions: SIL, take knife, cut orange, squeeze orange, pour juice, squeeze orange, pour juice, squeeze orange, pour juice, squeeze orange, pour juice, SIL

Breakfast dataset. The order of subactions: SIL, cut bun, smear butter, put toppingOnTop, SIL

50Salads dataset. The order of subactions: start, cut, place, cut, place, cut, place, cut, place, null, null, add oil, add pepper, mix dressing, end

50Salads dataset. The order of subactions: start, cut, place, cut, place, cut, place, peel cucumber, cut, place, mix ingredients, add oil, null, add pepper, null, mix dressing, serve salad onto plate, add dressing, end