We provide the code and models for the following report (arXiv Preprint):

  Appearance-and-Relation Networks for Video Classification
  Limin Wang, Wei Li, Wen Li, and Luc Van Gool
  in arXiv, 2017

• November 23th, 2017
  • Initialize the repo.

ARTNet aims to learn spatiotemporal features from videos in an end-to-end manner. Its construction is based on a newly-designed module, termed as SMART block. ARTNet is a simple and general video architecture and all these relased models are trained from scratch on video dataset. Currently, for an engineering compromise between accuracy and efficiency, ARTNet is instantiated with the ResNet-18 architecture and trained on the input volume of 112*112*16.

The training of ARTNet is based on our modified Caffe toolbox. Specical thanks to @zbwglory for modifying this code.

The training code is under folder of models/.

These models are trained on the Kinetics dataset from scratch and tested on the validation set. Our training is performed based on the input volume of 112*112*16. The test is performed by cropping 25 clips from the videos.

The fine tuning process is conducted based on the TSN framework, where segment number is 2.

The fine tuning code is under folder of fine_tune/

These models learned on the Kinetics dataset are transferred to the HMDB51 and UCF101 datasets. The fine-tuning process is done with TSN framework where the segment number is 2. The performance is reported over three splits by using only RGB input.