Getting Started

This document provides basic tutorials for the usage of MMAction. For installation, please refer to INSTALL.md. For data deployment, please refer to DATASET.md.

An example on UCF-101

We first give an example of testing and training action recognition models on UCF101.

Prepare data

First of all, please follow PREPARING_UCF101.md for data preparation.

Test a reference model

Reference models are stored in MODEL_ZOO.md. We download a reference model spatial stream BN-Inception at $MMACTION/modelzoo using:

wget -c https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmaction/models/ucf101/tsn_2d_rgb_bninception_seg3_f1s1_b32_g8-98160339.pth -P ./modelzoo/

Then, together with provided configs files, we run the following code to test with multiple GPUs:

./tools/dist_test_recognizer.sh test_configs/TSN/ucf101/tsn_rgb_bninception.py tsn_2d_rgb_bninception_seg3_f1s1_b32_g8-98160339.pth 8

When testing 3D ConvNets, the oversample we used is 10 clips x 3 crops by default. For some extremely large models, it might be difficult for so many samples to fit on 1 GPU. When it happens, you can use commands for heavy test instead:

./tools/dist_test_recognizer_heavy.sh test_configs/CSN\ircsn_kinetics400_se_rgb_r152_seg1_32x2.py ircsn_kinetics400_se_rgb_r152_f32s2_ig65m_fbai-9d6ed879.pth 8 --batch_size=5

Train a model with multiple GPUs

To reproduce the model, we provide training scripts as follows:

./tools/dist_train_recognizer.sh configs/TSN/ucf101/tsn_rgb_bninception.py 8 --validate

--validate: performs evaluation every k (default=1) epochs during the training, which help diagnose training process.

More examples

The procedure is not limited to action recognition in UCF101. To perform spatial-temporal detection on AVA, we can train a baseline model by running

./tools/dist_train_detector.sh configs/ava/ava_fast_rcnn_nl_r50_c4_1x_kinetics_pretrain_crop.py 8 --validate

and evaluate a reference model by running

wget -c wget -c https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmaction/models/ava/fast_rcnn_ava2.1_nl_r50_c4_1x_f32s2_kin-e2495b48.pth -P modelzoo/
python tools/test_detector.py ava_fast_rcnn_nl_r50_c4_1x_kinetics_pretrain_crop.py modelzoo/fast_rcnn_ava2.1_nl_r50_c4_1x_f32s2_kin-e2495b48.pth --out ava_fast_rcnn_nl_r50_multiscale.pkl --gpus 8 --eval bbox

To perform temporal action detection on THUMOS14, we can training a baseline model by running

./tools/dist_train_localizer.sh configs/thumos14/ssn_thumos14_rgb_bn_inception.py 8

and evaluate a reference model by running

wget -c wget -c https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmaction/models/thumos14/ssn_thumos14_rgb_bn_inception_tag-dac9ddb0.pth -P modelzoo/
python tools/test_detector.py configs/thumos14/ssn_thumos14_rgb_bn_inception.py modelzoo/ssn_thumos14_rgb_bn_inception_tag-dac9ddb0.pth --gpus 8 --out ssn_thumos14_rgb_bn_inception.pkl --eval thumos14

More Abstract Usage

Inference with pretrained models

We provide testing scripts to evaluate a whole dataset.

Test a dataset

python tools/test_${ARCH}.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] [other task-specific arguments]

Arguments:

${ARCH} could be
- "recognizer" for action recognition (TSN, I3D, SlowFast, R(2+1)D, CSN, ...)
- "localizer" for temporal action detection/localization (SSN)
- "detector" for spatial-temporal action detection (a re-implmented Fast-RCNN baseline)
${CONFIG_FILE} is the config file stored in $MMACTION/test_configs.
${CHECKPOINT_FILE} is the checkpoint file. Please refer to MODEL_ZOO.md for more details.

Train a model

MMAction implements distributed training and non-distributed training, powered by the same engine of mmdetection.

Train with multiple GPUs (Recommended)

Training with multiple GPUs follows the rules below:

./tools/dist_train_${ARCH}.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]

${ARCH} could be
- "recognizer" for action recognition (TSN, I3D, ...)
- "localizer" for temporal action detection/localization (SSN)
- "detector" for spatial-temporal action detection (a re-implmented Fast-RCNN baseline)
${CONFIG_FILE} is the config file stored in $MMACTION/configs.
${GPU_NUM} is the number of GPU (default: 8). If you are using number other than 8, please adjust the learning rate in the config file linearly.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!