You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Put downloaded data into the following directory structure:
- gendata/
- ntu/
- ntu120/
- nturgbd_raw/
- nturgb+d_skeletons/ # from `nturgbd_skeletons_s001_to_s017.zip`
...
- nturgb+d_skeletons120/ # from `nturgbd_skeletons_s018_to_s032.zip`
...
Generating Data
Generate NTU RGB+D 60 or NTU RGB+D 120 dataset:
cd ./gendata/ntu # or cd ./gendata/ntu120
# Get skeleton of each performer
python3 get_raw_skes_data.py
# Remove the bad skeleton
python3 get_raw_denoised_data.py
# Transform the skeleton to the center of the first frame
python3 seq_transformation.py
Training & Testing
Training
Change the config file depending on what you want.
To train model on NTU RGB+D 60/120 with joint, bone or motion modalities, run the following command.
# Example: training STTFormer on NTU RGB+D 60 cross subject under bone modality
python3 main.py --config config/ntu60_xsub_bone.yaml
Testing
To test the trained models saved in <work_dir>, run the following command:
# Example: testing STTFormer on NTU RGB+D 60 cross subject under bone modality
python3 main.py --config config/ntu60_xsub_bone.yaml --run_mode test --save_score True --weights work_dir/ntu60/xsub_bone/xsub_bone.pt
As with training, it should be noted that the data modality should correspond to the weight.
To ensemble the results of different modalities, run
# Example: ensemble three modalities of STTFormer on NTU RGB+D 60 cross subject
python3 ensemble.py --dataset ntu/xsub --joint_dir work_dir/ntu60/xsub_joint --bone_dir work_dir/ntu60/xsub_bone --joint_motion_dir work_dir/ntu60/xsub_joint_motion
Pretrained Models
The pretrained models will be available soon.
Acknowledgements
This repository is based on CTR-GCN. Thanks to the original authors for their work!
Citation
Please cite this work if you find it useful:.
@article{Qiu2022SpatioTemporalTT,
title={Spatio-Temporal Tuples Transformer for Skeleton-Based Action Recognition},
author={Helei Qiu and Biao Hou and Bo Ren and Xiaohua Zhang},
journal={ArXiv},
year={2022},
volume={abs/2201.02849}
}
About
Spatio-Temporal Tuples Transformer for Skeleton-Based Action Recognition