PoSynDA is a novel framework for 3D Human Pose Estimation (3D HPE) that addresses the challenges of adapting to new datasets due to the scarcity of 2D-3D pose pairs in target domain training sets. This repository contains the official PyTorch implementation of the PoSynDA method as described in our paper.

• Domain-Adaptation: Generative, target-specific source augmentation with a multi-hypothesis approach.
• Optimization Strategy: Teacher-student learning paradigm for efficient model training.
• Efficient Domain Adaptation: Low-rank adaptation for fine-tuning.

• Python 3.x
• PyTorch >= 0.4.0
• einops
• timm
• tensorboard
• Other dependencies

You should download MATLAB if you want to evaluate our model on MPI-INF-3DHP dataset.

1. Clone this repository:

   git clone https://github.com/hbing-l/PoSynDA.git
   cd PoSynDA
   

2. Install the required packages:

   pip install -r requirements.txt
   

Our model is evaluated on Human3.6M and MPI-INF-3DHP datasets.

• Human3.6M: Instructions and processed data link
• MPI-INF-3DHP: Instructions and processed data link

h36m_transfer.py is the code to transfer H36M S1 to S5, S6, S7, S8, and h36m_3dhp_transfer.py is the code to transfer H36M dataset to 3DHP dataset. To train the PoSynDA model on the target dataset (e.g. 3DHP), run:

python h36m_3dhp_transfer.py -k cpn_ft_h36m_dbb -num_proposals 3 -timestep 1000 -c checkpoint/ -gpu 0 --nolog

For evaluation of the provided model.

python h36m_3dhp_transfer.py -c checkpoint -gpu 0 --nolog --evaluate best_epoch.bin

Our method achieves a 58.2mm MPJPE on the Human3.6M dataset without using 3D labels from the target domain, comparable to the target-specific MixSTE model (58.2mm vs. 57.9mm).

If you find this work useful for your research, please consider citing our paper:

@article{liu2023posynda,
  title={PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation},
  author={Liu, Hanbing and He, Jun-Yan and Cheng, Zhi-Qi and Xiang, Wangmeng and Yang, Qize and Chai, Wenhao and Wang, Gaoang and Bao, Xu and Luo, Bin and Geng, Yifeng and others},
  journal={arXiv preprint arXiv:2308.09678},
  year={2023}
}

We would like to thank all the following contributors and researchers who made this project possible.

• VideoPose3D
• DiffusionDet
• MixSTE
• video-to-pose3D
• D3DP

This project is licensed under the MIT License - see the LICENSE.md file for details.