[Dual-Diffusion for Binocular 3D Human Pose Estimation, NeurIPS2024]

Dual-Diffusion Model Architecture

Dual-Diffusion Process

Our code is built on top of DDIM and DiffPose.

Environment

You can create the environment via:

pip install -r requirement.txt

Dataset

The data includes:

1. labels (update the labels_path in the config.)
2. 2D poses estimated from 2D pose estimator (put into the ./data directory)
3. statistical or fixed mean and standard deviation of the estimated 2D poses (put into the ./data directory)

human3.6

The H36M dataset is preprocessed according to steps 1-5 in learnabel_triangulation. Then the binocular labels are generated according to stereo_estimation.

MHAD

The MHAD dataset is preprocessed by MHAD_Berkeley_preprocess. Then the binocular labels are generated according to MHAD_Berkeley_stereo.

The labels can be downloaded [here](【超级会员V4】我通过百度网盘分享的文件：data 链接：https://pan.baidu.com/s/1nYRICIhMJFZmkq-bR_05dg 提取码：S2fq 复制这段内容打开「百度网盘APP即可获取」).

Note that we only provide the 2D poses estimated from RSB-Pose.

Training

• To train a model, run:

CUDA_VISIBLE_DEVICES=0 python main_diffpose_2view_frame.py --train \
--config mhad_2view_diffpose_uvxyz_rsb152.yml --batch_size 1024 \
--model_diff_path checkpoints/ckpt_mhad_rsb152.pth \
--doc mhad_2view_diffpose_uvxyz_rsb152 --exp exp --ni

Testing

We provide the pre-trained model (with RSB-Pose 2D Pose as input) [here](通过网盘分享的文件：checkpoints 链接: https://pan.baidu.com/s/1cP10IVxcOQGHEfYoxdvFVw?pwd=qdd3 提取码: qdd3 --来自百度网盘超级会员v4的分享). To evaluate it, put it into the ./checkpoint directory.

• To evaluate, run:

CUDA_VISIBLE_DEVICES=0 python main_diffpose_2view_frame.py \
--config mhad_2view_diffpose_uvxyz_rsb152.yml \
--test_timesteps 1 \
--model_diff_path checkpoints/ckpt_mhad_rsb152.pth \
--doc t_mhad_2view_diffpose_uvxyz_rsb152 --exp exp --ni

Bibtex

If you find our work useful in your research, please consider citing:

@inproceedings{
    anonymous2024dualdiffusion,
    title={Dual-Diffusion for Binocular 3D Human Pose Estimation},
    author={Anonymous},
    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024},
    url={https://openreview.net/forum?id=NT8Z5NjwxF}
} 

Acknowledgement

Part of our code is borrowed from DDIM, DiffPose, learnabel_triangulation, and stereo_estimation. We thank the authors for releasing the codes.