PAFUSE: Part-based Diffusion for 3D Whole-Body Pose Estimation

Nermin Samet    Cédric Rommel    David Picard    Eduardo Valle

ECCV Workshop 2024

Abstract

We introduce a novel approach for 3D whole-body pose estimation, addressing the challenge of scale- and deformability- variance across body parts brought by the challenge of extending the 17 major joints on the human body to fine-grained keypoints on the face and hands. In addition to addressing the challenge of exploiting motion in unevenly sampled data, we combine stable diffusion to a hierarchical part representation which predicts the relative locations of fine-grained keypoints within each part (e.g., face) with respect to the part's local reference frame. On the H3WB dataset, our method greatly outperforms the current state of the art, which fails to exploit the temporal information. We also show considerable improvements compared to other spatiotemporal 3D human-pose estimation approaches that fail to account for the body part specificities.

BibTeX

@inproceedings{samet2024pafuse,
      title={PAFUSE: Part-based Diffusion for 3D Whole-Body Pose Estimation},
      author={Nermin Samet, C{\'{e}}dric Rommel, David Picard, Eduardo Valle},
      year={2024},
      booktitle={ECCV Workshop Towards a Complete Analysis of People: Fine-grained Understanding for Real-World Applications}
}