ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation

Cédric Rommel    Victor Letzelter    Nermin Samet    Renaud Marlet    Matthieu Cord    Patrick Pérez    Eduardo Valle

NeurIPS 2024

Abstract

We propose ManiPose, a manifold-constrained multi-hypothesis model for human-pose 2D-to-3D lifting. We provide theoretical and empirical evidence that, due to the depth ambiguity inherent to monocular 3D human pose estimation, traditional regression models suffer from pose-topology consistency issues, which standard evaluation metrics (MPJPE, P-MPJPE and PCK) fail to assess. ManiPose addresses depth ambiguity by proposing multiple candidate 3D poses for each 2D input, each with its estimated plausibility. Unlike previous multi-hypothesis approaches, ManiPose forgoes generative models, greatly facilitating its training and usage. By constraining the outputs to lie on the human pose manifold, ManiPose guarantees the consistency of all hypothetical poses, in contrast to previous works. We showcase the performance of ManiPose on real-world datasets, where it outperforms state-of-the-art models in pose consistency by a large margin while being very competitive on the MPJPE metric.

BibTeX

@article{rommel2024manipose,
  title={ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation},
  author={Rommel, C{\'e}dric and Letzelter, Victor and Samet, Nermin and Marlet, Renaud and Cord, Matthieu and P{\'e}rez, Patrick and Valle, Eduardo},
  journal={Advances in Neural Information Processing Systems},
  year={2024}
}