Position: Let's Strengthen Verifiability If We Can't Enforce Reproducibility

Samet Hicsonmez    Nermin Samet    Renaud Marlet

NeurIPS 2026

Overview of the proposed verification workflow. At submission time, training and inference logs are attached to the submission package and verified by one or several reviewers (Level 1). The metric verification step (Level 2), indicated with red background, requires the submission of JSON file(s) that contain model predictions on target dataset(s), along with an evaluation script. The assigned metric validator runs the evaluation script and compares with the numbers in the main paper. Finally, the AC updates the corresponding statuses (Tab. 1) in the submission site.

Abstract

In the field of Machine Learning, many papers contain empirical results supporting claimed statements or illustrating the performance of a proposed method. However, most practitioners know that (1) results are generally hard to reproduce, and increasingly so, (2) code is not often available to do so, and (3) it hinders the development of research. In this position paper, we analyze and quantify these issues, and make concrete proposals to improve result checkability, if not reproducibility.

BibTeX

@inproceedings{hicsonmez2026verifiability,
  title     = {Position: Let's Strengthen Verifiability If We Can't Enforce Reproducibility},
  author    = {Hicsonmez, Samet and Samet, Nermin and Marlet, Renaud},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}