Skip to content

Governance and Maintenance

lrdbench is maintained as a research benchmark framework. The maintainer reviews changes for scientific interpretation risk, reproducibility impact, and compatibility with the public benchmark contract.

Maintenance Scope

The maintained public surface includes:

  • the Python package under src/lrdbench;
  • tracked smoke, public-small, and public-medium benchmark suites;
  • the estimator interface and author test utilities;
  • the manifest schema exercised by public suites;
  • the public output contract;
  • documentation, examples, issue templates, and release workflows.

Local manuscript workflows, generated reports, local caches, and ignored paper-support artefacts are not part of the public release contract.

Compatibility Policy

Within a major release series:

  • existing public manifest fields should remain valid;
  • required output files and required output columns should not be removed or renamed;
  • core metric names should not be repurposed;
  • estimator-interface changes should be additive where possible;
  • breaking changes require migration notes and a major-version release.

Additive metrics, optional output files, new suites, new estimators, and new diagnostics may be introduced in minor releases when they do not invalidate existing public comparisons.

Contribution Review

Estimator and benchmark submissions should include:

  • a stable estimator or suite name;
  • version metadata;
  • declared target estimand and assumptions;
  • reproducible manifests or tests;
  • interpretation caveats where relevant;
  • output contract validation for generated reports.

Submissions that affect public benchmark interpretation should be reviewed against the known limitations, estimator contract, leaderboard policy, and reproducibility guide.

Release Process

Public releases should:

  • pass lint, tests, docs build, package build, and distribution checks;
  • validate representative installed-package benchmark runs;
  • update changelog, citation metadata, and migration notes when public surfaces change;
  • tag the release in Git;
  • archive or DOI-tag release artefacts when an archive is available.

Choose an unused package version in src/lrdbench/__init__.py before creating its matching v<version> tag. A merge does not create a package release. The release workflow builds the source distribution and a wheel from it, checks metadata and packaged assets, and smoke-tests the installed wheel outside the checkout. A mismatched version tag fails before publication.

Manual dispatch of the release workflow builds and validates downloadable artefacts only. Publication requires a pushed version tag and uses the pypi GitHub environment with PyPI Trusted Publishing. The maintainer must configure that environment and the matching trusted publisher on PyPI before releasing.

Releases are archived on Zenodo; cite the concept DOI 10.5281/zenodo.20937726, which resolves to the latest archived version.