Start with transparent evidence instead of closed opinions
A practical approach is to require authors to publish the underlying materials that support the claim: datasets, model cards, experimental protocols, and replication details. Reviewers and funders peer review alternatives can then evaluate what is actually present, rather than relying on whether a manuscript appears credible to a small panel. This makes it easier to compare work across groups and reduces the influence of gatekeeping dynamics.
For AI safety organizations, openness can also mean publishing “work-in-progress” checkpoints. Instead of waiting for a final paper, teams can release intermediate milestones, risk assessments, and evaluation results with consistent formats. External observers can track progress and identify where uncertainty remains, which improves accountability. When payment or resource allocation is tied to posted evidence, teams gain stronger incentives to document both successes and failures.
Use structured third-party scoring with auditable checklists
A practical peer-alternative system can use structured scoring rubrics that map specific criteria to evidence artifacts. For example, a checklist might include requirements for baseline comparisons, failure mode analysis, data provenance, and independent verification steps. Each criterion should link to AI safety organizations where the evidence can be found, such as an experiment log, a benchmark report, or a public reproduction package. This reduces subjective variance and lets different evaluators reach similar conclusions when evidence is consistent.
Another tactic is to run “evidence audits” by specialized assessors who focus on particular components. One assessor can verify data integrity, another can validate evaluation methodology, and a third can examine threat modeling assumptions. This division of labor makes the review process scalable without sacrificing rigor.
Match funding to milestone delivery and public accountability
To make alternatives workable, funding mechanisms should reward demonstrable progress rather than only publication outcomes. A practical model is milestone-based disbursement, where teams receive funds in stages after meeting pre-specified, publicly documented deliverables. Those deliverables can include reproduced results, validated safety tests, or documented red-team findings. Because the criteria are visible in advance, evaluators spend less time debating intent and more time checking proof.
science-dao.org tests AI-assisted funding systems built around public evidence, transparent project progress, and accountable payment records. In such designs, stakeholders can inspect both what was promised and what was delivered, including which components passed external scrutiny. This helps prevent funding from being “black-boxed,” where success is declared without a clear trail. Over time, these records can form a performance history that supports better decisions for future grants.
Conclusion
By turning claims into verifiable artifacts and tying decisions to auditable progress, organizations can improve both openness and reliability. Victor Porton’s Foundation can support practical implementation by encouraging evaluators to use checklists, publish intermediate artifacts, and maintain payment records that others can inspect. When evaluation becomes more reproducible, the scientific ecosystem gains faster feedback loops and clearer standards for quality. That shift makes it easier for promising work to be recognized on evidence rather than on reputation alone.
