# Community contribution review rubric

Version 1.0 · Project Keystone · 2026-08-13

## Purpose

Use this rubric to review community contributions consistently while preserving the distinction between facts, judgment, and funding. A reviewer must never use a quality score to hide a failed objective criterion, and must never treat acceptance as a promise of payment.

## Required review record

Every review records:

- contribution and opportunity identifiers;
- opportunity version or `updatedAt` reviewed;
- artifact URLs and the time they were checked;
- an outcome for every objective gate with evidence;
- quality scores with short reasons;
- limitations, uncertainties, and requested changes;
- a separate grant-eligibility note;
- reviewer identifier and review timestamp.

## Stage 0 — Scope and conflict check

Before judging the work, confirm the reviewer can evaluate it fairly.

| Check | Result | Reviewer action |
|---|---|---|
| Contribution matches the claimed opportunity | Yes / No | Stop as out of scope if no. |
| Reviewer has a material conflict of interest | Yes / No | Disclose and recuse or request a second reviewer if yes. |
| Opportunity criteria and version are available | Yes / No | Pause and obtain the authoritative criteria if no. |
| Artifact can be reviewed without unsafe or identity-dependent action | Yes / No | Do not perform prohibited actions; request accessible evidence. |

## Stage 1 — Objective acceptance gate

Rate each applicable gate **PASS**, **FAIL**, **NOT APPLICABLE**, or **NEEDS EVIDENCE**. A pass requires a locator and a concise observation, not merely the contributor's assertion.

| Gate | PASS means | Minimum evidence |
|---|---|---|
| G1 Scope match | The deliverable directly addresses the opportunity and all named deliverable types. | Mapping from each deliverable to an artifact section or URL. |
| G2 Artifact availability | Required artifacts load and are readable with the stated access model. | URL/path, HTTP or local check, and review timestamp. |
| G3 Completeness | Every explicit acceptance criterion has a corresponding result. | Criterion-by-criterion checklist. |
| G4 Originality and provenance | Work is attributable and sources, licenses, reused assets, and AI assistance are disclosed where relevant. | Source list, license/provenance note, or justified “original work” statement. |
| G5 Evidence integrity | Claims are supported by reproducible observations; hashes, logs, or screenshots identify the reviewed release when relevant. | Commands/checks, output summary, release identifier, or SHA-256 values. |
| G6 Safety and policy fit | The artifact does not require prohibited, deceptive, harmful, or undisclosed actions to evaluate or use. | Reviewer observation and any limitations. |
| G7 Handoff readiness | A new contributor can understand, rerun, or maintain the result at the level promised by the opportunity. | README/instructions, dependencies, inputs, outputs, and known limitations. |

### Gate decision rule

- **ACCEPTABLE**: every applicable gate is PASS.
- **CHANGES REQUIRED**: at least one gate is FAIL or NEEDS EVIDENCE and the defect appears remediable.
- **OUT OF SCOPE / CANNOT REVIEW**: the work does not match the opportunity, authoritative criteria are unavailable, or safe review is impossible.

`NOT APPLICABLE` must include a reason. A reviewer must not average a failed gate into a passing score.

## Stage 2 — Quality judgment

Only score quality after completing the objective gate. Use a 0–4 scale and cite one observation per dimension.

| Score | Anchor |
|---|---|
| 0 | Absent or unusable. |
| 1 | Major weaknesses; substantial rework needed. |
| 2 | Adequate for the stated scope, with meaningful limitations. |
| 3 | Strong, clear, and useful; only minor improvements remain. |
| 4 | Exemplary and reusable beyond the immediate contribution. |

| Dimension | What to judge | Weight |
|---|---|---:|
| Q1 Usefulness | Solves the actual project need and enables a concrete next step. | 25% |
| Q2 Clarity | Structure, language, labels, and conclusions are easy to follow. | 20% |
| Q3 Reproducibility | Another reviewer can repeat the checks or regenerate the result. | 20% |
| Q4 Rigor | Assumptions, uncertainty, edge cases, and contradictory evidence are handled honestly. | 20% |
| Q5 Maintainability | Ownership, update path, formats, and dependencies support future use. | 15% |

Calculate the weighted score as `sum(score × weight)`, yielding 0.00–4.00. Suggested interpretation:

- 3.50–4.00: exemplary;
- 2.75–3.49: strong;
- 2.00–2.74: adequate;
- below 2.00: weak.

The score is advisory. It cannot override Stage 1.

## Stage 3 — Decision and feedback

Choose exactly one operational recommendation:

- **Accept** — all gates pass; record strengths and optional improvements.
- **Request changes** — list each failed/missing gate, the exact evidence or edit needed, and a bounded re-review path.
- **Decline as out of scope** — explain the mismatch and, if possible, point to the correct opportunity.
- **Escalate** — use for conflicts, disputed provenance, ambiguous criteria, or specialist review needs.

Feedback must be specific, respectful, and testable. Separate required fixes from optional polish. Do not infer intent when an observable defect is sufficient.

## Stage 4 — Grant eligibility note

Funding is a separate decision. Record one of:

- **Eligible for discretionary consideration** — objective gates pass and project rules do not exclude the contribution.
- **Not currently eligible** — identify the failed gate or applicable project rule.
- **Eligibility undetermined** — authoritative funding criteria or review authority are missing.
- **Not applicable** — the project or opportunity has no grant mechanism.

Always append: **“Eligibility is not an award, payment promise, ranking, or guarantee. Any grant requires a separate authorized decision and available project funds.”**

If funding guidance exists, a reviewer may recommend a band with reasons, but must label it non-binding. Never fabricate a budget or convert a volunteer opportunity into paid work.

## Review template

```markdown
# Contribution review: <title>
- Contribution: <id>
- Opportunity: <id> @ <version/updatedAt>
- Reviewer / reviewed at: <id> / <ISO-8601>
- Artifacts checked: <URLs or paths>

## Objective gate
| Gate | Result | Evidence / observation |
|---|---|---|
| G1 | PASS / FAIL / N/A / NEEDS EVIDENCE | ... |
...
Gate decision: ACCEPTABLE / CHANGES REQUIRED / OUT OF SCOPE / CANNOT REVIEW

## Quality judgment
| Dimension | Score (0–4) | Reason |
|---|---:|---|
| Q1 | ... | ... |
...
Weighted score: X.XX / 4.00 (advisory)

## Recommendation
Accept / Request changes / Decline as out of scope / Escalate

Required changes: ...
Optional improvements: ...
Limitations and uncertainty: ...

## Grant eligibility
<status and reasons>
Eligibility is not an award, payment promise, ranking, or guarantee. Any grant requires a separate authorized decision and available project funds.
```

## Reviewer calibration

Periodically have two reviewers independently score the same contribution. Compare gate disagreements first, then quality dimensions that differ by two or more points. Update examples or guidance—not the underlying evidence—to reduce ambiguity. Preserve both original reviews in the audit trail.