# Production LLM release checklist

Licensed under CC BY 4.0. Attribution details:
`https://futurecraft.pro/artifacts/production-llm-release-manifest/LICENSE-CC-BY-4.0.md`.

Release ID: ____________________

Owner: ____________________

Review date: ____________________

## Freeze the candidate

- [ ] The manifest validates against `manifest.schema.json`.
- [ ] Prompt, policy, router, retrieval, toolset, adapter, dataset, and runbook
  references are immutable and resolvable.
- [ ] The source commit and build ID identify the artifact that will run.
- [ ] The task's input/output contract and required capabilities match production.
- [ ] No prompt text, secret, user payload, raw trace, or personal data is stored in the
  manifest.

## Check policy and execution

- [ ] Authentication and tenant context come from verified application state.
- [ ] Allowed providers satisfy the declared data regions and classification.
- [ ] Tool permissions are narrower than the model's possible tool requests.
- [ ] Provider attempts and tool steps fit inside the total deadline.
- [ ] Every retried write is idempotent or has an explicit reconciliation path.
- [ ] Human-review triggers pause the side effect rather than merely add a warning.

## Run the evaluation gate

- [ ] The current and candidate releases ran on the same dataset version.
- [ ] Contract, task-quality, latency, and cost thresholds were set before the run.
- [ ] Each declared slice has enough cases to support its threshold.
- [ ] Candidate scores come from stored run results, not a hand-edited dashboard value.
- [ ] Every failed case was inspected; an average score did not hide a policy breach.
- [ ] New production failures have a redacted reproduction case or a documented reason
  why they cannot enter the dataset.

## Exercise rollout and rollback

- [ ] Canary traffic is limited to eligible requests and one known release ID.
- [ ] Guardrail windows, owners, and alert routes are active before traffic moves.
- [ ] The previous release is still addressable.
- [ ] Rollback was tested against the current schema, tools, and data migrations.
- [ ] A partial stream, committed side effect, and provider timeout have known recovery
  behavior.
- [ ] The kill switch stops new work without corrupting in-flight state.

## Verify after deployment

- [ ] Traces contain the release ID and component references from this manifest.
- [ ] Task success, contract failures, latency, fallback, and cost are visible by
  release.
- [ ] Logs and traces were checked for accidental secrets or user payloads.
- [ ] The canary review compares product outcomes as well as model-level scores.
- [ ] The decision to expand, hold, or roll back is recorded with the evidence used.

If one required item is unknown, the release is not ready for unattended rollout.
Record the exception, owner, expiry, and safe operating limit instead of silently
checking the box.