Stable assignment · outcome guardrails · MIT + CC BY 4.0

Prompt A/B experiment fixture

Keep one subject in one prompt variant, expose allocation drift, and summarize the primary outcome separately from guardrail failures and latency.

The fixture does not call Langfuse or an LLM, calculate statistical significance, or select a winner. It covers assignment and reporting mechanics.

Assign once

SHA-256 hashing keeps a subject in the same variant for one experiment key.

Separate guardrails

Primary outcome, guardrail failures, and latency remain distinct fields.

Do not fake certainty

The summary reports allocation drift but deliberately declares no winner.

Files

experiment.ts Stable assignment and outcome summaries experiment_test.ts Seven deterministic Deno tests run-checklist.md English offline and canary checklist run-checklist.ru.md Russian offline and canary checklist README.md Scope, test commands, and evidence boundary

Verification

Deno formatting, type checking, linting, and all seven tests passed against the published files before release.

MIT license CC BY 4.0 license Read the Langfuse prompt experiment guide