0
Write an eval rubric for a prompt before you scale it up
I'm about to scale up usage of this prompt: prompt for use_case. Before I do, help me write an evaluation rubric so I can measure quality systematically instead of spot-checking. Requirements:
1. Derive 4-6 scoring criteria directly from what the prompt itself demands (if the prompt requires a specific format, that's a criterion; if it requires groundedness, that's a criterion) rather than generic LLM-output quality metrics that don't reflect this specific prompt's actual requirements.
2. Make each criterion as objectively checkable as possible: state whether it's programmatically checkable (regex, schema, exact match), or genuinely requires an LLM-judge or human, and don't overstate how automatable a subjective criterion is.
3. For each criterion, write the LLM-judge instruction that would grade it, calibrated with one example of a passing and one failing output, so the judge prompt itself is reproducible and not vague.
4. Define a pass/fail threshold for the overall rubric (e.g. must pass all disqualifying criteria plus 80% of others) rather than leaving 'good enough' undefined.
5. Suggest how many sample outputs I'd need to eval before I can trust an aggregate pass rate, given the stakes of use_case.
Prompt: prompt
Use case and stakes: use_case