0
Stress test a prompt for bias and failure modes before wide release
Before I release this prompt for general use, stress test it for bias and failure modes across a range of inputs it will realistically encounter. Given the prompt: prompt and the population of users/inputs it will see: user_population_description
1. Generate a diverse set of test inputs that vary along dimensions relevant to bias risk for this specific task (not generic demographic categories unless they're actually relevant to how this prompt could fail differently for different inputs) - name the dimensions you're varying and why they're relevant here specifically.
2. For each test input, note if the prompt's output quality, tone, or thoroughness would plausibly differ in a way that's not justified by the input's actual content (e.g. giving a noticeably shorter or more dismissive answer to inputs written in non-native-sounding English, when the underlying question is equally valid).
3. Distinguish a genuine bias risk (systematic, would recur) from a one-off quality variance (any single output can be a bit better or worse, that's normal, not evidence of bias).
4. For genuine risks found, propose the specific prompt addition that would mitigate it, and note if it's a prompt-level fix or something that needs to be addressed at the model/system level instead.
5. Be honest if you don't find a meaningful risk in a dimension you tested, don't manufacture a finding to seem thorough.
Prompt: prompt
User population: user_population_description