0
Debug a prompt that produces inconsistent output formatting across runs
⁂auto-checked, 4 days oldAauraNovice
The prompt
This prompt produces the right content but inconsistent formatting run to run (sometimes markdown, sometimes plain text, sometimes different heading structure). Help me fix the actual cause. Given the prompt and 3-4 example outputs showing the inconsistency: prompt / example_outputs
1. Check whether the prompt actually specifies a format at all, versus implies one loosely through an example that could be read as illustrative rather than prescriptive - this is the most common root cause and I want it ruled in or out explicitly first.
2. If a format is specified, check whether it's specified precisely enough (e.g. 'use headers' doesn't say what level, what if any bold/italic is expected) - ambiguity in the spec, not model unreliability, is often the real cause.
3. Check if the inconsistency correlates with anything about the input (longer inputs get different formatting than short ones, certain topics trigger different structure) rather than being purely random, since that points to a different fix (an input-dependent rule) than pure underspecification.
4. Propose the exact formatting specification to add, and if useful, one formatted example to lock in the structure as few-shot, weighing that against the added token cost.
5. If none of the above explains it, say so and suggest it may need output post-processing/validation rather than a prompt fix, since not every consistency problem is solvable purely at the prompt level.
Prompt: prompt
Example outputs showing inconsistency: example_outputs
Where it has been run
| Model | Version | People | Broke | Auto | Last |
|---|---|---|---|---|---|
| Claude | Haiku 4.5 | 0 | 0 | 1/1 | 4 days ago |
Example output
Root cause confirmed: the prompt says 'organize your answer clearly' with no explicit format spec at all, that phrase is doing zero prescriptive work, so the model is choosing a reasonable structure fresh each time, which is why it varies.
Correlation check: the variation does correlate with input length, short inputs get a plain paragraph, longer multi-part inputs get headers and bullets, which makes sense as the model adapting to apparent complexity, but that's exactly the inconsistency you don't want if you need a fixed downstream format regardless of input.
Fix: replace 'organize your answer clearly' with an explicit spec: 'Always use this exact structure: an H2 heading per section titled `## Findings`, `## Recommendation`, `## Next Steps`, in that order, even if a section is short.' This removes the model's discretion entirely rather than hoping it infers consistency.
One formatted example included as few-shot to lock in heading level and section naming exactly, since 'H2' alone can still be read multiple ways in terms of surrounding whitespace and casing; the added ~60 tokens is worth it here since your use case pipes this output directly into a renderer that needs exact structure.
0 comments
Sign in to comment or report what this prompt did for you.
Sign inNo comments yet. Run the prompt and report what you got.