A Hacker News thread titled "Opus 5.0 drives incoherence into the stratosphere" reached 84 points and 46 comments within days of posting.
The discussion centers on measurable rises in contradictory or off-topic output from the latest Opus release compared with earlier checkpoints.
Model: Opus 5.0 | Discussion score: 84 points | Comments: 46 | Source: Hacker News thread
What the Thread Reports
Users posted side-by-side prompt tests showing Opus 5.0 producing conflicting facts within the same response at rates roughly double those seen in Opus 4.1.
Several comments included raw logs where the model reversed its own stated constraints mid-generation, a pattern not observed at the same frequency in prior releases.
Measured Incoherence Rates
Early testers shared internal counts from 200-prompt batches. One reproducible set showed:
| Version | Contradiction rate | Average tokens before drift | Sample size |
|---|---|---|---|
| Opus 4.1 | 7% | 620 | 200 |
| Opus 5.0 | 14-18% | 310 | 200 |
The numbers come from repeated runs on identical prompts across temperature 0.7 settings.
How to Reproduce the Tests
Run the same 200-prompt set used in the thread. Store outputs and apply a simple contradiction detector script available in the linked repository comments.
Compare results against your own temperature and system-prompt settings before drawing conclusions for production use.
Pros and Cons Highlighted
- Faster response times reported on short factual queries.
- Noticeable drop in long-context consistency on tasks exceeding 4k tokens.
- Stronger creative phrasing in some fiction prompts, offset by higher revision needs.
Alternatives Mentioned
Commenters compared outputs directly with Claude 3.5 Sonnet and GPT-4o on the same prompts.
| Model | Contradiction rate (same test) | Context length before drift |
|---|---|---|
| Opus 5.0 | 14-18% | 310 tokens |
| Claude 3.5 Sonnet | 8% | 580 tokens |
| GPT-4o | 9% | 490 tokens |
Who Should Test Opus 5.0 Now
Teams running short, low-stakes creative tasks may benefit from the speed gains. Projects requiring strict factual consistency over long outputs should continue with 4.1 or switch to the alternatives above until further patches appear.
Bottom line: The thread supplies concrete numbers showing doubled incoherence on standard test sets, giving practitioners clear data to decide whether to adopt or wait.
The discussion indicates that prompt-level mitigations can reduce but not eliminate the gap, suggesting the issue sits at the model level rather than user technique alone.
Top comments (0)