# Livenerf: Tracking Opus 5.5 Nerfs via HN Data

> Published 2026-09-30 · https://www.promptzone.com/miles_dvorak/livenerf-tracking-opus-55-nerfs-via-hn-data-1bgi

Livenerf appeared on Hacker News in a thread that reached 459 points and 193 comments. The repo at https://github.com/ninjahawk/livenerf supplies scripts that repeatedly query Opus 5.5 on fixed prompt sets and log output length, factual accuracy, and refusal rates.

## What Livenerf Measures

The tool runs a fixed suite of 200 prompts daily. It records token count, answer completeness against reference facts, and rate of safety refusals. Results are stored in CSV files with timestamps for trend analysis.

Users can add custom prompt lists or change the target model endpoint. The default configuration targets Opus 5.5 through the Anthropic API.

## HN Thread Metrics and Reactions

The discussion accumulated 459 points within 48 hours. Commenters posted side-by-side outputs from September and October runs showing 18 % shorter responses on technical questions and a 9 % rise in refusals for borderline prompts.

Several users shared their own 7-day logs confirming the same downward trend on coding and math tasks.

## How to Run the Tests

Clone the repository and install dependencies with `pip install -r requirements.txt`. Set the ANTHROPIC_API_KEY environment variable, then execute `python livenerf.py --model opus-5.5 --prompts default.csv --days 7`.

Results appear in the `results/` folder as dated CSV files. A simple plot script generates line charts of token count and refusal percentage.

## Tradeoffs Reported by Testers

- Requires a paid Anthropic key; free tiers are blocked after roughly 50 calls.
- Only measures observable output changes, not internal weight updates.
- Works for any API-accessible model, not limited to Opus 5.5.
- No statistical significance testing built in; users must add their own.

## Comparison with Other Nerf Trackers

| Tool              | Prompts per run | Output metrics     | Open source | API cost |
|-------------------|-----------------|--------------------|-------------|----------|
| Livenerf          | 200             | length, refusals   | Yes         | Paid     |
| OpenRouter logs   | Variable        | latency only       | No          | Paid     |
| Custom eval sets  | User-defined    | accuracy + length  | Yes         | Varies   |

Livenerf is the only option that ships ready-made daily scheduling and CSV export.

## Who Should Use Livenerf

Developers maintaining production prompts benefit from automated daily checks. Researchers tracking model drift can extend the prompt set to domain-specific tasks. Casual users without API budgets or interest in longitudinal data should skip it.

> **Bottom line:** Livenerf gives the first public, reproducible dataset showing measurable output shrinkage in Opus 5.5.

The repo’s commit history already contains two weeks of baseline data, allowing new users to compare against an established trend immediately.