A new interactive quiz on Hacker News challenges readers to identify which of several LLM outputs contains a watermark. The thread received 11 points and 5 comments.
What Watermarking Actually Does
LLM watermarking embeds statistical signals into generated text by biasing token selection toward specific patterns. Detectors later scan for those patterns without needing the original model weights.
The quiz presents multiple short passages and asks users to pick the watermarked one. Early comments note that human detection accuracy stays low once the watermark strength is tuned for readability.
Current Detection Performance
No public benchmark numbers appear in the thread itself. Independent tests on similar schemes report true-positive rates between 85% and 95% at 1% false-positive rate when the watermark strength parameter sits at 2.0–3.0.
Detection degrades sharply once text is paraphrased or translated. One comment in the thread mentions that even light editing drops detection below 60%.
How to Try the Quiz
Visit the page directly at https://sgoedecke.github.io/watermark-quiz/. No installation or API key is required. Each attempt shows four short outputs; select the one you believe is watermarked and receive immediate feedback.
The quiz uses a fixed set of examples rather than live generation, so results are reproducible across visitors.
Pros and Cons of Public Watermark Quizzes
- Provides immediate, side-by-side comparison of watermarked versus clean text
- Requires zero setup or compute
- Limited to a handful of static examples
- Does not expose the underlying watermarking algorithm or strength parameter
Alternatives and Detection Tools
Several open-source detectors exist for comparison.
| Tool | Access | False Positive | Notes |
|---|---|---|---|
| Watermark-Detect (HF) | Hugging Face space | ~1% | Requires logit access |
| GPTZero | Web API | 2–4% | Commercial, no code |
| Original quiz | Static page | N/A | Human-only test |
The quiz serves as a human baseline rather than an automated detector.
Who Should Use This
Researchers testing watermark robustness and prompt engineers who want quick intuition on detectability will find it useful. Teams needing production-grade detection should move to logit-based or API-provided tools instead.
Bottom line: The quiz offers a fast, no-setup way to experience how visible current watermarks remain to humans.
Developers building detection pipelines can treat the quiz as a quick sanity check before running larger automated evaluations.
Top comments (0)