Agnost AI launched on Hacker News with a tool that pulls structured user feedback directly from agent conversation logs. The YC S26 company positions the product for teams running production agents who currently review transcripts by hand.
What It Is / How It Works
Agnost AI ingests conversation histories between users and AI agents. It identifies explicit and implicit feedback signals such as corrections, satisfaction statements, and task success indicators. The system then outputs tagged feedback items that can feed into product roadmaps or fine-tuning datasets.
The product requires no additional instrumentation beyond existing agent logs. Users connect their conversation store and receive a feed of extracted insights.
How to Try It
Visit the site at agnost.ai and connect a sample dataset or live agent endpoint. The launch thread on Hacker News shows early users testing with exported chat histories from common frameworks.
No public benchmarks were shared in the announcement. The four comments on the 19-point thread focused on data privacy and integration effort rather than performance numbers.
Pros and Cons
- Works on existing logs without code changes
- Outputs structured data ready for downstream tools
- Limited to signals present in text conversations
- Requires sufficient conversation volume to surface patterns
Early HN commenters noted privacy concerns when logs contain sensitive user data.
Alternatives and Comparisons
Teams currently rely on manual review, spreadsheet tagging, or general observability platforms. Agnost AI targets the specific gap between raw logs and actionable feedback.
| Approach | Speed | Structure | Cost model |
|---|---|---|---|
| Manual review | Slow | Inconsistent | Engineer time |
| LangSmith / Helicone | Fast | Generic metrics | Usage-based |
| Agnost AI | Automated | Feedback-focused | Not disclosed |
Who Should Use This
Product teams running customer-facing agents with hundreds of conversations per week gain the clearest benefit. Research groups with smaller datasets or strict data residency rules should evaluate privacy controls first.
Bottom Line / Verdict
Agnost AI automates a repetitive task that most agent teams still perform manually. Its value depends on whether the extracted feedback proves more reliable than current ad-hoc methods.
The launch reflects a broader shift toward treating agent conversations as primary product data rather than temporary artifacts.
Top comments (0)