# Can Typed Decisions Replace LLM Judges?

> Published 2026-09-21 · https://www.promptzone.com/santiago_abbott/can-typed-decisions-replace-llm-judges-45ol

Openlayer posted **jevals** on GitHub this week, a library that replaces LLM judges with typed Jev decisions. The Show HN thread on Hacker News gathered 15 points.

> **Tool:** jevals | **Core idea:** typed Jev decisions | **Source:** [GitHub](https://github.com/openlayer-ai/jevals) | **HN score:** 15 points

## What It Is

jevals turns evaluation outputs into strongly typed decisions instead of free-text judgments from another LLM. Each decision follows a defined schema that can be checked programmatically.

The approach removes the second LLM call that most current eval pipelines use. Developers define the expected decision structure once, then run checks directly against model outputs.

## How It Works

Users declare decision types in code. The library validates that model responses match the declared schema and records the outcome as a typed result.

No additional model call is required for judgment. The typed structure itself serves as the evaluation signal.

## Benchmarks and Numbers

The GitHub repository shows the core implementation and example usage. No public benchmark table appears in the initial release.

Early visibility on Hacker News reached 15 points with zero comments, indicating limited discussion so far.

## How to Try It

Clone the repository and install the package locally. Import the decision types into an existing evaluation script and replace any LLM-as-judge step with a typed Jev definition.

Run the validation functions against your model outputs to generate structured results that can be asserted or aggregated.

## Pros and Cons

- Removes extra LLM call and associated cost
- Produces machine-checkable outputs instead of text
- Requires upfront schema definition
- Limited community feedback available at launch

## Alternatives and Comparisons

Common LLM judge setups rely on models such as GPT-4 or Claude to score outputs. Frameworks like LangSmith and DeepEval still use this pattern.

| Feature          | jevals              | Typical LLM Judge | LangSmith Eval |
|------------------|---------------------|-------------------|----------------|
| Extra model call | No                  | Yes               | Yes            |
| Output type      | Typed decision      | Free text         | Structured log |
| Schema enforcement | Built-in          | Manual            | Partial        |
| Cost per eval    | Lower               | Higher            | Higher         |

## Who Should Use This

Teams running large-scale automated evaluations benefit most. Projects that already maintain strict output schemas gain immediate value.

Teams that need subjective quality judgments or lack defined decision types should continue with existing LLM judge methods until schemas are established.

> **Bottom line:** jevals offers a direct way to remove LLM judges from evaluation loops when typed decisions fit the task.

The release signals growing interest in deterministic evaluation layers that reduce dependence on secondary model calls.