Inside the eval gate — Operant

A cheaper model takes over a task only after it passes the eval gate.

Here's each step in the live console with sample values.

A learned rule sends one pattern to a cheaper model with a skill.

A new rule starts in shadow and changes no traffic.

The gate replays held out work on your baseline model and on the cheaper model with the skill.

A judge scores each pair and a mean parity of 0.8 or more passes this rule passed at 95% with Claude Opus 4.8 as the judge scribe learns the skill from one set and a separate holdout stays for the gate.

A passed gate moves no traffic until a person ratifies the rule.

DemoteuShadow sends the pattern back to its old model at once.

Read the full post on the Operant blog.