chat/greeting
● passed- Judged on
- “The reply greets the user by name and offers one next step.”
- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 1.6s
eve-eval-trend
3 runs · 14 evals · generated 28/07/2026, 14:30
Getting worse
Judge score 0.81 → 0.76 across 3 runs.
Pass rate 85%, unchanged.
What moved
- ↓
weather/forecastpassing → failing, judge 0.93 → 0.00 - ↓
intents/handover2 flips, still passing - ↑
chat/handofffailing → passing, judge 0.41 → 0.88
9 unchanged · 1 not run · 1 still failing: weather/units
Per eval
● passed▲ scored✕ failed– skipped· not in that run
Each card covers the last 3 runs, oldest → latest.
chat/handoff
● passed- Judged on
- “The reply names the queue it is handing over to.”
- Pass rate
- 67%
- Runs
- ● 2✕ 1
- Last duration
- 2.2s
edge/a-deliberately-long-eval-id-that-runs-past-the-card-header-and-keeps-going
● passed- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 900ms
intents/handover
● passed- Judged on
- “The reply routes to a human when the user asks for one.”
- Pass rate
- 67%
- Runs
- ● 2✕ 1
- Last duration
- 1.2s
multi/three-conditions
● passed- Judged on
- “The reply answers the question that was asked.”
- “The reply stays under four sentences.”
- “The reply does not invent a source it was not given.”
- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 4.1s
search/retrieval
● passed- Judged on
- “The answer cites at least one retrieved document.”
- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 3.1s
tools/geocode
● passed- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 310ms
weather/forecast
✕ failed- Judged on
- “The reply gives a forecast for the requested day, not the current conditions.”
- Pass rate
- 67%
- Runs
- ● 2✕ 1
- Last duration
- 30.0s
weather/units
▲ scored- Judged on
- “Temperatures are given in Celsius.”
- Pass rate
- 0%
- Runs
- ▲ 3
- Last duration
- 1.9s
edge/no-assertions
● passed- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 120ms
edge/with-image
● passed- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 21.0s
edge/with-output
● passed- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 2.0s
intents/seasonal
– skipped- Pass rate
- —
- Runs
- – 3
- Last duration
- —
tools/weather-lookup
● passed- Pass rate
- 100%
- Runs
- ● 3
- Last duration
- 240ms
Runs
29/07/2026, 09:15 ● 11 passed▲ 1 scored✕ 1 failed– 1 skipped
chat/greeting ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply greets the user by name and offers one next step.” |
soft | 0.94 | 0.70 | ● pass |
chat/handoff ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply names the queue it is handing over to.” |
soft | 0.88 | 0.70 | ● pass |
search/retrieval ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The answer cites at least one retrieved document.” |
soft | 0.72 | 0.70 | ● pass |
tools/weather-lookup ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
tools/geocode ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.91 | 0.70 | ● pass |
weather/forecast ✕ failed↑ Scroll to top
✕ Timed out after 30000ms waiting for the tool call
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply gives a forecast for the requested day, not the current conditions.” |
soft | 0.00 | 0.70 | ✕ miss |
weather/units ▲ scored↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “Temperatures are given in Celsius.” |
soft | 0.58 | 0.70 | ✕ miss |
intents/handover ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply routes to a human when the user asks for one.” |
soft | 0.95 | 0.70 | ● pass |
intents/seasonal – skipped↑ Scroll to top
– skipped: only runs between November and January
No assertions recorded.
multi/three-conditions ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.89 | 0.70 | ● pass |
edge/a-deliberately-long-eval-id-that-runs-past-the-card-header-and-keeps-going ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.93 | 0.70 | ● pass |
edge/no-assertions ● passed↑ Scroll to top
No assertions recorded.
edge/with-output ● passed↑ Scroll to top
It is currently 14°C and overcast in Bristol, with light rain expected after 4pm. Winds are light from the south-west at around 11 km/h.
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
edge/with-image ● passed↑ Scroll to top
Here is the fox on a skateboard.
image review: most — the fox and the skatepark are there, but its tail is not curled around its paws.
- image review: most — the fox and the skatepark are there, but its tail is not curled around its paws.
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
28/07/2026, 09:15 ● 11 passed▲ 1 scored✕ 1 failed– 1 skipped
chat/greeting ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply greets the user by name and offers one next step.” |
soft | 0.97 | 0.70 | ● pass |
chat/handoff ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply names the queue it is handing over to.” |
soft | 0.88 | 0.70 | ● pass |
search/retrieval ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The answer cites at least one retrieved document.” |
soft | 0.72 | 0.70 | ● pass |
tools/weather-lookup ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
tools/geocode ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.94 | 0.70 | ● pass |
weather/forecast ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply gives a forecast for the requested day, not the current conditions.” |
soft | 0.95 | 0.70 | ● pass |
weather/units ▲ scored↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “Temperatures are given in Celsius.” |
soft | 0.58 | 0.70 | ✕ miss |
intents/handover ✕ failed↑ Scroll to top
✕ no matching intent
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply routes to a human when the user asks for one.” |
soft | 0.30 | 0.70 | ✕ miss |
intents/seasonal – skipped↑ Scroll to top
– skipped: only runs between November and January
No assertions recorded.
multi/three-conditions ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.92 | 0.70 | ● pass |
edge/a-deliberately-long-eval-id-that-runs-past-the-card-header-and-keeps-going ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.96 | 0.70 | ● pass |
edge/no-assertions ● passed↑ Scroll to top
No assertions recorded.
edge/with-output ● passed↑ Scroll to top
It is currently 14°C and overcast in Bristol, with light rain expected after 4pm. Winds are light from the south-west at around 11 km/h.
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
edge/with-image ● passed↑ Scroll to top
Here is the fox on a skateboard.
image review: most — the fox and the skatepark are there, but its tail is not curled around its paws.
- image review: most — the fox and the skatepark are there, but its tail is not curled around its paws.
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
27/07/2026, 09:15 ● 11 passed▲ 1 scored✕ 1 failed– 1 skipped
chat/greeting ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply greets the user by name and offers one next step.” |
soft | 0.95 | 0.70 | ● pass |
chat/handoff ✕ failed↑ Scroll to top
✕ assistant handed over without naming the queue
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply names the queue it is handing over to.” |
soft | 0.41 | 0.70 | ✕ miss |
search/retrieval ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The answer cites at least one retrieved document.” |
soft | 0.72 | 0.70 | ● pass |
tools/weather-lookup ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
tools/geocode ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.92 | 0.70 | ● pass |
weather/forecast ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply gives a forecast for the requested day, not the current conditions.” |
soft | 0.93 | 0.70 | ● pass |
weather/units ▲ scored↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “Temperatures are given in Celsius.” |
soft | 0.58 | 0.70 | ✕ miss |
intents/handover ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA “The reply routes to a human when the user asks for one.” |
soft | 0.95 | 0.70 | ● pass |
intents/seasonal – skipped↑ Scroll to top
– skipped: only runs between November and January
No assertions recorded.
multi/three-conditions ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.90 | 0.70 | ● pass |
edge/a-deliberately-long-eval-id-that-runs-past-the-card-header-and-keeps-going ● passed↑ Scroll to top
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| judge.autoevals.closedQA | soft | 0.94 | 0.70 | ● pass |
edge/no-assertions ● passed↑ Scroll to top
No assertions recorded.
edge/with-output ● passed↑ Scroll to top
It is currently 14°C and overcast in Bristol, with light rain expected after 4pm. Winds are light from the south-west at around 11 km/h.
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |
edge/with-image ● passed↑ Scroll to top
Here is the fox on a skateboard.
image review: most — the fox and the skatepark are there, but its tail is not curled around its paws.
- image review: most — the fox and the skatepark are there, but its tail is not curled around its paws.
| Assertion | Severity | Score | Min | Result |
|---|---|---|---|---|
| succeeded | gate | 1.00 | — | ● pass |