The lowest confidence routing acts on. An agent Choice answered below it keeps continuity, and
so does an artifact Choice. The thread Likelihood must also be clear of the middle: a message
leaves the thread only when its calibrated probability of continuing (see
ROUTING_CONTINUES_CALIBRATION) is at most 1 - DECISION_ROUTING_MIN_CONFIDENCE.
What the bar means in a chat. The calibration was fitted on a corpus that is mostly switches
(174 switches, 127 continuations), and a chat is mostly continuations. A calibrated 0.30 is a 30%
chance that the message continues only at the corpus's mix; at a 90%-continue prior it is about an
84% chance (see ROUTING_CONTINUES_CALIBRATION). In raw terms, routing leaves the thread when
Jev's raw probability is at most about 0.80 (LLM Decision's, about 0.78). Before calibration the
bar was a raw 0.30. This value was not chosen for a chat's mix. Routing ships off, and the bar
should be set on the prior-weighted figures below before routing is turned on.
Measured at a calibrated 0.30 in the Phase 2 Decision Eval (plan Task 2.4, 2026-09-29), on that
corpus. Jev left the thread on 155 points, and 150 of them were real switches. So it caught 86.2%
of the switches (150/174) and wrongly left 3.9% of the continuations (5/127). LLM Decision caught
72.4% of the switches and also wrongly left 5 continuations.
At the corpus's mix (58% switches), 96.8% of Jev's leaves were correct, and 96.2% of LLM
Decision's.
At a 90%-continue prior, closer to a chat, about 71% of Jev's leaves are correct, and about
67% of LLM Decision's. Per 100 messages, Jev makes about 8.6 correct leaves and sends about 3.5
continuations to another agent. The false-leave rate rests on 5 of 127 continuations (Wilson 95%
interval 1.7% to 8.9%), so Jev's figure could be anywhere from about 52% to 85%.
Accuracy at that prior is 95.1% (Jev) and 93.7% (LLM Decision), against 90.0% for always
continuing. Accuracy hides the gap above, because a missed switch and a continuation sent to the
wrong agent don't cost the same.
Those figures are an upper bound on what routing does. They score the thread Likelihood
alone, on each point's mean over five repeats. Routing makes one call, and leaves the thread only
when the agent Choice also reaches this confidence and names another agent the chat allows, and
the answer arrives within DECISION_ROUTING_TIMEOUT_MS; and a switch counted here may have
gone to the wrong agent. The Decision Eval scorecard's production-verdict table measures routing
end to end, per run; it needs a re-run of the eval, since the stored runs predate calibration.
The agent Choice's confidence is not calibrated yet: the corpus labels continue or switch, not
which agent.
The lowest confidence routing acts on. An agent Choice answered below it keeps continuity, and so does an artifact Choice. The thread Likelihood must also be clear of the middle: a message leaves the thread only when its calibrated probability of continuing (see ROUTING_CONTINUES_CALIBRATION) is at most
1 - DECISION_ROUTING_MIN_CONFIDENCE.What the bar means in a chat. The calibration was fitted on a corpus that is mostly switches (174 switches, 127 continuations), and a chat is mostly continuations. A calibrated 0.30 is a 30% chance that the message continues only at the corpus's mix; at a 90%-continue prior it is about an 84% chance (see ROUTING_CONTINUES_CALIBRATION). In raw terms, routing leaves the thread when Jev's raw probability is at most about 0.80 (LLM Decision's, about 0.78). Before calibration the bar was a raw 0.30. This value was not chosen for a chat's mix. Routing ships off, and the bar should be set on the prior-weighted figures below before routing is turned on.
Measured at a calibrated 0.30 in the Phase 2 Decision Eval (plan Task 2.4, 2026-09-29), on that corpus. Jev left the thread on 155 points, and 150 of them were real switches. So it caught 86.2% of the switches (150/174) and wrongly left 3.9% of the continuations (5/127). LLM Decision caught 72.4% of the switches and also wrongly left 5 continuations.
Those figures are an upper bound on what routing does. They score the thread Likelihood alone, on each point's mean over five repeats. Routing makes one call, and leaves the thread only when the agent Choice also reaches this confidence and names another agent the chat allows, and the answer arrives within DECISION_ROUTING_TIMEOUT_MS; and a switch counted here may have gone to the wrong agent. The Decision Eval scorecard's production-verdict table measures routing end to end, per run; it needs a re-run of the eval, since the stored runs predate calibration. The agent Choice's confidence is not calibrated yet: the corpus labels continue or switch, not which agent.