How long, in milliseconds, routing waits for the decision before it keeps today's continuity.
The call is abandoned client-side at this point, and the server is asked to bound its model call
to the same figure.
The budget. The prompt-based intent check this replaces was removed at about 300 ms, so the
decision has to come in under that, network included (plan Task 3.9).
This value is over that budget, deliberately. In the Phase 2 Decision Eval (plan Task 2.4,
2026-09-29) Jev answered this decision in 187 ms at p50 and 258 ms at p95 over 301 points,
measured in-process on the server. At 250 ms routing gave up on about 1 call in 20 before any
network time, and a budget under 300 ms, network included, would give up on more. 350 ms leaves
room for the network. The cost is bounded and safe: a call that misses the deadline keeps
continuity, as if routing were off; the wait falls only on an unmentioned message in a chat with
EnableDecisionRouting on, which is off by default; and it is at most this long. LLM Decision, the
failover, answered in about 520 ms at p50, so while Jev is unavailable routing times out and keeps
continuity. Revisit the figure, or the plan's budget, once client-side latency is measured.
How long, in milliseconds, routing waits for the decision before it keeps today's continuity. The call is abandoned client-side at this point, and the server is asked to bound its model call to the same figure.
The budget. The prompt-based intent check this replaces was removed at about 300 ms, so the decision has to come in under that, network included (plan Task 3.9).
This value is over that budget, deliberately. In the Phase 2 Decision Eval (plan Task 2.4, 2026-09-29) Jev answered this decision in 187 ms at p50 and 258 ms at p95 over 301 points, measured in-process on the server. At 250 ms routing gave up on about 1 call in 20 before any network time, and a budget under 300 ms, network included, would give up on more. 350 ms leaves room for the network. The cost is bounded and safe: a call that misses the deadline keeps continuity, as if routing were off; the wait falls only on an unmentioned message in a chat with
EnableDecisionRoutingon, which is off by default; and it is at most this long. LLM Decision, the failover, answered in about 520 ms at p50, so while Jev is unavailable routing times out and keeps continuity. Revisit the figure, or the plan's budget, once client-side latency is measured.