Platt calibration of the discovery answers, each tied to the exact model it was fitted on (see
FindDecisionCalibration in @memberjunction/ai-core-plus): the MJ decision model that answered
(ModelName, from modelInfo.modelName) and the model its driver reports behind it
(ResolvedModel, from DecisionResult.ResolvedModel). A model's raw answers are not calibrated:
Jev's raw any-applies Likelihood sits between 0.1 and 0.8 whatever the request. Any other model,
including Jev at another version or LLM Decision answered by another chat model, is treated as
unsure, so its answer suggests nothing.
Fitted on the agent-discovery Decision Eval (2026-09-29; 258 requests, 3 repeats per model, 774
answers each): the Choice's confidence against whether it named the labelled agent (594 answers
to requests meant for an agent), and the Likelihood against whether the request was meant for an
agent (all 774). Every fit converged (5 to 7 Newton iterations; a 60-digit refit agrees to 4
places).
Jev at its pinned APIName, typesafe/jev-1.13-20260917, which it reports back as the
resolved model.
LLM Decision when its chat model is GPT-OSS-120B, which answered all 774 of its runs.
Refit, and add the new pair, whenever a model, its version, the questions or the catalog's shape
changes.
Platt calibration of the discovery answers, each tied to the exact model it was fitted on (see
FindDecisionCalibrationin@memberjunction/ai-core-plus): the MJ decision model that answered (ModelName, frommodelInfo.modelName) and the model its driver reports behind it (ResolvedModel, fromDecisionResult.ResolvedModel). A model's raw answers are not calibrated: Jev's raw any-applies Likelihood sits between 0.1 and 0.8 whatever the request. Any other model, including Jev at another version orLLM Decisionanswered by another chat model, is treated as unsure, so its answer suggests nothing.Fitted on the agent-discovery Decision Eval (2026-09-29; 258 requests, 3 repeats per model, 774 answers each): the Choice's confidence against whether it named the labelled agent (594 answers to requests meant for an agent), and the Likelihood against whether the request was meant for an agent (all 774). Every fit converged (5 to 7 Newton iterations; a 60-digit refit agrees to 4 places).
APIName,typesafe/jev-1.13-20260917, which it reports back as the resolved model.Refit, and add the new pair, whenever a model, its version, the questions or the catalog's shape changes.