Extract the normalized leaderboard score from a model's holdout metrics
(preferred — the honest, search-never-saw-it number, §8.2), falling back to
the training metrics. The score is normalized so higher is always better:
error metrics (RMSE/MAE/loss) are negated, ranking metrics (AUC/F1/accuracy)
pass through. Returns 0 when the metric is absent (a failed/empty run).
Exported for unit testing the normalization in isolation.
Parameters
holdoutMetricsJson: string
the model's HoldoutMetrics JSON (preferred)
trainMetricsJson: string
the model's Metrics JSON (fallback)
successMetric: string
the plan's SuccessMetric (e.g. AUC, RMSE)
problemType: "classification"|"regression"
classification vs. regression (informs the error-metric direction)
Extract the normalized leaderboard score from a model's holdout metrics (preferred — the honest, search-never-saw-it number, §8.2), falling back to the training metrics. The score is normalized so higher is always better: error metrics (RMSE/MAE/loss) are negated, ranking metrics (AUC/F1/accuracy) pass through. Returns
0when the metric is absent (a failed/empty run).Exported for unit testing the normalization in isolation.