Machine Learning Metrics

Sentence BLEU (1-4 gram, brevity penalty, optional smoothing) and ROUGE-1, ROUGE-2, ROUGE-L F1 for text against references.

Reliability bins, expected and maximum calibration error and Brier score from binary labels and predicted probabilities.

Confusion matrix from true and predicted labels: accuracy, precision, recall, specificity, F1, MCC, kappa, averages.

Binary or multiclass log loss (cross entropy) with epsilon clipping, plus the no-skill baseline and worst samples.

Precision-recall curve points and average precision (step-wise sum) from binary labels and scores, with the baseline.

MAE, MSE, RMSE, R2, adjusted R2, MAPE, SMAPE, explained variance, median and max error from actual vs predicted numbers.

ROC curve points and AUC from binary labels and scores via the Mann-Whitney rank-sum identity, with ties counted half.

Sweep every score threshold for binary labels and return the best F1, Youden J and accuracy thresholds with their metrics.