Evaluate imputation performance against known values — evaluate

Computes type-specific metrics for each trait on the validation and test splits. When a trait_map is supplied, metrics are dispatched per trait type; otherwise the function falls back to continuous-only metrics (RMSE, Pearson r, 95% coverage).

Usage

evaluate_imputation(pred, truth, splits, pred_se = NULL, trait_map = NULL)

Arguments

pred: predicted values: either a numeric matrix in latent scale (same dimensions as truth), or a "pigauto_pred" object from predict.pigauto_fit.
truth: numeric matrix of true values in latent scale (from pigauto_data$X_scaled).
splits: list (output of make_missing_splits).
pred_se: numeric matrix of prediction SEs on the scale of the predictions being evaluated. For a pigauto_pred, this means the latent predictions in pred$imputed_latent; it does not accept original-scale SEs as an override. Used for empirical coverage of the symmetric $\hat y +/- 1.96 SE$ interval on the latent scale for continuous and proportion traits. This coverage_95 is distinct from the conformal bound coverage metric reported by evaluate(). When pred is a pigauto_pred, an explicit pred_se takes precedence; otherwise its se_latent is used when available.
trait_map: list of trait descriptors (from pigauto_data). If NULL and pred is not a pigauto_pred, the v0.1 all-continuous evaluation is used.

Value

A data.frame with columns split, trait, type, n, and type-specific metric columns.

Details

Predictions must exclude the cells being scored from both fitting and prediction inputs. Use predict(fit, evaluation = TRUE) or evaluate(fit). Plain predict(fit) uses observed values for production imputation and must not be scored as held-out data. Predictions with provenance are rejected if scored indices were not held out. For raw matrices or older prediction objects this cannot be checked; callers must ensure the cells were held out themselves.

Metrics by trait type:

continuous: RMSE on the latent scale, Pearson r, and coverage_95: empirical coverage of the symmetric $\hat y +/- 1.96 SE$ interval on that scale, when SE is supplied
proportion: RMSE on the latent scale, Pearson r, and coverage_95 on that scale, when SE is supplied
count: RMSE, MAE, and Pearson r on the z-scored log1p latent scale
ordinal: RMSE on the z-scored latent scale, Spearman rho
binary: Accuracy, Brier score
categorical: Accuracy
zi_count: RMSE, MAE, and Pearson r on the original count scale, comparing observed counts with predicted expected counts; zero-accuracy and Brier score on the zero/nonzero gate
multi_proportion: Aitchison distance in CLR space, RMSE on the z-scored CLR latent scale, and MAE on the decoded simplex

For binary and categorical traits the function accepts either a pigauto_pred object (preferred, gives access to probabilities) or raw matrices (latent scale).

Examples

# \donttest{
truth <- matrix(c(1, 2, 3, 4), nrow = 2L)
splits <- list(test_idx = 1L, val_idx = 2L, n = 2L, p = 2L)
# Equal inputs illustrate the interface and output shape only: RMSE is zero
# by construction, correlation is unavailable with one cell per split, and
# this is not a performance evaluation.
evaluate_imputation(truth, truth, splits)
#>      split  trait       type n rmse pearson_r coverage_95 mae spearman_rho
#> val    val trait1 continuous 1    0        NA          NA  NA           NA
#> test  test trait1 continuous 1    0        NA          NA  NA           NA
#>      accuracy brier zero_accuracy aitchison rmse_clr simplex_mae
#> val        NA    NA            NA        NA       NA          NA
#> test       NA    NA            NA        NA       NA          NA
# }