Evaluate a fitted pigauto model on its test set

Computes type-specific performance metrics for each trait. By default the model is evaluated on the test split stored in the fit object, but an alternative data / splits can be supplied.

Usage

evaluate(fit, data = NULL, splits = NULL)

Arguments

fit

pigauto_fit object.

data

pigauto_data object (default: NULL, uses the training data stored in the fit via predict()).

splits

splits object (default: NULL, uses fit$splits).

Value

A data.frame with columns: method, trait, type, metric, value, n_test.

Details

Metrics by trait type:

continuous

RMSE, Pearson r, MAE

count

RMSE, MAE, Pearson r

binary

Accuracy, Brier score

categorical

Accuracy (overall)

ordinal

RMSE (on integer scale), Spearman rho

When conformal scores are present in the fit, conformal coverage at the 95\ ordinal traits.

When the fit includes a baseline, baseline metrics are appended with method = "baseline" for direct comparison.

Examples

# \donttest{
data(avonet300, tree300)
tree <- ape::keep.tip(tree300, tree300$tip.label[seq_len(30L)])
traits <- avonet300[match(tree$tip.label, avonet300$Species_Key),
                     c("Mass", "Wing.Length"), drop = FALSE]
rownames(traits) <- tree$tip.label
pd <- preprocess_traits(traits, tree)
splits <- make_missing_splits(pd$X_scaled, trait_map = pd$trait_map)
fit <- fit_pigauto(pd, tree, splits = splits, epochs = 5L,
                   verbose = FALSE)
#> Error: Lantern is not loaded. Please use `install_torch()` to install additional dependencies.
eval_df <- evaluate(fit, data = pd)
#> Error: object 'fit' not found
eval_df[eval_df$metric == "rmse", ]
#> Error: object 'eval_df' not found
# }