Performs stratified k-fold cross-validation by rotating which cells serve as the test set. Returns per-fold and aggregated metrics.
Arguments
- data
pigauto_data object (output of
preprocess_traits).- tree
phylo object.
- k
integer. Number of folds (default 5).
- seeds
optional integer vector. Seeds for replicate runs. The default
NULLperforms one run using the current RNG stream.- epochs
integer. Training epochs per fold (default 500).
- verbose
logical. Print progress (default TRUE).
- ...
Additional arguments passed to
fit_pigauto(e.g.hidden_dim,k_eigen,use_attention).
Value
A list of class "pigauto_cv" with:
- results
Data.frame with columns: fold, rep, trait, type, metric, value.
- summary
Data.frame with mean and sd across folds/reps for each trait + metric.
- conformal_coverage
Data.frame of coverage per trait across folds (if available).
- k
Number of folds.
- n_reps
Number of replicates.
Examples
# \donttest{
data(avonet300, tree300, package = "pigauto")
tree <- ape::keep.tip(tree300, tree300$tip.label[seq_len(12L)])
traits <- avonet300[match(tree$tip.label, avonet300$Species_Key),
c("Mass", "Wing.Length"), drop = FALSE]
rownames(traits) <- tree$tip.label
cv <- cross_validate(preprocess_traits(traits, tree), tree, k = 3L,
seeds = 1L, epochs = 1L, verbose = FALSE,
k_eigen = 4L, hidden_dim = 8L,
use_transformer_blocks = FALSE)
#> Warning: phylo_signal_gate requires the 'phytools' package; returning NA for all traits.
#> Warning: Small validation set for 2 trait(s): Mass (n=4), Wing.Length (n=4). Calibrated gate and conformal scores will be noisy for these trait(s). 95% split-conformal coverage is NOT achievable for 2 trait(s) with fewer than 19 validation cells (Mass (n=4), Wing.Length (n=4)): the achievable ceiling is n_val / (n_val + 1), which only reaches 0.95 at n_val >= 19. See `?fit_pigauto` under 'Calibration at small n' for smoothing options.
#> Warning: the standard deviation is zero
#> Warning: the standard deviation is zero
#> Warning: phylo_signal_gate requires the 'phytools' package; returning NA for all traits.
#> Warning: Small validation set for 2 trait(s): Mass (n=4), Wing.Length (n=4). Calibrated gate and conformal scores will be noisy for these trait(s). 95% split-conformal coverage is NOT achievable for 2 trait(s) with fewer than 19 validation cells (Mass (n=4), Wing.Length (n=4)): the achievable ceiling is n_val / (n_val + 1), which only reaches 0.95 at n_val >= 19. See `?fit_pigauto` under 'Calibration at small n' for smoothing options.
#> Warning: the standard deviation is zero
#> Warning: the standard deviation is zero
#> Warning: the standard deviation is zero
#> Warning: the standard deviation is zero
#> Warning: phylo_signal_gate requires the 'phytools' package; returning NA for all traits.
#> Warning: Small validation set for 2 trait(s): Mass (n=4), Wing.Length (n=4). Calibrated gate and conformal scores will be noisy for these trait(s). 95% split-conformal coverage is NOT achievable for 2 trait(s) with fewer than 19 validation cells (Mass (n=4), Wing.Length (n=4)): the achievable ceiling is n_val / (n_val + 1), which only reaches 0.95 at n_val >= 19. See `?fit_pigauto` under 'Calibration at small n' for smoothing options.
print(cv)
#> 3-fold cross-validation (1 replicate)
#> ──────────────────────────────────────────────────
#> Mass [continuous]
#> rmse 0.9299 +/- 0.3663 (n=3)
#> coverage_95 0.8333 +/- 0.1443 (n=3)
#> pearson_r 0.6226 +/- NA (n=1)
#> Wing.Length [continuous]
#> rmse 1.1607 +/- 0.6091 (n=3)
#> pearson_r 0.2234 +/- 0.3873 (n=2)
#> coverage_95 0.8333 +/- 0.2887 (n=3)
#>
#> Conformal coverage (95% target):
#> Mass 0.833 +/- 0.289
#> Wing.Length 0.750 +/- 0.250
summary(cv)
#> 3-fold cross-validation (1 replicate)
#> ──────────────────────────────────────────────────────────────────────
#> Trait Type RMSE (mean+/-sd) r (mean+/-sd) Acc (mean+/-sd)
#> ─────────────────────────────────────────────────────────────────────────────────────
#> Mass continuous 0.930 +/- 0.366 0.623 +/- NA ─
#> Wing.Length continuous 1.161 +/- 0.609 0.223 +/- 0.387 ─
# }