Generates trait data under various evolutionary models, introduces missing data, fits both the Brownian motion baseline and the full pigauto GNN, and compares performance. This is the recommended way to assess pigauto on data with known properties before applying it to real data.
Usage
simulate_benchmark(
n_species = 100L,
n_traits = 4L,
scenarios = c("BM", "OU", "regime_shift", "nonlinear", "mixed"),
missing_frac = 0.25,
n_reps = 3L,
epochs = 500L,
verbose = TRUE,
seed = NULL,
gnn = TRUE,
...
)Arguments
- n_species
integer. Number of tips in the simulated tree (default 100).
- n_traits
integer. Number of continuous traits (default 4). Ignored for
scenario = "mixed", which generates a fixed trait set.- scenarios
character vector. Subset of
c("BM", "OU", "regime_shift", "nonlinear", "mixed"). Default runs all.- missing_frac
numeric. Fraction of observed cells held out (default 0.25).
- n_reps
integer. Number of replicate trees per scenario (default 3).
- epochs
integer. Maximum GNN training epochs (default 500).
- verbose
logical. Print progress (default
TRUE).- seed
optional integer. When supplied, makes the generated benchmark replicates reproducible.
- gnn
logical. Whether the pigauto arm trains the GNN. Default
TRUE, so this function keeps comparing the baseline with the GNN-corrected model even thoughfit_pigautoandimputedefault tognn = FALSE.- ...
additional arguments passed to
fit_pigauto.
Value
An object of class "pigauto_benchmark" with:
- results
data.frame with columns:
scenario,rep,method,trait,type,metric,value,n_test.- summary
data.frame averaged across replicates.
- scenarios
character vector of scenarios run.
- n_reps
integer.
- n_species
integer.
Details
Available scenarios:
"BM"Pure Brownian motion – the baseline is exact, so the GNN should tie or slightly improve via inter-trait correlations.
"OU"Ornstein-Uhlenbeck – stabilising selection constrains variation. BM over-estimates evolutionary variance.
"regime_shift"Two-regime BM – clade-specific optima create bimodal distributions that BM cannot capture.
"nonlinear"Non-linear inter-trait relationships – the GNN's multi-layer message passing can capture quadratic and interaction effects that BM's linear covariance misses.
"mixed"Mixed trait types: 2 continuous + 1 binary + 1 categorical (3 levels). Tests the full type pipeline.
Examples
# \donttest{
bench <- simulate_benchmark(n_species = 20L, n_traits = 2L,
scenarios = "BM", epochs = 5L, n_reps = 1L,
verbose = FALSE)
#> Warning: phylo_signal_gate requires the 'phytools' package; returning NA for all traits.
#> Warning: Small validation set for 2 trait(s): trait1 (n=1), trait2 (n=1). Calibrated gate and conformal scores will be noisy for these trait(s). 95% split-conformal coverage is NOT achievable for 2 trait(s) with fewer than 19 validation cells (trait1 (n=1), trait2 (n=1)): the achievable ceiling is n_val / (n_val + 1), which only reaches 0.95 at n_val >= 19. See `?fit_pigauto` under 'Calibration at small n' for smoothing options.
bench$summary
#> scenario method trait type metric mean sd
#> 1 BM baseline trait1 continuous mae 0.7581627 NA
#> 2 BM baseline trait1 continuous pearson_r 0.9037474 NA
#> 3 BM baseline trait1 continuous rmse 0.8727888 NA
#> 4 BM baseline trait2 continuous mae 0.6409889 NA
#> 5 BM baseline trait2 continuous pearson_r NaN NA
#> 6 BM baseline trait2 continuous rmse 0.6831186 NA
#> 7 BM pigauto trait1 continuous conformal_coverage_95 0.1666667 NA
#> 8 BM pigauto trait1 continuous mae 0.7581627 NA
#> 9 BM pigauto trait1 continuous pearson_r 0.9037474 NA
#> 10 BM pigauto trait1 continuous rmse 0.8727888 NA
#> 11 BM pigauto trait2 continuous conformal_coverage_95 0.0000000 NA
#> 12 BM pigauto trait2 continuous mae 0.6409889 NA
#> 13 BM pigauto trait2 continuous pearson_r NaN NA
#> 14 BM pigauto trait2 continuous rmse 0.6831186 NA
#> n_reps
#> 1 1
#> 2 1
#> 3 1
#> 4 1
#> 5 0
#> 6 1
#> 7 1
#> 8 1
#> 9 1
#> 10 1
#> 11 1
#> 12 1
#> 13 0
#> 14 1
plot(bench)
# }