Rank currently missing cells or species for follow-up observation

Use this function with a fitted pigauto_result (returned by impute) to shortlist currently missing species-trait cells or species for follow-up measurement. It estimates expected BM variance reduction for continuous-family traits and label-propagation entropy reduction for discrete traits. These model-based scores can guide a shortlist, but they do not guarantee a gain in a field study or downstream analysis and have not been validated against field-sampling outcomes. They use the BM or label-propagation baseline formulas, not predictions from the fitted GNN.

Usage

suggest_next_observation(
  result,
  top_n = 10L,
  by = c("cell", "species"),
  types = c("continuous", "count", "ordinal", "proportion", "binary", "categorical",
    "zi_count", "multi_proportion")
)

# S3 method for class 'pigauto_active'
print(x, ...)

Arguments

result: A pigauto_result object returned by impute. Must have been produced from single-obs data; multi-obs inputs error with a clear message.
top_n: integer, default 10L. Number of suggestions to return. With by = "cell", rows are sorted globally by decreasing delta and truncated to the first top_n rows after applying the types filter. Set top_n large enough to retain the candidates you need before applying any further filters, then compare candidates within the same trait and metric; cross-trait or cross-metric scores do not represent a common biological sampling value. With by = "species", species are ranked by summed variance reduction when positive, otherwise by summed entropy reduction.
by: character, one of "cell" (default) or "species". "cell" returns individual (species, trait) pairs. "species" returns species, delta_var_total, delta_entropy_total, and n_traits_missing, with the two reductions summed separately across currently missing traits.
types: character vector of pigauto trait types to include. Default includes all eight supported types: continuous, count, ordinal, proportion, binary, categorical, zi_count, and multi_proportion.
x: A pigauto_active object returned by suggest_next_observation().
...: Further arguments passed to the data-frame print method when rankings are nonempty.

Value

A data.frame of class "pigauto_active". Columns when by = "cell": species, trait, type, metric ("variance" or "entropy"), delta, delta_var_total (NA for entropy-only rows), and delta_entropy_total (NA for variance-only rows), sorted globally by decreasing delta, including across metrics. Cross-metric order is approximate because the metrics have different units. When by = "species" and rows are returned, columns are: species, delta_var_total, delta_entropy_total, n_traits_missing, sorted by delta_var_total when positive and otherwise by delta_entropy_total. If no candidates remain, the empty result instead has the seven cell-mode columns and does not include n_traits_missing.

Invisibly returns x. For nonempty rankings, the printed object is converted to a plain data frame, so the returned object no longer has the "pigauto_active" class or its "by" attribute. An empty result returns the original object unchanged.

Details

A practical reading guide

Choose by = "cell" for species-trait candidates or by = "species" for a species-level shortlist.
For cell results, compare scores only within the same trait and metric. Variance and entropy use different units, and variance scores for different traits may use different transformed scales. Species totals are approximate across-trait rankings.
Set top_n large enough before filtering; the function truncates the ranking first. Use the scores to shortlist candidates, then choose field measurements using feasibility and biological priorities. The scores have not been validated against field outcomes.

Two metrics are supported, dispatched by trait type:

Continuous-family traits (continuous, count, ordinal, proportion) use the BM variance-reduction formula. This formula is derived from a Sherman-Morrison rank-1 inverse update on the BM conditional MVN: adding species $s$ to the observed set updates the inverse correlation matrix by a known closed form. The equation below gives the expected total variance reduction when candidate cell $(s, t)$ is observed. $$ \Delta V(s, t) = \sigma_t^2 \sum_{i \in \mathrm{miss}_t} \frac{D_{ik}^2}{\alpha_k} $$ where $D = R_{mm} - R_{mo} R_{oo}^{-1} R_{om}$ is the residual matrix at currently-missing cells, $\alpha_k = D_{kk}$ is the current relative leverage of cell $k$, and $\sigma_t^2$ is the REML BM variance for trait $t$. Within one trait, a larger $\Delta V$ means the model expects a greater reduction in the remaining BM prediction variance across missing cells if that candidate is measured.

These variance scores use the untransformed tree correlation (Pagel's $\lambda = 1$), even when the fitted baseline used another value, so their candidate ranking may differ from one calculated under the fitted baseline. Treat them as screening scores.

Discrete traits (binary, categorical) use a label-propagation expected-entropy-reduction formula. Here sim contains phylogenetic similarity weights between species; observed classes with larger weights contribute more to the LP probabilities. The current LP probability at species $i$ is $p_i = \mathrm{sim}[i, \mathrm{obs}] y_{\mathrm{obs}} / \sum \mathrm{sim}[i, \mathrm{obs}]$, with entropy $H(p_i) = -\sum_k p_{i,k} \log p_{i,k}$. After observing $s_{\mathrm{new}}$ with unknown class $y_{\mathrm{new}}$, the new LP probability has a closed form, and the expected entropy is averaged over $P(y_{\mathrm{new}})$ = current LP estimate at $s_{\mathrm{new}}$. Total expected entropy reduction sums across all currently-missing cells (the entropy at $s_{\mathrm{new}}$ itself drops to 0).

Scores require at least one missing cell. Continuous-family and multi-proportion variance scores require at least five observed cells. Binary and categorical entropy scores require at least one observed class. For zi_count, gate entropy requires at least one observed gate value; its magnitude variance contribution is zero when fewer than five non-zero magnitudes are observed. Traits that do not meet these conditions are omitted, so the result can be empty.

Variance and entropy are not directly comparable. For by = "cell", rows are sorted globally by delta and then truncated to top_n, including across metrics. Compare candidates within the same trait and metric. If you plan to filter the returned rows, set top_n large enough to retain the candidates you want; filtering cannot recover rows already removed by the limit. For by = "species", output has four columns: species, delta_var_total, delta_entropy_total, and n_traits_missing; the two totals are summed separately across that species' included missing traits. Species are ranked by delta_var_total when it is positive, otherwise by delta_entropy_total; this is only an approximate cross-metric ranking because the measures have different units.

Variance reductions across traits can also be measured on different transformed scales, so variance-only species totals are not a common measure of biological value. Treat these outputs as model-based screening scores: compare cell candidates within the same trait and metric, and use cross-trait species rankings to shortlist candidates rather than to claim equal sampling value. Choose among shortlisted candidates using field feasibility and biological priorities. The package does not validate these rankings against field-sampling outcomes.

zi_count: observing a missing zi_count cell reveals the gate value (entropy reduction at the gate column, computed via the LP binary formula) AND, with probability $p_{\mathrm{gate}}$ (current LP estimate at $s_{\mathrm{new}}$), reveals a magnitude (variance reduction at the magnitude column, computed via the BM Sherman-Morrison formula on the gate=1 subset). Output rows for zi_count populate BOTH delta_var_total (= expected magnitude variance reduction = $p_{\mathrm{gate}} \times \Delta V_{\mathrm{mag}}$) AND delta_entropy_total (= gate entropy reduction). metric is set to "variance" so the row sorts on the magnitude scale; delta_entropy_total is available for users who care about gate-uncertainty separately. The current magnitude calculation also treats structurally missing magnitudes for species with an observed zero gate as targets, although those values cannot be observed. Thus the total can overstate variance reduction across magnitudes that could actually be measured.

multi_proportion: observing a row reveals all K simplex components simultaneously. Per-component variance reductions are computed via BM Sherman-Morrison on each CLR-z latent column, summed across components. metric is "variance"; delta_var_total is the K-component sum.

Printing

The print method always reports the ranking mode and number of rows. Nonempty rankings are printed as a plain data frame, with ... forwarded to its print method, followed by the proxy-ranking caution. An empty result means no eligible candidates under the selected trait types and scoring requirements. Missing cells may remain when their trait does not meet a minimum-observation requirement. The console message reports candidate eligibility; it does not mean that the whole dataset is complete. The nonempty branch invisibly returns the plain data frame without the by attribute; the empty branch invisibly returns the original x.

Examples

# \donttest{
data(avonet300, tree300, package = "pigauto")
tree <- ape::keep.tip(tree300, tree300$tip.label[seq_len(30L)])
traits <- avonet300[match(tree$tip.label, avonet300$Species_Key),
                     c("Mass", "Wing.Length"), drop = FALSE]
rownames(traits) <- tree$tip.label
traits$Mass[seq_len(3L)] <- NA_real_
res <- impute(traits, tree, epochs = 5L, verbose = FALSE)
#> Warning: Small validation set for 2 trait(s): Mass (n=3), Wing.Length (n=4). Calibrated gate and conformal scores will be noisy for these trait(s). 95% split-conformal coverage is NOT achievable for 2 trait(s) with fewer than 19 validation cells (Mass (n=3), Wing.Length (n=4)): the achievable ceiling is n_val / (n_val + 1), which only reaches 0.95 at n_val >= 19. See `?fit_pigauto` under 'Calibration at small n' for smoothing options.
# Request enough rows before filtering to one trait.
ranked <- suggest_next_observation(res, top_n = 50L)
subset(ranked, trait == "Mass")
#> Active-imputation suggestions (by = cell, n = 3)
#>                  species trait       type   metric     delta delta_var_total
#> 1       Eudromia_formosa  Mass continuous variance 0.5128092       0.5128092
#> 2 Nothoprocta_pentlandii  Mass continuous variance 0.5128092       0.5128092
#> 3         Rhea_americana  Mass continuous variance 0.4431481       0.4431481
#>   delta_entropy_total
#> 1                  NA
#> 2                  NA
#> 3                  NA
#> 
#> Use these model-based proxy rankings with their stated assumptions; they are not an optimal sampling guarantee.
suggest_next_observation(res, top_n = 5L, by = "species")
#> Active-imputation suggestions (by = species, n = 3)
#>                  species delta_var_total delta_entropy_total n_traits_missing
#> 1       Eudromia_formosa       0.5128092                   0                1
#> 2 Nothoprocta_pentlandii       0.5128092                   0                1
#> 3         Rhea_americana       0.4431481                   0                1
#> 
#> Use these model-based proxy rankings with their stated assumptions; they are not an optimal sampling guarantee.
# }