compare {loo} | R Documentation

## Model comparison

### Description

This function will be deprecated in a future release. We recommend using the new `loo_compare` function instead.

### Usage

    compare(..., x = list())

### Arguments

`...` | At least two objects returned by `loo` (or `waic`).
`x` | A list of at least two objects returned by `loo` (or `waic`). This argument can be used as an alternative to specifying the objects in `...`.

### Details

When comparing two fitted models, we can estimate the difference in their expected predictive accuracy by the difference in `elpd_loo` or `elpd_waic` (or multiplied by -2, if desired, to be on the deviance scale).

When that difference, `elpd_diff`, is positive then the expected predictive accuracy for the second model is higher. A negative `elpd_diff` favors the first model.

When using `compare()` with more than two models, the values in the `elpd_diff` and `se_diff` columns of the returned matrix are computed by making pairwise comparisons between each model and the model with the best ELPD (i.e., the model in the first row). Although the `elpd_diff` column is equal to the difference in `elpd_loo`, do not expect the `se_diff` column to be equal to the the difference in `se_elpd_loo`.

To compute the standard error of the difference in ELPD we use a paired estimate to take advantage of the fact that the same set of N data points was used to fit both models. These calculations should be most useful when N is large, because then non-normality of the distribution is not such an issue when estimating the uncertainty in these sums. These standard errors, for all their flaws, should give a better sense of uncertainty than what is obtained using the current standard approach of comparing differences of deviances to a Chi-squared distribution, a practice derived for Gaussian linear models or asymptotically, and which only applies to nested models in any case.

### Value

A vector or matrix with class `'compare.loo'` that has its own print method. If exactly two objects are provided in `...` or `x`, then the difference in expected predictive accuracy and the standard error of the difference are returned. If more than two objects are provided then a matrix of summary information is returned (see Details).

### References

Vehtari, A., Gelman, A., and Gabry, J. (2017a). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. 27(5), 1413–1432. doi:10.1007/s11222-016-9696-4. ( journal, preprint arXiv:1507.04544).

Vehtari, A., Gelman, A., and Gabry, J. (2017b). Pareto smoothed importance sampling. arXiv preprint: http://arxiv.org/abs/1507.02646/

### Examples

    ## Not run:
    loo1 <- loo(log_lik1)
    loo2 <- loo(log_lik2)
    print(compare(loo1, loo2), digits = 3)
    print(compare(x = list(loo1, loo2)))

    waic1 <- waic(log_lik1)
    waic2 <- waic(log_lik2)
    compare(waic1, waic2)

    ## End(Not run)

[Package loo version 2.1.0 Index]