# Accessing the contents of a stanfit object

#### Stan Development Team

#### 2017-12-20

This vignette demonstrates how to access most of data stored in a stanfit object. A stanfit object (an object of class `"stanfit"`) contains the output derived from fitting a Stan model using Markov chain Monte Carlo or one of Stan’s variational approximations (meanfield or full-rank). Throughout the document we’ll use the stanfit object obtained from fitting the Eight Schools example model:

    library(rstan)
    fit <- stan_demo("eight_schools", refresh = 0)

    Warning: There were 3 divergent transitions after warmup. Increasing adapt_delta above 0.8 may help. See
    http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup

    Warning: Examine the pairs() plot to diagnose sampling problems

    class(fit)

    [1] "stanfit"
    attr(,"package")
    [1] "rstan"

## Posterior draws

There are several functions that can be used to access the draws from the posterior distribution stored in a stanfit object. These are `extract`, `as.matrix`, `as.data.frame`, and `as.array`, each of which returns the draws in a different format.

#### extract()

The `extract` function (with its default arguments) returns a list with named components corresponding to the model parameters.

    list_of_draws <- extract(fit)
    print(names(list_of_draws))

    [1] "mu"    "tau"   "eta"   "theta" "lp__"

In this model the parameters `mu` and `tau` are scalars and `theta` is a vector with eight elements. This means that the draws for `mu` and `tau` will be vectors (with length equal to the number of post-warmup iterations times the number of chains) and the draws for `theta` will be a matrix, with each column corresponding to one of the eight components:

    head(list_of_draws$mu)

    [1]  7.0159958  4.2101908  6.1037848 14.8239628  0.1161633  3.9453097

    head(list_of_draws$tau)

    [1]  3.908264 10.407696  6.246178  1.820808 18.081780  3.520686

    head(list_of_draws$theta)

    iterations      [,1]      [,2]       [,3]      [,4]       [,5]      [,6]
          [1,]  4.747945 15.101442 -0.4517133  7.122441  0.8912814 13.322522
          [2,] 19.884102  7.097918 -3.1776219  5.064851 -4.8662635  4.650350
          [3,] 12.615291 15.855845  2.9353426 11.450746  2.5903338 13.193853
          [4,] 13.241630 13.597602 14.8809754 15.058191 13.3526916 15.543044
          [5,] 16.700569 -8.382898 -9.5266002 -1.275855  1.9278800 -1.770517
          [6,]  6.415568  4.257496  9.2237422  8.368015  1.0804516 10.311312

    iterations      [,7]      [,8]
          [1,]  9.402782  9.786535
          [2,]  9.920389  5.556741
          [3,] 11.609662 12.162340
          [4,] 14.668241 18.416541
          [5,]  3.900139 12.378304
          [6,]  7.566363  2.586790

#### as.matrix(), as.data.frame(), as.array()

The `as.matrix`, `as.data.frame`, and `as.array` functions can also be used to retrieve the posterior draws from a stanfit object:

    matrix_of_draws <- as.matrix(fit)
    print(colnames(matrix_of_draws))

     [1] "mu"       "tau"      "eta[1]"   "eta[2]"   "eta[3]"   "eta[4]"
     [7] "eta[5]"   "eta[6]"   "eta[7]"   "eta[8]"   "theta[1]" "theta[2]"
    [13] "theta[3]" "theta[4]" "theta[5]" "theta[6]" "theta[7]" "theta[8]"
    [19] "lp__"

    df_of_draws <- as.data.frame(fit)
    print(colnames(df_of_draws))

     [1] "mu"       "tau"      "eta[1]"   "eta[2]"   "eta[3]"   "eta[4]"
     [7] "eta[5]"   "eta[6]"   "eta[7]"   "eta[8]"   "theta[1]" "theta[2]"
    [13] "theta[3]" "theta[4]" "theta[5]" "theta[6]" "theta[7]" "theta[8]"
    [19] "lp__"

    array_of_draws <- as.array(fit)
    print(dimnames(array_of_draws))

    $iterations
    NULL

    $chains
    [1] "chain:1" "chain:2" "chain:3" "chain:4"

    $parameters
     [1] "mu"       "tau"      "eta[1]"   "eta[2]"   "eta[3]"   "eta[4]"
     [7] "eta[5]"   "eta[6]"   "eta[7]"   "eta[8]"   "theta[1]" "theta[2]"
    [13] "theta[3]" "theta[4]" "theta[5]" "theta[6]" "theta[7]" "theta[8]"
    [19] "lp__"

The `as.matrix` and `as.data.frame` methods essentially return the same thing except in matrix and data frame form, respectively. The `as.array` method returns the draws from each chain separately and so has an additional dimension:

    print(dim(matrix_of_draws))
    print(dim(df_of_draws))
    print(dim(array_of_draws))

    [1] 4000   19
    [1] 4000   19
    [1] 1000    4   19

By default all of the functions for retrieving the posterior draws return the draws for all parameters (and generated quantities). The optional argument `pars` (a character vector) can be used if only a subset of the parameters is desired, for example:

    mu_and_theta1 <- as.matrix(fit, pars = c("mu", "theta[1]"))
    head(mu_and_theta1)

              parameters
    iterations         mu  theta[1]
          [1,]  7.8177242 12.274575
          [2,]  5.5542745 19.295752
          [3,]  6.6150886 -1.547473
          [4,] -0.1509324 15.382143
          [5,]  5.7539854  4.481522
          [6,] 14.9108815 16.461486

## Posterior summary statistics and convergence diagnostics

Summary statistics are obtained using the `summary` function. The object returned is a list with two components:

    fit_summary <- summary(fit)
    print(names(fit_summary))

    [1] "summary"   "c_summary"

In `fit_summary$summary` all chains are merged whereas `fit_summary$c_summary` contains summaries for each chain individually. Typically we want the summary for all chains merged, which is what we’ll focus on here.

The summary is a matrix with rows corresponding to parameters and columns to the various summary quantities. These include the posterior mean, the posterior standard deviation, and various quantiles computed from the draws. The `probs` argument can be used to specify which quantiles to compute and `pars` can be used to specify a subset of parameters to include in the summary.

For models fit using MCMC, also included in the summary are the Monte Carlo standard error (`se_mean`), the effective sample size (`n_eff`), and the R-hat statistic (`Rhat`).

    print(fit_summary$summary)

                     mean    se_mean        sd        2.5%         25%
    mu         8.00612673 0.11357604 5.0124652  -1.9086395   4.8304935
    tau        6.52407106 0.14399326 5.5035364   0.2570695   2.4435186
    eta[1]     0.39027950 0.01450986 0.9176839  -1.4151746  -0.2229871
    eta[2]    -0.02714791 0.01608620 0.8959665  -1.7494774  -0.6061099
    eta[3]    -0.19937176 0.01609075 0.9467837  -2.0330935  -0.8247246
    eta[4]    -0.04883221 0.01510897 0.8983992  -1.7732438  -0.6315857
    eta[5]    -0.33545441 0.01632818 0.8862725  -2.0481661  -0.9334793
    eta[6]    -0.19117708 0.01478583 0.8904066  -1.9363809  -0.7732976
    eta[7]     0.34017489 0.01771714 0.9021566  -1.5233884  -0.2258147
    eta[8]     0.01895383 0.01446641 0.9149361  -1.7638083  -0.5910187
    theta[1]  11.37366770 0.16177632 8.0963818  -1.6514378   5.9568906
    theta[2]   7.73729291 0.09799283 6.1976106  -4.8845294   4.0240734
    theta[3]   6.16652988 0.13874660 7.7020374 -10.7058733   1.9540518
    theta[4]   7.64577369 0.10361556 6.5532231  -5.7333372   3.8078704
    theta[5]   5.22178112 0.09778443 6.1844306  -8.7164617   1.6760337
    theta[6]   6.23133034 0.10534064 6.6623268  -8.0021844   2.4454840
    theta[7]  10.68129630 0.11793775 6.8087866  -0.9250543   6.0588372
    theta[8]   8.27486601 0.12834062 7.7982092  -7.8550364   3.7797876
    lp__     -39.56020206 0.07603432 2.6543214 -45.4155605 -41.2887323
                      50%         75%      97.5%    n_eff      Rhat
    mu         7.87342502  11.1025185  18.564115 1947.732 0.9998880
    tau        5.18273101   9.0970853  20.810654 1460.829 1.0010391
    eta[1]     0.38800996   1.0132058   2.124973 4000.000 0.9999969
    eta[2]    -0.04433177   0.5405025   1.724804 3102.250 1.0001709
    eta[3]    -0.20644973   0.4497925   1.600744 3462.176 0.9997832
    eta[4]    -0.05882383   0.5234715   1.776734 3535.649 1.0001126
    eta[5]    -0.35370307   0.2366283   1.478178 2946.177 1.0004805
    eta[6]    -0.19964870   0.3781558   1.655883 3626.478 1.0000576
    eta[7]     0.35056871   0.9236925   2.124564 2592.847 1.0002625
    eta[8]     0.02741137   0.6125757   1.828757 4000.000 0.9999567
    theta[1]  10.07420022  15.5148370  30.960803 2504.679 1.0006976
    theta[2]   7.60894835  11.4360902  20.607717 4000.000 1.0003381
    theta[3]   6.65585308  10.9089134  20.459568 3081.531 1.0007760
    theta[4]   7.63119374  11.4872849  21.621038 4000.000 1.0001057
    theta[5]   5.66053208   9.3108242  16.291796 4000.000 1.0001110
    theta[6]   6.56772243  10.5400815  18.722200 4000.000 1.0001475
    theta[7]  10.11963338  14.5122164  26.678408 3332.988 1.0003770
    theta[8]   8.09714473  12.5420376  24.960645 3691.998 0.9997813
    lp__     -39.31327130 -37.6471281 -34.997487 1218.674 1.0020585

If, for example, we wanted the only quantiles included to be 10% and 90%, and for only the parameters included to be `mu` and `tau`, we would specify that like this:

    mu_tau_summary <- summary(fit, pars = c("mu", "tau"), probs = c(0.1, 0.9))$summary
    print(mu_tau_summary)

            mean   se_mean       sd       10%      90%    n_eff     Rhat
    mu  8.006127 0.1135760 5.012465 1.8813988 14.34102 1947.732 0.999888
    tau 6.524071 0.1439933 5.503536 0.9256796 13.68029 1460.829 1.001039

Since `mu_tau_summary` is a matrix we can pull out columns using their names:

    mu_tau_80pct <- mu_tau_summary[, c("10%", "90%")]
    print(mu_tau_80pct)

              10%      90%
    mu  1.8813988 14.34102
    tau 0.9256796 13.68029

## Sampler diagnostics

For models fit using MCMC the stanfit object will also contain the values of parameters used for the sampler. The `get_sampler_params` function can be used to access this information.

The object returned by `get_sampler_params` is a list with one component (a matrix) per chain. Each of the matrices has number of columns corresponding to the number of sampler parameters and the column names provide the parameter names. The optional argument inc_warmup (defaulting to `TRUE`) indicates whether to include the warmup period.

    sampler_params <- get_sampler_params(fit, inc_warmup = FALSE)
    sampler_params_chain1 <- sampler_params[[1]]
    colnames(sampler_params_chain1)

    [1] "accept_stat__" "stepsize__"    "treedepth__"   "n_leapfrog__"
    [5] "divergent__"   "energy__"

To do things like calculate the average value of `accept_stat__` for each chain (or the maximum value of `treedepth__` for each chain if using the NUTS algorithm, etc.) the `sapply` function is useful as it will apply the same function to each component of `sampler_params`:

    mean_accept_stat_by_chain <- sapply(sampler_params, function(x) mean(x[, "accept_stat__"]))
    print(mean_accept_stat_by_chain)

    [1] 0.8753160 0.8383744 0.8553326 0.7967343

    max_treedepth_by_chain <- sapply(sampler_params, function(x) max(x[, "treedepth__"]))
    print(max_treedepth_by_chain)

    [1] 4 5 5 5

## Model code

The Stan program itself is also stored in the stanfit object and can be accessed using `get_stancode`:

    code <- get_stancode(fit)

The object `code` is a single string and is not very intelligible when printed:

    print(code)

    [1] "data {\n  int<lower=0> J;          // number of schools \n  real y[J];               // estimated treatment effects\n  real<lower=0> sigma[J];  // s.e. of effect estimates \n}\nparameters {\n  real mu; \n  real<lower=0> tau;\n  vector[J] eta;\n}\ntransformed parameters {\n  vector[J] theta;\n  theta = mu + tau * eta;\n}\nmodel {\n  target += normal_lpdf(eta | 0, 1);\n  target += normal_lpdf(y | theta, sigma);\n}"
    attr(,"model_name2")
    [1] "schools"

A readable version can be printed using `cat`:

    cat(code)

    data {
      int<lower=0> J;          // number of schools
      real y[J];               // estimated treatment effects
      real<lower=0> sigma[J];  // s.e. of effect estimates
    }
    parameters {
      real mu;
      real<lower=0> tau;
      vector[J] eta;
    }
    transformed parameters {
      vector[J] theta;
      theta = mu + tau * eta;
    }
    model {
      target += normal_lpdf(eta | 0, 1);
      target += normal_lpdf(y | theta, sigma);
    }

## Initial values

The `get_inits` function returns initial values as a list with one component per chain. Each component is itself a (named) list containing the initial values for each parameter for the corresponding chain:

    inits <- get_inits(fit)
    inits_chain1 <- inits[[1]]
    print(inits_chain1)

    $mu
    [1] 1.471246

    $tau
    [1] 3.32849

    $eta
    [1] -0.5277211  1.6655413 -1.3473345  1.0731082  1.1316256  0.4185387
    [7] -0.8133071 -1.6184322

    $theta
    [1] -0.2852688  7.0149828 -3.0133436  5.0430753  5.2378500  2.8643473
    [7] -1.2358387 -3.9156894

## (P)RNG seed

The `get_seed` function returns the (P)RNG seed as an integer:

    print(get_seed(fit))

    [1] 1498565053

## Warmup and sampling times

The `get_elapsed_time` function returns a matrix with the warmup and sampling times for each chain:

    print(get_elapsed_time(fit))

              warmup   sample
    chain:1 0.032646 0.031583
    chain:2 0.038124 0.029785
    chain:3 0.036940 0.033690
    chain:4 0.033448 0.023040