\documentclass[article,nojss]{jss}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% declarations for jss.cls %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\input{myCommands.sty}
\usepackage{amsmath}
\usepackage{graphicx}
%% almost as usual
\author{Devin Francom\\University of California Santa Cruz \And 
        Bruno Sans\'{o}\\University of California Santa Cruz}
\title{\pkg{BASS}: An \proglang{R} Package for Fitting and Performing Sensitivity Analysis of Bayesian Adaptive Spline Surfaces}

%% for pretty printing and a nice hypersummary also set:
\Plainauthor{Devin Francom, Bruno Sans\'{o}} %% comma-separated
\Plaintitle{BASS: An R Package for Sensitivity Analysis and Fitting of Bayesian Adaptive Spline Surfaces} %% without formatting
\Shorttitle{\pkg{BASS}: Bayesian Adaptive Spline Surfaces} %% a short title (if necessary)

%% an abstract and keywords
\Abstract{
  We present the \proglang{R} package \pkg{BASS} as a tool for nonparametric regression.  The primary focus of the package is fitting fully Bayesian adaptive spline surface (BASS) models and performing global sensitivity analyses of these models.  The BASS framework is similar to that of Bayesian multivariate adaptive regression splines (BMARS) from \cite{denison1998bayesian}, but with many added features.  The software is built to efficiently handle significant amounts of data with many continuous or categorical predictors and with functional response.  Under our Bayesian framework, most priors are automatic but these can be modified by the user to focus on parsimony and the avoidance of overfitting.  If directed to do so, the software uses parallel tempering to improve the reversible jump Markov chain Monte Carlo (RJMCMC) methods used to perform inference.  We discuss the implementation of these features and present the performance of \pkg{BASS} in a number of analyses of simulated and real data.
}
\Keywords{splines, functional data analysis, sensitivity analysis, nonparametric regression}
\Plainkeywords{splines, functional data analysis, sensitivity analysis, nonlinear regression} %% without formatting
%% at least one keyword must be supplied

%% publication information
%% NOTE: Typically, this can be left commented and will be filled out by the technical editor
%% \Volume{50}
%% \Issue{9}
%% \Month{June}
%% \Year{2012}
%% \Submitdate{2012-06-04}
%% \Acceptdate{2012-06-04}

%% The address of (at least) one author should be given
%% in the following format:
\Address{
  Devin Francom\\
  Department of Applied Mathematics and Statistics\\
  University of California Santa Cruz\\
  Santa Cruz, CA 95064\\
  E-mail: \email{dfrancom@ucsc.edu}
}
%% It is also possible to add a telephone and fax number
%% before the e-mail in the following format:
%% Telephone: +43/512/507-7103
%% Fax: +43/512/507-2851

%% for those who use Sweave please include the following line (with % symbols):
%% need no \usepackage{Sweave.sty}

%% end of declarations %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\VignetteIndexEntry{1}
%\VignetteEngine{knitr::knitr}

\begin{document}
<<setup, include=FALSE, cache=FALSE>>=
library(knitr)
# set global chunk options
opts_chunk$set(prompt=TRUE)
options(replace.assign=TRUE, width=77, prompt="R> ")
opts_chunk$set(fig.width=6, fig.height=6, fig.align='center', out.width='.4\\linewidth')#, tidy=T, tidy.opts=list(width.cutoff=60))
knitr::render_sweave() 
@


%% include your article here, just as usual
%% Note that you should use the \pkg{}, \proglang{} and \code{} commands.

%% Note: If there is markup in \(sub)section, then it has to be escape as above.

\section{Introduction}

The purpose of the \proglang{R} \citep{R} package \pkg{BASS} \citep{BASS} is to provide an easy-to-use implementation of Bayesian adaptive spline models for nonparametric regression.  It provides a combination of flexibility, scalability, interpretability and probabilistic accuracy that can be difficult to find in other nonparametric regression software packages.  The model form is flexible enough to capture local features that may be present in the data.  It is scalable to moderately large datasets in both the number of predictors and the number of observations.  It performs automatic variable selection.  It can build nonparametric functional regression models and incorporate categorical predictors.  The package can partition the variability of a resultant model using a nonlinear ANOVA decomposition, providing valuable interpretation to the predictors.  The Bayesian approach allows for model estimates and predictions that can be evaluated probabilistically. The package is  protected under the GNU General Public License, version 3 (GPL-3), and is available from the Comprehensive \proglang{R} Archive Network (CRAN) at \url{https://CRAN.R-project.org/package=BASS}.

The BASS framework builds on multivariate adaptive regression splines (MARS) from \cite{friedman1991multivariate}.  Well-developed software implementations of the MARS model are available in the \proglang{R} packages \pkg{earth}, \pkg{polspline} and \pkg{mda}.  The Bayesian version of MARS (BMARS) was first developed in \cite{denison1998bayesian}.  A \proglang{MATLAB} implementation of BMARS is available from the software website accompanying \cite{denison2002bayesian}.  

Our implementation is more similar to the BMARS implementation, though with some substantial changes to methodology as described in \cite{francom2016sensitivity}.  The primary motivation for developing this software was building surrogate models (or emulators) for complex and computationally expensive simulators (or computer models).  In particular, we wanted to build a fast and accurate surrogate model to use for uncertainty quantification in the presence of a large number of simulations and where each simulation had functional output.  Attributing the variance in the response of the surrogate to different combinations of predictors, a practice known as sensitivity analysis, is a valuable tool for determining which predictors and interactions are important.  One of the major benefits of polynomial spline surrogate models is that sensitivity analysis can be done analytically.  The \pkg{BASS} package has this functionality for scalar and functional response models.

There are a number of other \proglang{R} packages that use splines, such as \pkg{crs}, \pkg{gss}, \pkg{mgcv} and \pkg{R2BayesX}, the latter two of which include possible Bayesian inference methods.  These packages allow (or require) the user to specify which variables are allowed to interact in what way, as well as which variables are allowed to have nonlinear main effects. The \pkg{crs} package is more similar to the packages that fit MARS models in that it can learn the structure of the model from the data.  These packages report a single best model.  \pkg{BASS} reports an ensemble of models (posterior draws from the model space) that can be used to make probabilistic predictions.  In this way, it is more similar to Bayesian nonparametric regression packages like \pkg{BayesTree} and \pkg{tgp}.

We introduce the package as follows.  In Section \ref{sec:BASS}, we describe the modeling framework, including our methods for posterior sampling, modeling functional responses, and incorporating categorical inputs.  In Section \ref{sec:SA}, we describe the sensitivity analysis methods.  Then, in Section \ref{sec:ex}, we walk through six examples of how to use the package.  These are done with \pkg{knitr} in order to be reproducible by the reader.  Finally, in Section \ref{sec:summary}, we present a summary of the package capabilities.

\section{Bayesian adaptive spline surfaces}
\label{sec:BASS}

The BASS model, like the MARS and BMARS models, uses data to learn a set of data dependent basis functions that are tensor products of polynomial splines. The number of basis functions as well as the knots and variables used in each basis are chosen adaptively.  The BMARS approach uses reversible jump Markov chain Monte Carlo (RJMCMC) \citep{green1995reversible} to sample the posterior.  The BASS adaptation of BMARS includes the improvements of \cite{nott2005efficient} for more efficient posterior sampling as well as parallel tempering for better posterior exploration.  BASS also efficiently handles functional responses and allows for categorical variables.  We discuss each of these aspects below.  First, we introduce the BASS model and priors.

Let $y_i$ denote the dependent variable and $\bx_i$ denote a vector of $p$ independent variables, with $i=1,\ldots,n$.  Without loss of generality, let each independent variable be scaled to be between zero and one.  We model $y_i$ as 
\begin{align}
y_i &=f(\bx_i) + \epsilon_i,~~\epsilon_i\sim N(0,\sigma^2)\\
f(\bx) &= a_0+\sum_{m=1}^{M}a_mB_m(\bx)\\
B_m(\bx) &= \prod_{k=1}^{K_m} g_{km} [s_{km}(x_{v_{km}}-t_{km})]_+^\alpha \label{equ:basis}
\end{align}
where $s_{km}\in \{-1,1\}$ is referred to as a sign, $t_{km}\in [0,1]$ is a knot, $v_{km}$ selects a variable, $K_m$ is the degree of interaction and $g_{km}=[(s_{km}+1)/2-s_{km}t_{km}]^\alpha$ is a constant that makes the basis function have a maximum of one.  The function $[\cdot]_+$ is defined as $\text{max}(0,\cdot)$.  The power $\alpha$ determines the degree of the polynomial splines.  We allow for no repeats in $v_{1m},\dots,v_{K_mm}$, meaning that a variable can be used only once in a basis function.  $M$ is the number of basis functions, and $\ba$ is the $M+1$ vector of basis coefficients (including the intercept).  The only difference between this setup and that of MARS and BMARS is the inclusion of the constant $g_{km}$ in each element of the tensor product.  This normalizes the basis functions so that the basis coefficients $a_1,\dots,a_M$ are on the same scale, making computations more stable.

In the course of fitting this model, we seek to estimate $\btheta=\{\sigma^2,M,\ba,\bK,\bs,\bt,\bv\}$ where $\bK$ is the $M$-vector of interaction degrees, $\bs$ is the vector of signs $\{\{s_{km}\}_{k=1}^{K_m}\}_{m=1}^M$, $\bt$ the vector of knots and $\bv$ the vector of variables used (with $\bt$ and $\bv$ defined similar to $\bs$).  Under the Bayesian formulation, we specify a prior distribution for $\btheta$.  

First, consider the priors for the $\sigma^2$ and $\ba$. Let $\bB$ be the $n\times (M+1)$ matrix of basis functions (including the intercept).  Then we use Zellner's $g$-prior \citep{liang2008mixtures} for $\ba$ with
\begin{align}
	\ba|\sigma^2,\tau,\bB &\sim N(\bzero,\sigma^2 (\bB^\top\bB)^{-1}/\tau)\\
	\sigma^2 &\sim InvGamma(g_1,g_2)\\
	\tau &\sim Gamma(a_\tau,b_\tau)
\end{align}
with default settings $a_\tau=1$ and $b_\tau=1/n$ (shape and rate) to center the prior over the unit information prior and $g_1=g_2=0$ resulting in the non-informative prior $p(\sigma^2)\propto 1/\sigma^2$.  In practice, the default settings are sufficient for most cases, though it can be helpful to encode actual prior information into the prior for $\sigma^2$.

Now, consider the prior for the number of basis functions, $M$.  We use a Poisson prior for $M$, truncated to be between $0$ and $M_\text{max}$.  We give a Gamma hyperprior to the mean of the Poisson, $\lambda$.  If $c$ is the Poisson mass that has been truncated, i.e., $c=1-\sum_{m=0}^{M_\text{max}} e^{-\lambda}\lambda^m/m!$, then we have
\begin{align}
	p(M|\lambda) &= \frac{e^{-\lambda}\lambda^M}{cM!} \\
	\lambda &\sim Gamma(h_1,h_2)
\end{align}
where the default settings of $h_1=h_2=10$ (shape and rate) in most cases induce a small number of basis functions.  In practice, these hyperparameters can be key in order to prevent overfitting.  More specifically, we increase $h_2$ (by many orders of magnitude in some cases) to bring the prior for $\lambda$ very close to zero in an effort to thin out the tails of the Poisson and have fewer basis functions.  We use $M_\text{max}$ to give an upper bound to the computational cost, rather than to prevent overfitting.  This strategy results in better fitting models since setting $M_\text{max}$ too small often results in posterior sampling from only one mode.

The priors for $\bK$, $\bs$, $\bt$ and $\bv$ are uniform over a constrained space as described in \cite{francom2016sensitivity}.  The constraint in this prior makes sure basis functions have more than $b$ non-zero values.  Note that a basis function, as shown in Equation \ref{equ:basis}, is likely to have many zeros in it depending on how close the knot is to the edge of the space.  If a knot is too close to the edge of the space, there might only be a few non-zero values in the basis function.  A basis function with only a few non-zero values corresponds to very local fitting and usually results in edge effects. If we calculate the number of non-zero points in basis function $m$ to be $b_m$, this prior requires that $b_m>b$.  This is the BASS equivalent of specifying a minimum number of points in each partition in recursive partitioning.  In addition to specifying $b$, we also specify $K_\text{max}$, the maximum degree of interaction for each basis function.

Table \ref{table:t1} shows the parameters used in the \code{bass} function that we have discussed thus far, and what their mathematical symbols are.

\begin{table}[H]
\begin{center}
\begin{tabular}{|c|cccccccccc|}
	\hline
	Symbol & $K_\text{max}$ & $b$ & $h_1$ & $h_2$ & $g_1$ & $g_2$ & $\alpha$ & $M_\text{max}$ & $a_\tau$ & $b_\tau$\\
	\hline
	\code{bass} input & \code{maxInt} & \code{npart} & \code{h1} & \code{h2} & \code{g1} & \code{g2} & \code{degree} & \code{maxBasis} & \code{a.tau} & \code{b.tau}\\
	\hline
\end{tabular}
\end{center}
\caption{Translation from mathematical symbols to parameters used in \code{bass} function.}
\label{table:t1}
\end{table}

\subsection{Efficient posterior sampling}
Posterior sampling is complicated by the fact that the model is transdimensional (since $M$ is unknown).  Our RJMCMC scheme allows us to add, delete, or change a basis function consistent with the approach of \cite{nott2005efficient}.  That is, instead of proposing to add a completely random new basis function in a reversible jump step, we use a proposal generating distribution that favors the variables and degrees of interaction already included in the model.  For example, say there were five basis functions already in the model, each with degree of interaction two.  Say the maximum degree of interaction was three.  Then if we were proposing a new basis function we would sample the degree of interaction from $\{1,2,3\}$ with weights $\{w_1,w_1+5,w_1\}$, thus favoring two way interactions since we have seen more of them.  If the nominal weight $w_1$ is large compared to the number of basis functions, this distribution looks more uniform.  The value $w_2$ is the equivalent nominal weight for sampling variables to be included in a candidate basis function.  Both $w_1$ and $w_2$ default to five. If there are a large number of unimportant variables in the data, a small value of $w_2$ (relative to $M$) helps to make posterior sampling more efficient by not proposing basis functions that include the unimportant variables. 

We extend the framework of \cite{nott2005efficient} to allow for more than two-way interactions.  This ends up being non-trivial, since the RJMCMC acceptance ratio requires us to calculate the probability of sampling the proposed basis function.  The difficulty comes when we try to calculate the probability of sampling the particular variables, as this requires calculating the probability of a weighted sample without replacement (weighted since we do not sample variables uniformly, without replacement since variables cannot be used more than once in the same basis function).  This is equivalent to sampling from the multivariate Wallenius' noncentral hypergeometric distribution.  To determine the probability of such a sample, we use a function from the \proglang{R} package \pkg{BiasedUrn} \citep{BiasedUrn}.  Since the CRAN version of \pkg{BiasedUrn} allows for only 32 possible variables, we include a slightly altered version of the function in \pkg{BASS} to quickly evaluate the approximate density function of the multivariate Wallenius' noncentral hypergeometric distribution.

The computation behind posterior sampling becomes much more efficient when we recognize that each RJMCMC iteration only allows slight changes to our set of basis functions.  Thus, quantities like $\bB^\top\bB$, $\bB\ba$ and $\bB^\top\by$ can easily be updated rather than recalculated, as shown in \cite{francom2016sensitivity}. 

We perform $N_\text{MCMC}$ RJMCMC iterations and discard the first $N_\text{burn}$, after which every $N_\text{thin}$ iterations is kept.  This results in $(N_\text{MCMC}-N_\text{burn})/N_\text{thin}$ posterior samples.
Table \ref{table:t2} shows the parameters to the \code{bass} function discussed in this section, as well as their mathematical symbols.

\begin{table}[H]
	\begin{center}
		\begin{tabular}{|c|ccccc|}
			\hline
			Symbol & $w_1$ & $w_2$ & $N_\text{MCMC}$ & $N_\text{burn}$ & $N_\text{thin}$ \\
			\hline
			\code{bass} input & \code{w1} & \code{w2} & \code{nmcmc} & \code{nburn} & \code{thin} \\
			\hline
		\end{tabular}
	\end{center}
	\caption{Translation from mathematical symbols to parameters used to specify nominal weights of proposal distributions and number of RJMCMC iterations in the \code{bass} function.}
	\label{table:t2}
\end{table}

\subsection{Parallel tempering}
Posterior sampling with RJMCMC is prone to mixing problems (problems exploring all of the parameter space).  In our case, this is because only slight changes to the basis functions can be made in each iteration.  Thus, once we start sampling from one mode of the posterior, it can be hard to move to another mode if it requires changing many of the basis functions \citep{gramacy2010importance}.  

We are able to achieve better mixing by using parallel tempering.  This requires the specification of a temperature ladder, $1=t_1<t_2<\dots<t_T<\infty$.  For each temperature in the temperature ladder, a RJMCMC chain samples the posterior raised to the inverse temperature (i.e., if $\pi(\btheta|\by)$ is the posterior of interest, we sample from $\pi(\btheta|\by)^{1/t_i}$).  The chains at neighboring temperatures are allowed to swap states according to a Metropolis-Hastings acceptance ratio (see \cite{francom2016sensitivity} and references therein).  Only samples in the lowest temperature chain ($t_1$) are used for inference.  The high temperature chains mix over many posterior modes, allowing diverse models to be propagated to the low temperature chain.  We allow the chains to run without swapping for $N_{st}$ iterations at the beginning of the run to allow them to get close to their stationary distributions.

Specifying a temperature ladder can be difficult.  Temperatures need to be close enough to each other to allow for frequent swaps (with acceptance rates between 20 and 60\% \citep{altekar2004parallel}), and the highest temperature ($t_T$) needs to be high enough to be able to explore all the modes.  Future versions of this package may make some attempt at automatically specifying and altering a temperature ladder.  Further, a message passing interface (MPI) approach to handling the multiple chains could result in substantial speedup, and may be implemented in future versions of the package.

Table \ref{table:t3} shows the translation from parameters used for parallel tempering in the \code{bass} function to symbols we have used in this section.

\begin{table}[H]
	\begin{center}
		\begin{tabular}{|c|cc|}
			\hline
			Symbol & $(t_1,\dots,t_T)$ & $N_{st}$ \\
			\hline
			\code{bass} input & \code{temp.ladder} & \code{start.temper} \\
			\hline
		\end{tabular}
	\end{center}
	\caption{Translation from mathematical symbols to parameters used for parallel tempering in the \code{bass} function.}
	\label{table:t3}
\end{table}

\subsection{Functional response}
We handle Functional responses as though the variable indexing the functional response, like time or location, is one of the independent variables.  When the functional response is output onto the same functional variable grid for all samples, this results in more efficient calculations involving basis functions because of the Khatri-Rao product structure \citep{francom2016sensitivity}.  For example, this software is well suited to fit a model where the data are such that a combination of independent variables results in a time-series and the grid of times (say, $r_1,\dots,r_q$) is the same for each combination.

If there are multiple functional variables, we must specify a maximum degree of interaction for them, $K_{\text{max}}^F$.  For instance, if the functional output was a spatiotemporal field (a function of three variables) and we specify a maximum degree of functional interaction of two, we would not allow for interactions between both spatial dimensions and time.  We would specify the grid of spatial locations and time points as a matrix with three columns rather than a vector like we did in the time series example above.  We can also specify a value $b_F$, possibly different from $b$, that indicates the number of non-zero values required in the functional part of basis functions.  When functional responses are included, the values of $b$ and $b_F$ should be relative to the sample size and the size of the functional grid, respectively.

Table \ref{table:t4} shows parameters necessary to model functional responses in the \code{bass} function.  The response \code{y} should be specified as a matrix when the response is functional.

\begin{table}[H]
	\begin{center}
		\begin{tabular}{|c|ccc|}
			\hline
			Symbol & $(r_1,\dots,r_q)$ & $K_{\text{max}}^F$ & $b_F$  \\
			\hline
			\code{bass} input & \code{xx.func} & \code{maxInt.func} & \code{npart.func} \\
			\hline
		\end{tabular}
	\end{center}
	\caption{Translation from mathematical symbols to parameters used in the \code{bass} function when modeling functional data.}
	\label{table:t4}
\end{table}

\subsection{Categorical inputs}
We include categorical variables by allowing for basis functions to include indicators for categorical variables being in certain categories.  Our approach is the Bayesian version of \cite{friedman1991estimating} and is described in \cite{francom2017inferring}.  If a set of independent variables is separated into continuous variables $\bx$ and categorical variables $\bc$, then the $m^\text{th}$ basis function equivalent of Equation \ref{equ:basis} can be written as
\begin{align}
	B_m(\bx,\bc)&= \prod_{k=1}^{K_m} g_{km} [s_{km}(x_{v_{km}}-t_{km})]_+^\alpha \prod_{l=1}^{K^c_m} 1\left(c_{v_{lm}^c} \in C_{lm}\right)
\end{align}
where $K^c_m$ is the degree of interaction for the categorical predictors, $1(\cdot)$ is the indicator function, $v^c_{lm}$ indexes the categorical variables and $C_{lm}$ is a subset of the categories for variables $c_{v_{lm}^c}$.  We now allow for $K_m$ or $K_m^c$ to be zero, and specify a $K_\text{max}^c$ (\code{maxInt.cat} in the \code{bass} function).  

The priors we use for the degree of interaction, variables used and categories used are, in combination with the priors we used above, the same constrained uniform.  Thus, basis function $(B_m(\bx_1,\bc_1),\dots,B_m(\bx_n,\bc_n))$ is required to have at least $b$ non-zero values.

\section{Sensitivity analysis}
\label{sec:SA}

Global sensitivity analysis for nonlinear models using the Sobol' decomposition \citep{sobol2001global} is well developed, but often requires large numbers of evaluations of the models for Monte Carlo approximation of integrals \citep{saltelli2008global}.  The benefit of polynomial spline models is that Monte Carlo approximation is unnecessary because the integrals can be calculated analytically.  

The method decomposes a function $f(\bx)$ into main effects, two way interactions, and so on, up to $p$ way interactions so that
\begin{align}
	f(\bx) &=f_0 + \sum_{i=1}^pf_i(x_i) + \sum_{i=1}^{p}\sum_{j>i}f_{ij}(x_i,x_j) + \dots + f_{1\cdots p}(x_1,\dots,x_p) \label{equ:sob1}.
\end{align}
Each term in the sum above is constructed so that it is orthogonal to all the other terms.  This can be done by calculating 
\begin{align}
	f_0 &= \int f(\bx)d\bx\\
	f_i(x_i) &= \int f(\bx)d\bx_{-i}-f_0\\
	f_{ij}(x_i,x_j) &= \int f(\bx)d\bx_{-ij} - f_i(x_i) - f_j(x_j) - f_0
\end{align}
etc., for all the terms in Equation \ref{equ:sob1}.  Note that if we assume $\bx$ is uniformly distributed, these are conditional expectations (except $f_0$ is unconditional). The conditional expectations are centered at zero the way we have constructed them.  Since the terms in Equation \ref{equ:sob1} are orthogonal,
\begin{align}
	E(f^2(\bx)) &= f_0^2 + \sum_{i=1}^pE\left(f_i^2(x_i)\right) + \sum_{i=1}^{p}\sum_{j>i}E\left(f^2_{ij}(x_i,x_j)\right) + \dots + E\left(f^2_{1\cdots p}(x_1,\dots,x_p)\right).
\end{align}
Using the fact that $Var(f(\bx))=E(f^2(\bx))-f_0^2$ and that $E(f_{i_1\cdots i_s}(x_{i_1\cdots i_s})) = 0$ for all terms except $f_0$,
\begin{align}
	Var(f(\bx)) &= \sum_{i=1}^pVar(f_i(x_i)) + \sum_{i=1}^{p}\sum_{j>i}Var(f_{ij}(x_i,x_j)) + \dots + Var(f_{1\cdots p}(x_1,\dots,x_p)).
\end{align}
This is a decomposition of the variance of the model into variance due to each main effect, each two way interaction (after accounting for the associated main effects), etc.  All of these integrals are analytical in our case, with solutions given in \cite{francom2016sensitivity}.  Sensitivity indices for main effects and interactions are then defined as proportions of the total variance.  Total sensitivity for a particular variable can then be gauged by adding the main effect and all interactions associated with that variable and comparing to the total sensitivity indices for other variables.

We can obtain this variance decomposition for each posterior sample to get posterior distributions of sensitivity indices.  This can be time consuming, so the \code{sobol} function has an argument \code{mcmc.use} to specify which RJMCMC iterations should be used.  Calculations of the integrals above can be vectorized when basis functions are the same and only basis function coefficients change.  This is the case for many of the RJMCMC iterations, and the \code{sobol} function automatically determines this and accounts for it. (As a side note, this is also the case for the \code{predict} function).

\subsection{Functional response}
There are a few ways to think about sensitivity analysis for models with functional response.  One way is to get the sensitivity indices for the functional variables in the same way we get the sensitivity indices for the rest of the variables.  This results in a total variance decomposition.  Another approach is to obtain functional sensitivity indices, which would tell us how important a variable or interaction is as we change the functional variable.  This can be done by following the procedure just mentioned, but simply not integrating over the functional variable.  Hence, all of the expectations above would be conditional on the functional variable.  These approaches are explored in \cite{francom2016sensitivity}.  

By default, the \code{sobol} function gets sensitivity indices for the functional variables the same way it does for the other variables.  Setting \code{func.var = 1} gets the sensitivity indices as functions of the first (possibly only) functional variable (if there are multiple functional variables, this refers to the first column of the matrix \code{xx.func} passed to the \code{bass} function).

\subsection{Categorical inputs}
Under our categorical input extension, the necessary expectations to obtain the Sobol' decomposition are still analytical, as described in \cite{francom2017inferring}.  For the categorical variables, we replace the integrals with sums over categories.

\section{Examples}
\label{sec:ex}
We now demonstrate the capabilities of the package on a few examples.  For each example, we start by setting the seed (\code{set.seed(0)}) so that readers can replicate the results. First we load the package
<<>>=
library("BASS")
@
which we use for all the examples.

\subsection{Curve fitting}
<<child='child/ex1-2.Rnw'>>=
@

\subsection{Friedman function}
<<child='child/ex2-2.Rnw'>>=
@

\subsection{Friedman function with a categorical variable}
<<child='child/ex3-2.Rnw'>>=
@

\subsection{Friedman function with functional response}
<<child='child/ex4-2.Rnw'>>=
@

\subsection{Air foil data}
\label{sec:ex5}
<<child='child/ex5-2.Rnw'>>=
@

\subsection{Pollutant spill model}
<<child='child/ex6-2.Rnw'>>=
@

\section{Summary}
\label{sec:summary}
Our proposed BASS framework provides a powerful general tool for nonparametric regression settings.  It can be used for modeling with many continuous and categorical inputs, large sample size and functional response.  It provides posterior sensitivity analyses without integration error.  The MCMC approach to inference, especially using parallel tempering, yields posterior samples that can be used for probabilistic prediction. The \pkg{BASS} package makes these features accessible to users with minimal exposure.  These capabilities have been demostrated with a set of examples involving different dimensions, categorical variables, functional responses, and large datasets.  



%\section{Acknowledgments}
%This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. LLNL-JRNL-xxxxxx.

\bibliography{bibl}


\end{document}
