---
title: "Hmisc Examples"
author: "FE Harrell"
date: '`r Sys.Date()`'
output:
  html_notebook:
    toc: yes
    toc_depth: 3
    toc_float:
      collapsed: yes
  html_document:
    toc: yes
    toc_depth: 3
    toc_float:
      collapsed: yes
csl: american-medical-association.csl
bibliography: ~/bib/harrelfe.bib
---
# Introduction
This is a set of reproducible examples for the R[@R] `Hmisc` package[@Hmisc], put together in an `rmarkdown` notebook using `RStudio` and `knitr`.  When viewing the resulting [html file](http://data.vanderbilt.edu/fh/R/Hmisc/examples.nb.html) you can see all the code, and can download the entire `rmarkdown` script, which is especially helpful for seeing how `knitr` chunks are specified.  Graphics that have a [plotly](http://plot.ly/r) method for them are rendered using `plotly` instead of using defaults such as base graphics, `lattice`, or `ggplot2`.  That way the plots are somewhat interactive, e.g., allow for drill-down for more information without the viewer needing to install `R`.

Much of the tabular output produced here was created using `html` methods, which are especially easy to implement with `rmarkdown` and make for output that can be directly opened using word processors.  Jump to [Computing Environment](#compenv) for a list of packages used here, and their version numbers.

# Setup 
```{r setup,results='hide'}
require(Hmisc)
# Use the following require() if rendering with html output instead of notebook
# require(plotly)
# Use the folowing with html output instead of notebook
# knitrSet(lang='markdown', h=4.5, cache=TRUE)
knitrSet(lang='markdown', h=4.5)
# knitrSet redirects all messages to messages.txt
options(grType='plotly') # for certain graphics functions
mu <- markupSpecs$html   # markupSpecs is in Hmisc
cap  <- mu$cap           # function to output html caption
lcap <- mu$lcap          # for continuation for long caption
# These last 2 functions are used by the putHfig function in Hmisc
```
The following (<code>r mu$styles()</code>) defines HTML styles and javascript functions, such as <code>ffig, smg</code> and the function for expanding and collapsing text as done by the <code>expcoll</code> function. 
`r mu$styles()`

Example using <code>expcoll</code>.  Click on the down arrow to expand; it turns to an up arrow which can be clicked to contract.
`r mu$expcoll('Example table', html(data.frame(x1=runif(10), x2=letters[1:10]),file=FALSE))`

The following (<code>r mu$widescreen()</code>) causes the html notebook to use an entire wide screen.
`r mu$widescreen()`

# Fetching Data, Modifying Variables, and Printing Data Dictionary
The `getHdata` function is used to fetch a dataset from the Vanderbilt `DataSets` web site.  `upData` is used to

- create a new variable from an old one
- add labels to 2 variables
- add units to the new variable
- remove the old variable
- automatically move units of measurements from parenthetical expressions in labels to separate `units` attributed used by `Hmisc` and `rms` functions for table making and graphics

`contents` is used to print a data dictionary, run through an `html` method for nicer output.  Information about the data source may be found [here](http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/pbc.html).
```{r metadata,results='asis'}
getHdata(pbc)

# Have upData move units from labels to separate attribute
pbc <- upData(pbc,
              fu.yrs = fu.days / 365.25,
              labels = c(fu.yrs = 'Follow-up Time',
                         status = 'Death or Liver Transplantation'),
              units = c(fu.yrs = 'year'),
              drop  = 'fu.days',
              moveUnits=TRUE, html=TRUE)
# The following can also be done by running this command
# to put the results in a new browser tab:
# getHdata(pbc, 'contents')
html(contents(pbc), maxlevels=10, levelType='table')
```

# Descriptive Statistics Without Stratification
The html method is used for the `describe` function, and the output is put in a scrollable box.  Other than for the overall title and variable names and labels, the output size used here is 80 (0.8 &times; the usual font size[^1]).  But the graphical display of the descriptives statistics that follows this is probably better.

[^1]: The default is 75% size.

```{r describe,results='asis'}
d <- describe(pbc)
html(d, size=80, scroll=TRUE)

# prList is in Hmisc; useful for plotting or printing a list of objects
# Can just use plot(d) if don't care about the mess
# If using html output these 2 images would not be rendered no matter what
p <- plot(d)
# The option htmlfig=2 causes markupSpecs$html$cap() to be used to
# HTML-typeset as a figure caption and to put the sub-sub section
# marker ### in front of the caption.  htmlfig is the only reason
# results='asis' was needed in the chunk header
# We define a long caption for one of the plots, which does not appear
# in the table of contents
prList(p, lcap=c('', 'These are spike histograms'), htmlfig=2)
```
You can also re-form multiple `plotly` graphs into a [single HTML object](http://stackoverflow.com/questions/35193612).  Here we use the `putHfig` function to render the result, with a caption and a short caption for the table of contents.  This approach requires you to put <code>results='asis'</code> in the chunk header.  By default <code>putHfig</code> puts a horizontal line before the caption and figure.

```{r plotlym,results='asis'}
x <- htmltools::tagList(p)  #lapply(p, plotly::as.widget))
putHfig(x, 'This used the <code>htmltools tagList</code> function.',
        scap='Two <code>plotly</code> graphics combined into one')
```

You can also create figure captions outside of R code by using the <code>smg, fcap</code> HTML tags defined in <code>markupSpecs</code>.  The long caption not appearing in the table of contents will be in a separate line without ###.

# Stratified Descriptive Statistics
Produce stratified quantiles, means/SD, and proportions by treatment group.  Plot the results before rendering as an advanced html table:

- categorical variables: a single dot chart
- continuous variables: a series of extended box plots

```{r summaryM,results='asis'}
s <- summaryM(bili + albumin + stage + protime + sex + age + spiders +
              alk.phos + sgot + chol ~ drug, data=pbc,
							overall=FALSE, test=TRUE)
# If not using captions, just use plot(s, which='categorical')
putHfig(plot(s, which='categorical'),
        'Proportions and', mu$chisq(), 'tests for categorical variables')
```
To construct the caption outside of the code chunk use e.g. ### r cap('Proportions and', mu$chisq(), 'tests for categorical variables') where a backtick is placed before r and after the last ).

```{r summaryM2,results='asis'}
putHfig(plot(s, which='continuous', vars=1 : 4),
        'Extended box plots for the first 4 continuous variables')
```

```{r summaryM3,results='asis'}
putHfig(plot(s, which='continuous', vars=5 : 7),
        'Extended box plots for the remaining continuous variables')
```

```{r summaryM4}
html(s, caption='Baseline characteristics by randomized treatment',
     exclude1=TRUE, npct='both', digits=3,
     prmsd=TRUE, brmsd=TRUE, msdsize=mu$smaller2)
```
The following is a better way to display proportions, for categorical variables.  If computing marginal statistics by running the dataset through the `Hmisc` `addMarginal` function, the `plot` method with `options(grType='plotly')` is especially useful.

```{r summaryM5,results='asis'}
pbcm <- addMarginal(pbc, drug)
s <- summaryP(stage + sex + spiders ~ drug, data=pbc)
putHfig(plot(s, groups='drug'),
        'Proportions stratified by treatment')
s <- summaryP(stage + sex + spiders ~ drug, data=pbcm)
putHfig(plot(s, marginVal='All', marginLabel='All Treatment Groups'),
        'Proportions (large symbols) and proportions stratified by treatment (small symbols)',
				scap='Proportions with and without stratification by treatment')
```

# Computing Environment[^2] {#compenv}
`r mu$session()`

# Bibliographic File Managament
## Find and Install `.csl` Reference Style Files
```{r findbib,eval=FALSE}
# Note: mu was defined in an earlier code chunk
# Only need to install .csl file once.
mu$installcsl(rec=TRUE)   # get list of recommended styles
mu$installcsl()     # web search of styles meeting your criteria
# Install a .csl file to your project directory:
mu$installcsl('american-medical-association')
```
## Manage `citeulike` Bibliographic Databases
In the following code chunk, adding `cache=TRUE` to the chunk header would result in the code (including the file download) not having to be run each time the report is run.  `eval=FALSE` is set in the code chunk so the code actually isn't run.
```{r citeulike,eval=FALSE}
# Show full reference information for selected BibTeX keys on
# your citeulike database
cu <- mu$citeulikeShow
cu('harrelfe', c('cox58reg', 'cox72'))
# Automatically extract all BibTeX keys referenced in this document and
# show full reference information for them
# The function invisibly returns the vector of keys found
cu('harrelfe', file='examples.Rmd')
# Show and optionally export all articles with a given tag
cu('harrelfe', tags='missing-data')
# Copy and rename downloaded references to current project directory
file.copy('~/Downloads/harrelfe-missing-data.bib', 'missing-data.bib',
          overwrite=TRUE)
```

# References

[^2]: `mu` is a copy of the part of the `Hmisc` package object `markupSpecs` that is for html.  It includes a function `session` that renders the session environment (including package versions) in html.

<!---
To update html notebook on the server: cdatar examples.nb.html Hmisc
-->
