---
title: An Introduction to PCA

# Force pkgdown to use the pdf vignette and not override format/output
#pkgdown:
#  as_is: true
#  extension: pdf

# Produce a pinp document
output: pinp::pinp

# Use letters for affiliations
author:
  - name: Bryan A. Hanson
    affiliation: a

address:
  - code: a
    address: Prof. Emeritus, Dept. of Chemistry \& Biochemistry, DePauw University; \url{hanson@depauw.edu}

# Font size of the document, values of 9pt (default), 10pt, 11pt and 12pt
fontsize: 10pt

# Optional: Force one-column layout, default is two-column
one_column: false

# Optional: Enable one-sided layout, default is two-sided
one_sided: false

# Optional: Enable section numbering, default is unnumbered
numbersections: false

# Optional: Specify the depth of section number, default is 5
secnumdepth: 5

# Optional: Bibliography
#bibliography: ChemoSpec2D

# Optional: Enable a 'Draft' watermark on the document
watermark: false

# Customize footer, eg by referencing the vignette
footer_contents: "Intro to PCA"

# May improve floats at end of document if true
skip_final_break: true

# Required: Vignette metadata for inclusion in a package.
vignette: >
  %\VignetteIndexEntry{ChemoSpec: Introduction to PCA}
  %\VignetteKeywords{PCA}
  %\VignettePackage{ChemoSpec}
  %\VignetteEngine{knitr::rmarkdown}

header-includes:
  - \usepackage[version = 4]{mhchem}
---

```{r SetUp, echo = FALSE, eval = TRUE, results = "hide"}
# R options & configuration:
set.seed(9)
rm(list = ls())
suppressPackageStartupMessages(library("knitr"))
suppressPackageStartupMessages(library("kableExtra"))
suppressPackageStartupMessages(library("chemometrics"))
options(width =  50) # for pinp specifically (narrow cols)

# Stuff specifically for knitr:
opts_chunk$set(eval = TRUE, echo = FALSE, results = "hide",
  fig.width = 3.5, fig.height = 3.5)
```

The audience for this document is chemists, spectroscopists and people in allied fields who are using spectroscopy to analyze their data.

We will use two data sets here: one is a set of elemental analyses of glass artifacts; we use this relatively small, non-spectroscopic data set to help understand PCA fundamentals.  The second data set is a collection of IR spectra of plant oils.

# Conceptual Introduction to PCA

PCA is conducted on data sets composed of:

* Samples, typically in rows.
* Variables which were measured for each sample.

The purpose of PCA is *data reduction*.  This term refers to the goal of:

* Reducing the size of the data set by identifying variables that are not informative.  Such variables are also described as "noisy", in that they don't add anything to the study.  Such variables arise naturally in many situations.  Example: a survey about food preferences may include questions about political party.  The answers about political party may not be informative.
* Collapsing correlating variables.  Several of the variables measured in a study may actually be measures of the same underlying reality.  This is not to say they are noisy, but rather they may be redundant.  Example: a survey asks participants if they eat kale, and separately, if they eat quinoa.  Individuals may answer yes to both questions or no to both questions.  The answers may reflect the individuals preference for a healthy diet.  Either question alone may be sufficient.  PCA will collapse these correlating variables into one variable.

What does one get from PCA?

* An indication of how many principal components (PC) are needed to describe the data, generally presented as a *scree plot*.
* Scores, generally presented as one or more *score plots*.
* Loadings, generally presented as one or more *loading plots*.

These plots will be explained further in the next section.  Other things to know about PCA before going further:

* PCA is *principal* not *principle* components analysis!
* PCA is the "mother" of a number of other related techniques, so if you plan further study it is critical to understand PCA to the greatest degree possible.
* That said, it takes most people a long time to fully grasp what PCA does, especially from the mathematical perspective.  Don't expect to get all the nuances on the first pass!
* *And the problem* \ldots The results of PCA, scores and loadings, exist in a so-called abstract space.  This space is only distantly and indirectly related to the space in which the original samples reside.  Therefore, the results of PCA are frequently difficult to interepret in concrete terms.  See previous point.

# PCA Results Illustrated, No Code, No Math

This section is designed to illustrate the concepts of PCA, and how to interpret the plots that arise from PCA.

We'll use a data set which reports chemical analyses on archaeological glass artifacts that was designed to determine the origin of the artifacts.  Table \ref{tab:dataTaste} gives a little bit of the data set.\footnote{This is the \code{glass} data set in package \code{chemometrics}.}

```{r, dataTaste, results = "asis"}
data(glass)
DF <- as.data.frame(glass[1:5, 1:8])
kable(DF, format = "latex", caption = "A portion of the archaeological glass data set. Values are percentages.\\label{tab:dataTaste}") %>% kable_styling(c("striped", "bordered"))
```

There are 180 glass artifacts (the samples) in this data set (hence 180 rows), and the elements analyzed were \ce{Na2O}, \ce{MgO}, \ce{Al2O3}, \ce{SiO2}, \ce{P2O5}, \ce{SO3}, \ce{Cl}, \ce{K2O}, \ce{CaO}, \ce{MnO}, \ce{Fe2O3}, \ce{BaO}, and \ce{PbO} .\footnote{The elements are reported as their oxides in the form of weight percent.}

We'll perform PCA on the glass data set, show the three plots and then discuss them in turn.  Figure \ref{fig:glassScree} shows the scree plot, Figure \ref{fig:glassScores} shows the score plot and Figure \ref{fig:glassLoadings} shows the first loadings.


```{r glassScree, fig.cap = "Scree plot from PCA on the glass data set.\\label{fig:glassScree}"}
pca <- prcomp(glass)
plot(pca, main = "")
```

Figure \ref{fig:glassScree}, the scree plot, shows the amount of variance in the data set explained by each principal component (PCs are along the x axis, from 1 to 10).\footnote{Because there are 13 variables, the most PCs one could have is 13.  In theory, keeping all 13 PCs perfectly reproduces the original data set.}  For now, think of variance as the variation or variability in the data set, or better, think of it as the hidden signal in the data.  To interpret this plot, we look for the point at which the height of the bars suddenly levels off.  In this case, the first three PCs drop steadily downward, but from PC four and onward there is little additional variation (signal) that can be explained.  We would say that three PCs are enough to explain this data set.  In other words, the original 13 variables have been reduced to three, which is a great simplification.

```{r glassScores, fig.cap = "Score plot from PCA on the glass data set.\\label{fig:glassScores}"}
plot(pca$x[,1], pca$x[,2], type = "p",
  xlab = "Principal Component 1", ylab = "Principal Component 2")
```

In Figure \ref{fig:glassScores} one sees the scores for PC 1 plotted against the scores for PC 2.  There are 180 points in this plot because there is one point per sample (put another way, every sample has a score value for PC 1 and for PC 2).  This plot is interpreted by looking for clustering of samples, as well as for samples that are outliers, off by themselves.  Depending upon your eye, there are 3 to 5 clusters here.  Later we'll discuss how we can explore this further.

We could also plot PC 1 against PC 3, or PC 2 against PC 3.  These might show different clustering and separation of samples, but are not shown here.  There wouldn't be much point in plotting PC 4 or higher, as PCs 4 and higher are mostly noise, not signal, as established by the scree plot (Figure \ref{fig:glassScree}).

```{r glassLoadings, fig.cap = "Loadings plot for the first PC from PCA on the glass data set.\\label{fig:glassLoadings}"}
barplot(pca$rotation[,1], cex.names = 0.7, ylab = "Contribution")
```

The loadings plot, Figure \ref{fig:glassLoadings}, shows how much each measured variable contributes to the component and hence the separation of samples (in this case the loadings for PC 1).  We see that three elements have large loadings, and the other elements contribute little to the separation.  We would say separation along PC 1 is driven largely and *collectively* by the results for \ce{Na2O}, \ce{SiO2} and \ce{CaO}.\footnote{If you knew this would be the result ahead of time, you probably would not have taken the time and expense to analyze the uninformative elements.  However, we haven't looked at PC 2 or PC 3 so this conclusion is premature.}  The first PC should be interpreted as a composite of these variables -- these variables have been collapsed into one new variable, PC 1.

## Refinements 1

Rather than relying on a scree plot to determine the number of PCs that are important, we can present the same information in a table, see Table \ref{tab:screeTable}.  A general rule of thumb says to keep enough PCs to account for 95% of the variation (signal). The table tells us the same as the plot: keep three PCs.

```{r screeTable, results = "asis"}
eigensum <- sum(pca$sdev*pca$sdev)
variance <- 100*(pca$sdev*pca$sdev/eigensum)
cumvariance <- cumsum(variance)
labs <- paste("PC", 1:13, sep = " ")
DF <- data.frame(component = labs, variance = variance, cumulative = cumvariance)
kable(DF, format = "latex", digits = 0, caption = "Variance (signal) accounted for by PCs. Values in percent.\\label{tab:screeTable}")
```

## Refinements 2

The mathematics of PCA do not take into account anything about the samples other than the measured variables.  However, the researcher may well know something about the samples, for instance, they may fall into groups based on their origin.  If this is the case, the points on the score plot can be colored according to the group.  This may aid significantly in the interpretation.  Lucky for us, we can do this for the glass data set. The samples fell into four groups.  We'll re-do the score plot with colors corresponding to the known groups (Figure \ref{fig:glassScores2}).

```{r groups, fig.cap = "Score plot from PCA on the glass data set, with groups color-coded.\\label{fig:glassScores2}"}
gr <- structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("Group.1", "Group.2",
"Group.3", "Group.4"), class = "factor")
plot(pca$x[,1], pca$x[,2], type = "p", col = gr, pch = 20,
  xlab = "Principal Component 1", ylab = "Principal Component 2")
```

With this figure, we can see that the large group in the lower left corner (in black), which to the eye might have been two groups, is composed of related samples.

# A Spectroscopic Data Set

The archaeological data set has the advantage of only having a few variables, the percentages of each element in the glass artifacts.  If we move to a spectroscopic data set, the number of variables goes up dramatically.  A UV-Vis data set typically would have a few hundred to a thousand wavelength variables, an IR data set perhaps a few thousand data points, and a 1D NMR data set would typically have 16K or more data points.  As far as PCA is concerned, in these cases the scree plot and score plot do not change in appearance or interpretation.

However, the loading plot changes appearance dramatically.  This is because with hundreds to thousands of variables, one would not create a loading plot based on a bar chart (Figure \ref{fig:glassLoadings} is a bar chart).  Instead, the loading plot with many variables looks like a spectrum!  While the appearance is different, the interpretation is the same as for when there are only a few variables.

Let's illustrate with an IR data set.  We'll use a data set included with the \code{ChemoSpec} package.  This is a set of IR spectra of plant oils which are mixtures of triglycerides (triacylglyerols, which are esters), and free fatty acids.  Figure \ref{fig:IRSpectrum} shows a typical spectrum from the data set.\footnote{Plots in this vignette are deliberately made rather plain to focus on the data and to be consistent for ease-of-comparison.  Package \code{ChemoSpec} makes much more polished plots.}

```{r loadIR}
suppressPackageStartupMessages(library("ChemoSpec"))
data(SrE.IR)
```


```{r IRSpectrum, fig.cap = "Spectrum 1 from the IR data set.\\label{fig:IRSpecrum}"}
xl <- rev(range(SrE.IR$freq))
plot(SrE.IR$freq, SrE.IR$data[1,], type = "l", xlim = xl)
```
Next, we'll carry out PCA as before, and show the scree plot (Figure \ref{fig:IRScree}) and the score plot (Figure \ref{fig:IRScores}).  Note that these appear like the corresponding plots for the glass data set, and are interpreted in the same manner. However, the loadings plot, Figure \ref{fig:IRLoadings}, looks a lot like a spectrum, because it has `r length(SrE.IR$freq)` data points, and is plotted as a connected scatter plot and not as a bar chart (which would be difficult to read).

```{r IRScree, fig.cap = "Scree plot from PCA on the IR data set.\\label{fig:IRScree}"}
pca <- prcomp(SrE.IR$data)
plot(pca, main = "")
```

```{r IRScores, fig.cap = "Score plot from PCA on the IR data set.\\label{fig:IRScores}"}
plot(pca$x[,1], pca$x[,2], type = "p",
  xlab = "Principal Component 1", ylab = "Principal Component 2")
```

```{r IRLoadings, fig.cap = "Loadings plot for the first PC from PCA on the IR data set.\\label{fig:IRLoadings}"}
plot(SrE.IR$freq, pca$rotation[,1], type = "l", xlim = xl,
     xlab = "Wavelength", ylab = "Contribution")
```

Let's look at the carbonyl region of the loadings plot in detail.  Figure \ref{fig:IRLoadings2} shows the original spectrum in red, for reference, and the loadings in black.  One can see that the ester carbonyl contributes positively to the first loading, while the carboxylic acid carbonyl contributes negatively.

Finally, to make the point that the loading plot for many variables is essentially the same as the loading plot for just a few variables, Figure \ref{fig:IRLoadings3} shows the carbonyl region as a kind of bar plot.  If one connects the tips of the bars together, one gets the previous plot.

```{r IRLoadings2, fig.cap = "Loadings plot for the first PC from PCA on the IR data set, carbonyl region. Reference spectrum shown in red.\\label{fig:IRLoadings2}"}
plot(SrE.IR$freq, pca$rotation[,1], type = "l", xlim = c(1800, 1650),
     xlab = "Wavelength", ylab = "Contribution", ylim = c(-0.3, 0.3))
lines(SrE.IR$freq, SrE.IR$data[1,], col = "red")
abline(v = c(1743, 1708), lty = 2, col = "gray50")
```

```{r IRLoadings3, fig.cap = "Loadings plot for the first PC from PCA on the IR data set, carbonyl region, shown as a bar plot.\\label{fig:IRLoadings3}"}
plot(SrE.IR$freq, pca$rotation[,1], type = "h", xlim = c(1800, 1650),
     xlab = "Wavelength", ylab = "Contribution")
```
