---
title: "User Interaction in WebGL"
author: "Duncan Murdoch"
date: "`r format(Sys.time(), '%B %d, %Y')`"
output:
  rmarkdown::html_vignette:
    toc: yes
    fig_width: 5
    fig_height: 5
vignette: >
  %\VignetteIndexEntry{User Interaction in WebGL} 
  %\VignetteEngine{knitr::rmarkdown}
---


```{r setup, echo=FALSE, results="asis"}
source("setup.R")
set.seed(123)
```

## Introduction

This document describes how to use embedded Javascript to 
control a WebGL display in an HTML document.  For more 
general information, see [rgl Overview](rgl.html).

We start with two simple examples.  The next section gives 
reference information.

Consider the simple plot of the iris data.  We
insert a code chunk with label `plot3d` (which will be used below).
```{r plot3d, webgl=TRUE}
with(iris, plot3d(Sepal.Length, Sepal.Width, Petal.Length, 
                  type="s", col=as.numeric(Species)))
subid <- currentSubscene3d()
```

We might like a button on the web page to cause a change to the
display, e.g. a rotation of the plot.  First we add buttons, with
the "onclick" event set to a function described below:

    <button type="button" onclick="rotate(10)">Forward</button>
    <button type="button" onclick="rotate(-10)">Backward</button>

which produces these buttons: 
<button type="button" onclick="rotate(10)">Forward</button>
<button type="button" onclick="rotate(-10)">Backward</button>

We stored the subscene number that is currently active in
`subid` in the code chunk above, and use it as `r rinline("subid")`
in the script below.

The `rotate()` function makes use of the global `<prefix>rgl`
object.  The `knitr` WebGL support sets the prefix to the 
code chunk label, so the global is called `plot3drgl`:

    <script type="text/javascript">
    var rotate = function(angle) {
      plot3drgl.userMatrix[`r rinline("subid",
                           script=TRUE)`].rotate(angle, 0,1,0);
      plot3drgl.drawScene();
    };
    </script>
    
<script type="text/javascript">
var rotate = function(angle) {
  plot3drgl.userMatrix[`r subid`].rotate(angle, 0,1,0);
  plot3drgl.drawScene();
};
</script>

## Autogenerated controls

We can also change the contents of the plot using `r indexfns("toggleButton")`.
For example, we can redo the previous plot, but with the
three species as separate "spheres" objects and buttons to
toggle them:
```{r toggle, webgl = TRUE, rgl.newwindow = TRUE}
sphereid <- with(subset(iris, Species == "setosa"), 
     spheres3d(Sepal.Length, Sepal.Width, Petal.Length, 
                  col=as.numeric(Species),
                  radius = 0.211))
with(subset(iris, Species == "versicolor"), 
     spheres3d(Sepal.Length, Sepal.Width, Petal.Length, 
               col=as.numeric(Species),
     	       radius = 0.211))
with(subset(iris, Species == "virginica"), 
     spheres3d(Sepal.Length, Sepal.Width, Petal.Length, 
               col=as.numeric(Species),
     	       radius = 0.211))
aspect3d(1,1,1)
decorate3d()
subid <- currentSubscene3d()
```

```{r results="asis"}
toggleButton(sphereid, label = "setosa", prefix = "toggle", subscene = subid)
toggleButton(sphereid+1, label = "versicolor", prefix = "toggle", subscene = subid)
toggleButton(sphereid+2, label = "virginica", prefix = "toggle", subscene = subid)
```

Note that we need to use `results="asis"` for the button code.  
Normally we would also use `echo=FALSE`, though I didn't do so above;
then the buttons will end up side-by-side:

```{r results="asis", echo=FALSE}
toggleButton(sphereid, label = "setosa", prefix = "toggle", subscene = subid)
toggleButton(sphereid+1, label = "versicolor", prefix = "toggle", subscene = subid)
toggleButton(sphereid+2, label = "virginica", prefix = "toggle", subscene = subid)
```

An alternate control to achieve the same thing is `r indexfns("subsetSlider")`. 

```{r slider, webgl=TRUE}
```
```{r results="asis"}
subsetSlider(subsets = list(setosa = sphereid, 
                  versicolor = sphereid + 1, 
                  virginica = sphereid + 2, 
                  all = sphereid + 0:2),
             prefixes = "slider", subscenes = subid, 
             init = 3)
```

There are several other functions to generate the Javascript
code for controls.  `r indexfns("par3dinterpSetter")` generates
a function that approximates the result of `r linkfn("par3dinterp")`.
`r indexfns("propertySetter")` is a more general function to set
the value of properties of the scene.  Both generate Javascript
functions, but not the controls to use them; for that, use
`r indexfns("propertySlider")` or your own custom code.

For example, the following code (similar to the `r linkfn("play3d")`
example) rotates the scene in a complex way.
```{r userMatrix, webgl=TRUE}
```
```{r results="asis"}
M <- r3dDefaults$userMatrix
fn <- par3dinterp(time = (0:2)*0.75, userMatrix = list(M,
                                     rotate3d(M, pi/2, 1, 0, 0),
                                     rotate3d(M, pi/2, 0, 1, 0) ) )
propertySlider(setter = par3dinterpSetter(fn, 0, 1.5, steps=15, 
					   prefix = "userMatrix", 
					   subscene = subid),
	       step = 0.01)
```
Some things to note:  The generated Javascript slider has 150 increments,
so that motion appears smooth.  However, storing 150 `userMatrix` values
would take up a lot of space, so we use interpolation
in the Javascript code.  However, the Javascript code can only do 
linear interpolation, not the more complex spline-based SO(3)
interpolation done by `r linkfn("par3dinterp")`.  Because of this,
we need to output 15 steps from `r linkfn("par3dinterpSetter")`
so that the distortions of linear interpolation are not visible.  

Another function that auto-generates Javascript code is 
`r indexfns("clipplaneSlider")`.  This function allows the user to control
the location of a clipping plane by moving a slider.  Both it 
and `r linkfn("par3dinterpSetter")` are implemented
using the more general `r indexfns("propertySlider")`, which 
allows control of multiple objects in multiple scenes, but which 
does require knowledge of the internal representation of the scene
in its Javascript implementation.


Less general than `r linkfn("propertySetter")` is
`r indexfns("vertexSetter")`.  This function sets attributes
of individual vertices in a scene.  For example, to set the
x-coordinate of the closest point in the setosa group, and modify
its colour from black to white,

```{r vertex, webgl=TRUE}
```

```{r results="asis"}
setosa <- subset(iris, Species == "setosa")
which <- which.min(setosa$Sepal.Width)
init <- setosa$Sepal.Length[which]
propertySlider(
	vertexSetter(values = matrix(c(init,8,0,1,0,1,0,1), nrow=2),
	             attributes=c("x", "r", "g", "b"), 
		     vertices = which, objid = sphereid, 
		     prefix = "vertex"), 
	step=0.01)
```
	     

A related function is `r indexfns("ageSetter")`, though it uses
a very different specification of the attributes.
It is used when the slider controls the "age" of the scene, 
and attributes of vertices change with their age.  For example,
to show a point moving along a curve, we could do the following.  
Note that we need to specify multiple colours in the plot so that the
colour is not fixed, and can be controlled by the slider.  We
also give two `ageSetter` calls in a list; the `propertySlider`
will control both of them simultaneously.

```{r curve, webgl=TRUE}
time <- 0:500
xyz <- cbind(cos(time/20), sin(time/10), time)
lineid <- plot3d(xyz, type="l", col = c("black", "black"))["data"]
sphereid <- spheres3d(xyz[1, , drop=FALSE], radius = 8, col = "red")
```
```{r results="asis"}
propertySlider( list(ageSetter(births = time, ages = c(0, 0, 50),
			  colors = c("gray", "red", "gray"),
			  objids = lineid, prefixes = "curve"),
		     ageSetter(births = 0, ages = time,
		     	  vertices = xyz, objids = sphereid,
		     	  prefixes = "curve")),
		     maxS = max(time) + 50)
```

The final function of this type is `r indexfns("matrixSetter")`, for setting
up multiple controls to modify a matrix, typically `userMatrix`. This is used
when complex manipulation of a matrix requires several controls.

## User Defined Mouse Controls

`rgl` allows user defined mouse controls.  For these to work
within WebGL, you will need to write a Javascript version as
well as the R version.  This isn't easy:  R provides a lot
of support functions which are not easily available in Javascript.

TODO:  write a simple mouse control.

## Reference for `rglClass`

In writing the `writeWebGL()` function, I haven't tried to prevent access to 
anything.  On the other hand, I haven't provided access to
everything.  The parts documented here should remain relatively stable
(unless indicated otherwise).  Users may also consult the source
to `writeWebGL`, but should be aware that anything that isn't documented
here is subject to change without notice.  

`r indexclass("rglClass")`

As documented in `r linkfn("writeWebGL")`, the call
```{r eval=FALSE}
writeWebGL(..., prefix = "<prefix>")
```
will create a global object on the output page with name 
`<prefix>rgl` and Javascript class `rglClass`. 
This class has a large number of properties and methods, some of which are designed
to be available for use by other code on the web page. 

Most of the properties are stored as Javascript `Array` objects, indexed
by the `rgl` id of the subscene to which they apply.  There 
are also Javascript methods attached to the `rglClass` class.

### Methods

`r indexmethods("drawScene")`

After any change that will affect the display, code should
call `<prefix>rgl.drawScene()` to redraw the scene.

`r indexmethods(c("inSubscene", "addToSubscene", "delFromSubscene"))`

These methods each take two arguments:  `id` and `subscene`,
which should be the `rgl` ids of an object and a subscene.
`inSubscene` tests whether `id` is already included in the 
subscene, and the others 
add it or delete it from the subscene.

`r indexmethods(c("getSubsceneEntries"))`

This function takes a subscene id as argument, and returns an `Array`
containing all of the ids displayed in that subscene.

`r indexmethods(c("setSubsceneEntries"))`

This takes an `Array` of ids and a subscene id as arguments, and sets
the contents of the subscene to the ids.

### Properties

`r indexproperties(c("FOV", "listeners", "userMatrix", "zoom"))`

These correspond to the 
`r linkfn("par3d")` properties with the same names.  

- `FOV` and `zoom` are arrays of numbers. 
- `userMatrix` is an array
of `CanvasMatrix4` objects (documented in the file 
`system.file("WebGL/CanvasMatrix.src.js")`.
- Each `listeners` item is itself an array of subscene ids that "listen"
to mouse actions, i.e. `listeners[19]` would contain all
subscene ids that respond to mouse actions in subscene 19.

`r indexproperties("viewport")`

This property also corresponds to the
`r linkfn("par3d")` property, but should be considered to be 
read-only.

`r indexproperties(c("drawFns", "clipFns"))`

These two arrays contain the code to display 
each object in the scene.  The functions in the
`drawFns` array are called for each object
each time it is displayed.  The `clipFns` functions
are called when objects being clipped are drawn.

`r indexproperties(c("values", "offsets"))`

Most of the data about each object in a scene is contained in
the `values` property.  This is an array, indexed by object
id. The individual entries are numeric arrays.  Though they
are singly-indexed, the entries are meant to be interpreted
as matrices stored by row.  The first 3 columns are generally
the coordinates of a vertex, and remaining columns correspond
to other values from `r linkfn("rgl.attrib")`.  

The `offsets` property gives the (0-based) offset of the first 
column for a particular attribute in a named object.  Not all 
columns will be present in every object; if not
present, the corresponding `offsets` entry will be `-1`.
The entries are

Name | Meaning
-----|--------
vofs | Offset to 3 columns of vertex data in XYZ order
cofs | Offset to 4 columns of colour data in RGBA order
nofs | Offset to 3 columns of normal data
radofs | Offset to 1 column of sphere radius data
oofs | Offset to 2 columns of text or sprite origin data
tofs | Offset to 2 columns of texture coordinates
stride | Total number of columns in `values`

For example, to find the blue colour entry for vertex 
`i` in an object, one would first check if `offsets["cofs"]` was
`-1`, indicating that no colour information was present.  If
not, the entry could be found using
```
values[offsets["stride"]*(i-1) + offsets["cofs"] + 2]
```
This assumes `i` is specified using 1-based vertex counting
as in R, and writes `values` and `offsets` instead of the
fully specified `<prefix>rgl.values` and `<prefix>rgl.offsets`
for clarity.

Changes to `values` need to be pushed to the graphics system
to be reflected in the scene; see the calls to `gl.bindBuffer` and
`gl.bufferData` in the source to `r linkfn("propertySlider")`
for details.


## Index 

The following functions and `rglClass` properties and methods are described in this document:<br>

```{r echo=FALSE, results="asis"}
writeIndex(cols = 5)
```


