Parallel {secr} | R Documentation

## Multi-core Processing

### Description

From version 2.4.0 secr makes limited use of multiple cores (CPUs) through the package parallel. Only the few secr functions listed below make any use of parallel processing. Increased speed can be expected with `sim.secr` (e.g., x3 with 4 cores), but gains in `secr.fit` are much smaller and may be negative.

The function `par.secr.fit` added in 2.7.1 is useful if you have a set of different models you wish to fit: if you have as many cores as models then all will complete in about the time required to run the longest.

Function | Unit | Benefit | Notes
`secr.fit` | session likelihood | small-moderate | multi-session models only
`score.test` | model | moderate | multi-model comparisons only
`derived` | session | moderate | SE by parameter if one session
`mask.check` | spacing x buffer | moderate-large | no file output, suppresses messages
`sim.secr` | replicate | large | all models, suppresses messages
`ip.secr` | replicate | large |
`LLsurface` | parameter combination | large |
`par.secr.fit` | model | large |
`par.derived` | fitted model | large |
`par.region.N` | fitted model | large |
`fxi.contour` | individual | moderate-large | multiple individuals only
`fxi.secr` | individual | moderate-large | multiple individuals only
`fx.total` | individual | moderate-large |
`pmixProfileLL` | level of pmix | large |

‘Unit’ refers to the unit of work sent to each worker process. As a guide, a ‘large’ benefit means >60% reduction in process time with 4 CPUs.

parallel offers several different mechanisms, bringing together the functionality of multicore and snow. The mechanism used by secr is the simplest available, and is expected to work across all operating systems. Technically, it relies on Rscript and communication between the master and worker processes is via sockets. As stated in the parallel documentation "Users of Windows and Mac OS X may expect pop-up dialog boxes from the firewall asking if an R process should accept incoming connections".

To use multiple cores, install parallel from CRAN and set `ncores > 1` in the function call. Use `detectCores()` to get an idea of how many cores are available on your machine; this may (in Windows) include virtual cores over and above the number of physical cores. See RShowDoc("parallel", package = "parallel") in core R for explanation.

You may possibly get warnings from R about closing unused connections. These can safely be ignored.

In `sim.secr`, new datasets are generated in the master process, so there is no need to manage the random number streams in the worker processes.

In `secr.fit` the output component ‘proctime’ misrepresents the elapsed processing time when multiple cores are used.

Worker processes are created in `secr.fit` with `makeCluster` and the options `methods = FALSE`, `useXDR = .Platform$endian=='big'` (consistently from 2.10.0). `methods = TRUE` is required for S4 methods, but as these are not used directly in secr it seems worth saving the time needed to load methods. `useXDR = FALSE` saves time “where large amounts of data are to be transferred and all the nodes are little-endian”, which applies on Windows systems and some linux systems. Please report any issues.

### Warning

It appears that multicore operations in secr using parallel may fail if the packages snow and snowfall are also loaded. The error message is obscure:

“Error in UseMethod("sendData") : no applicable method for 'sendData' applied to an object of class "SOCK0node"”

### Examples

    ## Not run:

    sessionInfo()
    # R version 3.2.0 (2015-04-16)
    # Platform: x86_64-w64-mingw32/x64 (64-bit)
    # Running under: Windows 7 x64 (build 7601) Service Pack 1
    # quad-core i7 CPU, 16 Gb RAM
    # ...

    ## almost no benefit with secr.fit

    for (i in 1:5)
        print(system.time(secr.fit(ovenCH, buffer = 400, trace = FALSE, ncores = i)))

    #   user  system elapsed
    #  52.26    0.76   53.02
    #   user  system elapsed
    #   3.38    2.77   48.36
    #   user  system elapsed
    #   5.69    5.36   47.18
    #   user  system elapsed
    #   6.50    6.90   54.51
    #   user  system elapsed
    #   7.74    8.25   50.23

    ## better for simulation, up to a point

    for (i in 1:8)
        print(system.time(sim.secr(secrdemo.0, nsim = 20, tracelevel = 0, ncores = i)))

    #   user  system elapsed
    # 262.93    3.74  266.71
    #   user  system elapsed
    #   0.68    0.02  139.61
    #   user  system elapsed
    #   0.69    0.01  110.05
    #   user  system elapsed
    #   0.70    0.00   84.24
    #   user  system elapsed
    #   0.70    0.02   69.45
    #   user  system elapsed
    #   0.67    0.06   75.67
    #   user  system elapsed
    #   0.75    0.00   70.00
    #   user  system elapsed
    #   0.73    0.05   75.71

    for (i in 1:8)
        print(system.time(ip.secr (captdata, trace = FALSE, ncores = i)))

    #   user  system elapsed
    # 132.59    0.00  132.58
    #   user  system elapsed
    #   0.25    0.05   71.08
    #   user  system elapsed
    #   0.20    0.04   50.76
    #   user  system elapsed
    #   0.32    0.07   42.13
    #   user  system elapsed
    #   0.26    0.09   41.74
    #   user  system elapsed
    #   0.39    0.08   39.01
    #   user  system elapsed
    #   0.25    0.09   36.52
    #   user  system elapsed
    #   0.46    0.08   35.91

    for (i in 1:8)
        print(system.time(LLsurface(secrdemo.0, ncores = i)))

    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #  48.00    0.40   48.41
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.00    0.03   27.94
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.00    0.03   20.60
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.03    0.02   17.63
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.01    0.05   16.41
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.02    0.04   16.10
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.05    0.03   16.03
    #Evaluating log likelihood across grid of 121 points...
    #   user  system elapsed
    #   0.02    0.07   15.99

    ## End(Not run)

[Package secr version 3.2.1 Index]