Parallel {secr} | R Documentation |
From version 2.4.0 secr makes limited use of multiple cores (CPUs)
through the package parallel. Only the few secr
functions listed below make any use of parallel processing. Increased
speed can be expected with sim.secr
(e.g., x3 with 4 cores), but
gains in secr.fit
are much smaller and may be negative.
The function par.secr.fit
added in 2.7.1 is useful if you have a set of
different models you wish to fit: if you have as many cores as models
then all will complete in about the time required to run the longest.
Function | Unit | Benefit | Notes |
secr.fit | session likelihood | small-moderate | multi-session models only |
score.test | model | moderate | multi-model comparisons only |
derived | session | moderate | SE by parameter if one session |
mask.check | spacing x buffer | moderate-large | no file output, suppresses messages |
sim.secr | replicate | large | all models, suppresses messages |
ip.secr | replicate | large | |
LLsurface | parameter combination | large | |
par.secr.fit | model | large | |
par.derived | fitted model | large | |
par.region.N | fitted model | large | |
fxi.contour | individual | moderate-large | multiple individuals only |
fxi.secr | individual | moderate-large | multiple individuals only |
fx.total | individual | moderate-large | |
pmixProfileLL | level of pmix | large | |
‘Unit’ refers to the unit of work sent to each worker process. As a guide, a ‘large’ benefit means >60% reduction in process time with 4 CPUs.
parallel offers several different mechanisms, bringing together the functionality of multicore and snow. The mechanism used by secr is the simplest available, and is expected to work across all operating systems. Technically, it relies on Rscript and communication between the master and worker processes is via sockets. As stated in the parallel documentation "Users of Windows and Mac OS X may expect pop-up dialog boxes from the firewall asking if an R process should accept incoming connections".
To use multiple cores, install parallel from CRAN and set
ncores > 1
in the function call. Use detectCores()
to get
an idea of how many cores are available on your machine; this may (in
Windows) include virtual cores over and above the number of physical
cores. See RShowDoc("parallel", package = "parallel") in core R for
explanation.
You may possibly get warnings from R about closing unused connections. These can safely be ignored.
In sim.secr
, new datasets are generated in the master process, so
there is no need to manage the random number streams in the worker
processes.
In secr.fit
the output component ‘proctime’ misrepresents the
elapsed processing time when multiple cores are used.
Worker processes are created in secr.fit
with makeCluster
and the options methods = FALSE
, useXDR = .Platform$endian=='big'
(consistently from 2.10.0).
methods = TRUE
is required for S4 methods, but as these are not used directly
in secr it seems worth saving the time needed to load methods. useXDR = FALSE
saves time “where large amounts of data are to be transferred and all the nodes are little-endian”, which applies on Windows systems and some linux systems. Please report any issues.
It appears that multicore operations in secr using parallel may fail if the packages snow and snowfall are also loaded. The error message is obscure:
“Error in UseMethod("sendData") : no applicable method for 'sendData' applied to an object of class "SOCK0node"”
## Not run: sessionInfo() # R version 3.2.0 (2015-04-16) # Platform: x86_64-w64-mingw32/x64 (64-bit) # Running under: Windows 7 x64 (build 7601) Service Pack 1 # quad-core i7 CPU, 16 Gb RAM # ... ## almost no benefit with secr.fit for (i in 1:5) print(system.time(secr.fit(ovenCH, buffer = 400, trace = FALSE, ncores = i))) # user system elapsed # 52.26 0.76 53.02 # user system elapsed # 3.38 2.77 48.36 # user system elapsed # 5.69 5.36 47.18 # user system elapsed # 6.50 6.90 54.51 # user system elapsed # 7.74 8.25 50.23 ## better for simulation, up to a point for (i in 1:8) print(system.time(sim.secr(secrdemo.0, nsim = 20, tracelevel = 0, ncores = i))) # user system elapsed # 262.93 3.74 266.71 # user system elapsed # 0.68 0.02 139.61 # user system elapsed # 0.69 0.01 110.05 # user system elapsed # 0.70 0.00 84.24 # user system elapsed # 0.70 0.02 69.45 # user system elapsed # 0.67 0.06 75.67 # user system elapsed # 0.75 0.00 70.00 # user system elapsed # 0.73 0.05 75.71 for (i in 1:8) print(system.time(ip.secr (captdata, trace = FALSE, ncores = i))) # user system elapsed # 132.59 0.00 132.58 # user system elapsed # 0.25 0.05 71.08 # user system elapsed # 0.20 0.04 50.76 # user system elapsed # 0.32 0.07 42.13 # user system elapsed # 0.26 0.09 41.74 # user system elapsed # 0.39 0.08 39.01 # user system elapsed # 0.25 0.09 36.52 # user system elapsed # 0.46 0.08 35.91 for (i in 1:8) print(system.time(LLsurface(secrdemo.0, ncores = i))) #Evaluating log likelihood across grid of 121 points... # user system elapsed # 48.00 0.40 48.41 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.00 0.03 27.94 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.00 0.03 20.60 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.03 0.02 17.63 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.01 0.05 16.41 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.02 0.04 16.10 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.05 0.03 16.03 #Evaluating log likelihood across grid of 121 points... # user system elapsed # 0.02 0.07 15.99 ## End(Not run)