Performance exploration: Ano()
Hi @nperez
Following issues #7 (closed) and ces/multiApply#11 (closed), I want to have a more complete exploration of the performance of s2dv functions.
Here are some methods we've identified which may improve efficiency:
- Use apply() instead of Apply()
- Use compiled atomic function
The things to be kept in mind when testing:
- The size and complexity of the data
- multiple cores usage
I did some tests for Ano(), and the results are somehow different from Season() and Regression().
(1) s2dverification::Ano
(2) s2dv::Ano
(3) s2dv::Ano2
: use 2 cores
(4) s2dv::Ano_compile
: cmpfun(.Ano)
** I didn't use apply() because it is a bit complicated changing Ano() by this way. I don't wanna invest time in it for now.
First, I tested with small data. The results show:
- s2dverification::Ano is 10x faster than s2dv::Ano.
- Compilation doesn't help in this case, either. In fact, it is sometimes a bit faster than s2dv::Ano, but the insignificant result shows that it's not a solution.
- ncores = 2 is much slower than ncores = 1 (I tested many times and the results are consistent.)
# Small data
set.seed(1)
data1 <- array(rnorm(360), c(dat = 2, member = 3, sdate = 50, ftime = 12))
set.seed(2)
clim1 <- array(rnorm(72), c(dat = 2, member = 3, ftime = 12))
benchmark("s2dverification::Ano" = {
res_s2dverification <- s2dverification::Ano(data1, clim1)
},
"s2dv::Ano" = {
res_s2dv <- s2dv::Ano(data1, clim1, ncores = 1)
},
"s2dv::Ano2" = {
res_s2dv <- s2dv::Ano(data1, clim1, ncores = 2)
},
"s2dv::Ano_compile" = {
res_s2dv_compile <- Ano_compile(data1, clim1)
},
replications = 50,
columns = c("test", "replications", "elapsed",
"relative", "user.self", "sys.self"))
# test replications elapsed relative user.self sys.self
#2 s2dv::Ano 50 0.188 10.444 0.188 0.000
#4 s2dv::Ano_compile 50 0.236 13.111 0.236 0.000
#3 s2dv::Ano2 50 1.316 73.111 0.240 0.916
#1 s2dverification::Ano 50 0.018 1.000 0.018 0.000
I also tested with larger data. s2dv::Ano and Ano_compile are still similar, but only 4 times slower than s2dverification::Ano. And 2 cores is still slower than 1 core.
# Large data
set.seed(1)
data2 <- array(rnorm(360), c(dat = 2, member = 3, sdate = 50, ftime = 12, lon = 360, lat = 180))
set.seed(2)
clim2 <- array(rnorm(72), c(dat = 2, member = 3, ftime = 12, lon = 360, lat = 180))
#(1)
system.time(res_s2dverification <- s2dverification::Ano(data2, clim2))
user system elapsed
6.552 1.444 8.040
#(2)
system.time(res_s2dv <- Ano(data2, clim2, ncores = 1))
user system elapsed
26.234 6.416 32.829
#(3)
system.time( res_s2dv <- Ano(data2, clim2, ncores = 2))
user system elapsed
27.681 9.488 37.439
#(4)
system.time(res_s2dv_compile <- Ano_compile(data2, clim2))
user system elapsed
26.226 6.013 32.432
The results (especially ncores = 2) still confuse me. I don't know if this is a real issue or if we should keep investigating it. After all, as you said, multiApply doesn't really have a 'problem' as long as we use it appropriately (e.g., deal with large data).
Do you have any idea about the ncores result? The complete code is here: /home/Earth/aho/aho-testtest/s2dv/benchmark/benchmark-Ano.R
Cheers,
An-Chi