Commit cb0a5956 authored by Nicolau Manubens Gil's avatar Nicolau Manubens Gil
Browse files

Merge branch 'develop-upgrade-to-2.0.0' into 'master'

Updated documentation.

See merge request !8
parents 44b0b413 8a6682c5
Pipeline #765 passed with stage
in 1 minute and 3 seconds
......@@ -6,7 +6,7 @@ Authors@R: c(
person("Nicolau", "Manubens", , "nicolau.manubens@bsc.es", role = "aut"),
person("Alasdair", "Hunter", , "alasdair.hunter@bsc.es", role = "aut"),
person("Nuria", "Perez", , "nuria.perez@bsc.es", role = "cre"))
Description: The base apply function and its variants, as well as the related functions in the 'plyr' package, typically apply user-defined functions to a single argument (or a list of vectorized arguments in the case of mapply). The 'multiApply' package extends this paradigm to efficiently apply functions taking one or a list of multiple unidimensional or multidimensional arguments (or combinations thereof) as input, which can have different numbers of dimensions as well as different dimension lengths, and returning one or a list of unidimensional or multidimensional arrays as output. This saves development time by preventing the R user from writing error-prone and memory-unefficient loops dealing with multiple complex arrays. In contrast to apply and variants, this package suggests the use of 'target dimensions' as opposite to the 'margins' for specifying the dimensions relevant to the function to be applied. Also, two remarkable features of multiApply are the support for functions returning multiple array outputs and the transparent use of multi-core.
Description: The base apply function and its variants, as well as the related functions in the 'plyr' package, typically apply user-defined functions to a single argument (or a list of vectorized arguments in the case of mapply). The 'multiApply' package extends this paradigm with its only function, Apply, which efficiently applies functions taking one or a list of multiple unidimensional or multidimensional numeric arrays (or combinations thereof) as input. The input arrays can have different numbers of dimensions as well as different dimension lengths, and the applied function can return one or a list of unidimensional or multidimensional arrays as output. This saves development time by preventing the R user from writing often error-prone and memory-unefficient loops dealing with multiple complex arrays. Also, a remarkable feature of Apply is the transparent use of multi-core through its parameter 'ncores'. In contrast to the base apply function, this package suggests the use of 'target dimensions' as opposite to the 'margins' for specifying the dimensions relevant to the function to be applied.
Depends:
R (>= 3.2.0)
Imports:
......
......@@ -101,14 +101,17 @@ Apply <- function(data, target_dims = NULL, fun, ...,
"found across different inputs in 'data'. Please check ",
"carefully the assumed names below are correct, or provide ",
"dimension names for safety, or disable the parameter ",
"'guess_dimension_names'.", dim_names_string)
"'guess_dim_names'.", dim_names_string)
}
# Check fun
if (is.character(fun)) {
try({fun <- get(fun)}, silent = TRUE)
fun_name <- fun
err <- try({
fun <- get(fun)
}, silent = TRUE)
if (!is.function(fun)) {
stop("Could not find the function '", fun, "'.")
stop("Could not find the function '", fun_name, "'.")
}
}
if (!is.function(fun)) {
......
## multiApply [![build status](https://earth.bsc.es/gitlab/ces/multiApply/badges/master/build.svg)](https://earth.bsc.es/gitlab/ces/multiApply/commits/master) [![CRAN version](http://www.r-pkg.org/badges/version/multiApply)](https://cran.r-project.org/package=multiApply) [![coverage report](https://earth.bsc.es/gitlab/ces/multiApply/badges/master/coverage.svg)](https://earth.bsc.es/gitlab/ces/multiApply/commits/master) [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) [![CRAN RStudio Downloads](https://cranlogs.r-pkg.org/badges/multiApply)](https://cran.rstudio.com/web/packages/multiApply/index.html)
This package includes the function `Apply` as its only function. It extends the `apply` function to applications in which a function needs to be applied simultaneously over multiple input arrays. Although this can be done manually with for loops and calls to the base apply function, in many cases it can be a challenging task which can very easily result in error-prone or memory-unefficient code.
This package includes the function `Apply` as its only function. It extends the `apply` function to applications in which a function needs to be applied simultaneously over multiple input arrays. Although this can be done manually with for loops and calls to the base `apply` function, it can often be a challenging task which can easily result in error-prone or memory-unefficient code.
A very simple example follows showing the kind of situation where `Apply` can be useful: imagine you have two arrays, each containing five 2x2 matrices, and you want to perform the multiplication of each of the five pairs of matrices. Next, one of the best ways to do this with base R:
A very simple example follows showing the kind of situation where `Apply` can be useful: imagine you have two arrays, each containing five 2x2 matrices, and you want to perform the multiplication of each of the five pairs of matrices. Next, one of the best ways to do this with base R (plus some helper libraries):
```r
library(plyr)
......@@ -16,7 +16,7 @@ D <- aaply(X = abind(A, B, along = 4),
FUN = function(x) x[,,1] %*% x[,,2])
```
Although it is not excessively complex, the choosen example is very simple and the complexity would increase as the function to apply required additional dimensions or inputs, and would be unapplicable if multiple outputs were to be returned. In addition, the function to apply (matrix multiplication) had to be redefined for this particular case (multiplication of the first matrix by the second).
Since the choosen use case is very simple, this solution is not excessively complex, but the complexity would increase as the function to apply required additional dimensions or inputs, and would be unapplicable if multiple outputs were to be returned. In addition, the function to apply (matrix multiplication) had to be redefined for this particular case (multiplication of the first matrix along the third dimension by the second along the third dimension).
Next, an example of how to reach the same results using `Apply`:
......@@ -31,7 +31,16 @@ D <- Apply(data = list(A, B),
fun = "%*%")$output1
```
This solution takes half the time to complete, and is cleaner and extensible to functions receiving any number of inputs with any number of dimensions, or returning any number of outputs. Although the peak RAM usage (as measured with `peakRAM`) of both solutions in this example is about the same, it is challenging to avoid memory duplications when using custom code in more complex applications, and can usually require hours of dedication. `Apply` scales well to large inputs and has been designed to be fast and avoid memory duplications.
This solution takes half the time to complete (as measured with `microbenchmark` with inputs of different sizes), and is cleaner and extensible to functions receiving any number of inputs with any number of dimensions, or returning any number of outputs. Although the peak RAM usage (as measured with `peakRAM`) of both solutions in this example is about the same, it is challenging to avoid memory duplications when using custom code in more complex applications, and can usually require hours of dedication. `Apply` scales well to large inputs and has been designed to be fast and avoid memory duplications.
Additionally, multi-code computation can be enabled via the `ncores` parameter, as shown next. Although in this minimalist example using multi-core would make the execution slower, in applications where the inputs are larger the wall-clock time is reduced dramatically.
```r
D <- Apply(data = list(A, B),
target_dims = c(2, 3),
fun = "%*%",
ncores = 4)$output1
```
In contrast to `apply` and variants, this package suggests the use of 'target dimensions' as opposite to the 'margins' for specifying the dimensions relevant to the function to be applied. Additionally, it supports functions returning multiple vector or arrays, and can transparently uses multi-core.
......@@ -44,7 +53,7 @@ install.packages('multiApply')
library(multiApply)
```
Also, you can install the latest stable version from this GitHub repository as follows:
Also, you can install the latest stable version from the GitHub repository as follows:
```r
devtools::install_git('https://earth.bsc.es/gitlab/ces/multiApply')
......
No preview for this file type
......@@ -1197,6 +1197,74 @@ test_that("in1: 2 dim; in2: 1 dim; targ. dims: 0-2, 0-1; out1: 1 dim; out2: 1 va
## not shared target dims
#})
# Real cases
test_that("real use case - standardization", {
standardization <- function(x, mean, deviation){
(x - mean) / deviation
}
x <- array(1:(2*3*4), dim = c(mod = 2, lon = 3, lat = 4))
y <- array(1:12, dim = c(lon = 3, lat = 4))
z <- array(1:12, dim = c(lon = 3, lat = 4))
expected_result <- array(c(0:11 / z, rep(1, 3 * 4)), dim = c(3, 4, mod = 2))
expect_equal(
Apply(data = list(x,y,z),
target_dims = list(c('lon', 'lat'),
c('lon', 'lat'),
c('lon', 'lat')),
fun = standardization)$output1,
expected_result
)
names(dim(expected_result)) <- c('lon', 'lat', 'mod')
expect_equal(
Apply(data = list(x,y,z),
margins = list('mod', NULL, NULL),
fun = standardization,
output_dims = c('lon', 'lat')
)$output1,
expected_result
)
expect_equal(
Apply(data = list(x,y,z),
margins = list(c('mod', 'lat'), 'lat', 'lat'),
fun = standardization,
output_dims = c('lon')
)$output1,
multiApply:::.aperm2(expected_result, c(1, 3, 2))
)
x <- multiApply:::.aperm2(x, c(3, 2, 1))
expect_equal(
Apply(data = list(x,y,z),
target_dims = list(c('lon', 'lat'),
c('lon', 'lat'),
c('lon', 'lat')),
fun = standardization,
output_dims = c('lon', 'lat')
)$output1,
expected_result
)
})
# Test .aperm2
test_that(".aperm2", {
data <- seq(as.POSIXct('1990-11-01'),
length.out = 6,
by = as.difftime(1, units = 'days'))
dim(data) <- c(3, 2)
expect_equal(
class(multiApply:::.aperm2(data, c(2, 1))),
c('POSIXct', 'POSIXt')
)
})
# TODOS:
# TESTS FOR MARGINS
# TESTS FOR DISORDERED TARGET_DIMS
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment