Memory issues when loading Era5Land datasets that could previously be loaded
Hi @vagudets,
Summary
Bug: StartR now shows memory issues where it previously was able to load Era5Land data without issue.
Example
Attaching the part of the script in question:
library(startR)
library(lubridate)
lonmin <- -22
lonmax <- 45
latmin <- 27
latmax <- 72
sdates_obs <- format(rep(ymd(paste0(1993:2018, "0201")), each = 12) + months(0:11), "%Y%m")
# Observations: reanalysis ERA5-Land
# This configuration used to load before...
obs <- startR::Start(
dat = "/esarchive/recon/ecmwf/era5land/monthly_mean/$var$_f1h/$var$_$sdate$.nc",
var = "tas",
time = indices(1),
latitude = values(list(latmin, latmax)),
sdate = sdates_obs,
latitude_reorder = Sort(decreasing = FALSE),
longitude = values(list(lonmin, lonmax)),
longitude_reorder = CircularSort(-180, 180),
synonims = list(var = c("var", "variable"),
longitude = c("lon", "longitude"),
latitude = c("lat", "latitude")),
return_vars = list(latitude = "dat", longitude = "dat"),
num_procs = 1,
retrieve = TRUE
)
Error in bigmemory::big.matrix(nrow = prod(final_dims), ncol = 1) :
Error: memory could not be allocated for instance of type big.matrix
Module and Package Version
- R version is 4.1.2
- startR version is 2.3.1
- Bug can be reproduced in WS or Nord3