Start(): disordered data when num_procs > 1
As reported by @llledo , on CTE-Power9, when the parameter num_procs
is set to a value greater than 1 (loading data from files onto R RAM memory with more than 1 parallel process), the resulting data is disordered. This is not happening on workstations or fat nodes.
Probably this is due to the recent updates in Power9. I should try reinstall all dependencies of startR on power9, I will do it next week. In the meantime, load data with num_procs = 1
.
.libPaths(new = '/gpfs/projects/bsc32/share/R_libs/3.5/')
library(startR)
library(lubridate)
args <- c("ERAI", "sfcWind", "JFM", "1996", "1999", "0")
model <- args[1]
var <- args[2]
season <- args[3]
firsty <- as.numeric(args[4])
lasty <- as.numeric(args[5])
lags <- as.numeric(unlist(strsplit(args[6],",")))
period <- paste0(firsty,"-",lasty)
years <- firsty:lasty
if(season=="JFM") {
months <- 1:3
firstd <- paste0(firsty,"0101")
}
if(season=="OND") {
months <- 10:12
firstd <- paste0(firsty,"1001")
}
sdates <- ymd(firstd) + months(0:2) + rep(years(0:(length(years)-1)),each=3)
if(model=="ERAI") { dataset <- "/esarchive/recon/ecmwf/erainterim/daily_mean/$var$_f6h/$var$_$sdate$.nc" }
if(model=="MERRA2") { dataset <- "/esarchive/recon/nasa/merra_v2/daily_mean/$var$_f1h/$var$_$sdate$.nc" }
if(model=="ERA5") { dataset <- "/esarchive/recon/ecmwf/era5/daily_mean/$var$_f1h/$var$_$sdate$.nc" }
wind <- Start(dataset = dataset,
var = var,
sdate = indices(1),
time = indices(1),
latitude = 'all',
longitude = 'all',
return_vars = list(time = NULL, latitude = NULL, longitude = NULL),
num_procs = 2, retrieve = TRUE,
synonims=list(longitude = c('lon', 'longitude'),
latitude = c('lat', 'latitude'))
)
# should display a lon-lat field
image(wind[1,1,1,1,,])