Compute(): Error in !attr(selectors, "values") : invalid argument type when using experimental and (date-corresponding) observational data
I get this error when dispatching chunks with Compute() when start() is called "Using experimental and (date-corresponding) observational data " i.e. with an argument with 2 dimensions.
This is relevant for @lpalma and @llledo as well
The start() call identifies well the dimensions (with retrieve= false) and data is correctly downloaded with retrieve = true. However, when using retrive=F and then letting compute download the data in chunks I get this error.
Error in !attr(selectors, "values") : invalid argument type
(startR version is ‘0.1.2’)
Here is a simplified example:
library(multiApply) # Apply() function
library(startR) # Start() to load data
exp_path = paste0('/esarchive/exp/ncep/cfs-v2/weekly_mean/s2s/$var$_f24h/',
'$var$_$sdate$.nc')
obs_path <- paste0("/esarchive/recon/ecmwf/era5/weekly_mean/",
"$var$_f1h/$var$_$file_date$.nc")
hcst.sdates<-c( "19990104", "19990107", "19990111", "19990114", "19990118", "19990121", "19990125", "19990128", "20000104", "20000107", "20000111", "20000114",
"20000118", "20000121", "20000125", "20000128")
var_name<-'tas'
exp <- Start(dat = exp_path,
var = var_name,
sdate = hcst.sdates, #YYYYMMDD
time = indices(1:4),
latitude = indices(1),#values(list(lats.min, lats.max)),
longitude = indices(1),#values(list(lons.min, lons.max)),
ensemble='all',
return_vars = list(lat = NULL,
lon = NULL,
time = c('sdate')),
synonims = list(latitude=c('lat','latitude'),
longitude=c('lon','longitude')),
retrieve = F)
Obtain dates to load from obs
dates <- attr(exp, 'Variables')$common$time
file_date <- sapply(dates, format, '%Y%m%d')
dim(file_date) <- dim(dates)
obs <- Start(dat = obs_path,
var = var_name,
file_date= file_date, # dim('sdate','time')
latitude = indices(1),#values(list(lats.min, lats.max)),
longitude = indices(1),#values(list(lons.min, lons.max)),
return_vars = list(latitude = 'dat',
longitude = 'dat',
time = 'file_date'),
split_multiselected_dims = TRUE,
retrieve=F)
score_calc<-function(forecast,reference){
em<- Apply(forecast,'ensemble',mean,na.rm=TRUE)[[1]]
bias<-Apply(reference,c('sdate'),function(x) x-em)[[1]]
dim(bias)<-c(length(bias))
return(bias)
}
step <- Step(score_calc,
list(forecast = c('sdate','ensemble'),
reference = c('sdate')),
list(bias = c('sdate')),
use_libraries = c('multiApply'))
wf <- AddStep(list(exp,obs), step)
res <- Compute(wf,
chunks = list( time=4,
latitude=1,
longitude=1))#
'''
'''
> Full output:
> * Loading chunk 1 out of 4 ...
>Error in R_nc4_open: No such file or directory
>Error in R_nc4_open: No such file or directory
>Error in !attr(selectors, "values") : invalid argument type
>In addition: Warning messages:
>1: ! Warning: Parameter 'pattern_dims' not specified. Taking the first dimension,
>! 'dat' as 'pattern_dims'.
>2: ! Warning: Could not find any pattern dim with explicit data set descriptions (in
>! the form of list of lists). Taking the first pattern dim, 'dat', as
>! dimension with pattern specifications.
'''