Skip to content
GitLab
Closed
Issue created by Andrea@amanriquDeveloper

Retrieve files with wildcard * in path

Hi @aho and @nperez , I am trying to read together a set of files that have a different parameter in the path name . It seem sthat it should be possible with the the ' path_glob_permissive' option but it is not working. can you check what I am doing wrong? Thanks!

These are the files i want to read (the part i want to substitute with * is the last number, the rest is defined):

/esarchive/oper/S2S4E-data/weekly_statistics/S2S/tas/20200611/tas_20200611_24.nc

/esarchive/oper/S2S4E-data/weekly_statistics/S2S/tas/20200618/tas_20200618_25.nc

/esarchive/oper/S2S4E-data/weekly_statistics/S2S/tas/20200625/tas_20200625_26.nc

/esarchive/oper/S2S4E-data/weekly_statistics/S2S/tas/20200702/tas_20200702_27.nc ....

(up to 15 dates)

and this is what I have tried:

library(startR)
sdates.seq.thu <- format(seq(as.Date(paste(2020,06,11,sep='-')),as.Date(paste(2020,09,17,sep='-')),by='weeks'),format='%Y%m%d')


exp <- Start(dat = "/esarchive/oper/S2S4E-data/weekly_statistics/S2S/$var$/$sdate$/$var$_$sdate$_*.nc", 
              var = "tas",
              sdate = sdates.seq.thu,  
              time = 'all',
              ensemble = "all",
              latitude = indices(1:121),
              longitude = indices(1:240),
              path_glob_permissive = 1, # to allow '*' to be different in each file (1 level from the end)
            #  return_vars = list(latitude = 'dat',
	    #                longitude = 'dat',
            #                time = 'dat'),
	      retrieve = T)

And i get this error:

* Exploring files... This will take a variable amount of time depending
*   on the issued request and the performance of the file server...
Error in Start(dat = "/esarchive/oper/S2S4E-data/weekly_statistics/S2S/$var$/$sdate$/$var$_$sdate$_*.nc",  : 
  No data files found for any of the specified datasets.
In addition: Warning messages:
1: ! Warning: Parameter 'pattern_dims' not specified. Taking the first dimension,
!   'dat' as 'pattern_dims'. 
2: ! Warning: Could not find any pattern dim with explicit data set descriptions (in
!   the form of list of lists). Taking the first pattern dim, 'dat', as
!   dimension with pattern specifications. 
3: ! Warning: Too complex path pattern specified for dat1. Double check carefully the
!   '$Files' fetched for this dataset or specify a simpler path pattern.
Edited by Andrea