Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • startR startR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 29
    • Issues 29
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 8
    • Merge requests 8
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth Sciences
  • startRstartR
  • Issues
  • #104
Closed
Open
Created Jun 02, 2021 by Nuria Pérez-Zanón@nperezMaintainer

Three dependencies on file pattern

Hi @aho

There is one case in which it is needed to load files from different versions for different start dates. Given that they are DCPP files, they have also chunk label.

I have been exploring different possibilities but I haven't been able to load the data. I guess it is not possible because three 'depends' are needed. It said on the FAQ#8 that 2 is the maximum of dependencies.

I have also explored the ath_glob_permissive to try to avoid the 'chunk_depends' but it only takes the first pattern found (as far as I understood in the documentation).

In this case, the states 1962 and 1981 are desired to be loaded from v20200731 and state 1990 from 20200101

A code (with many commented lines, apologises) is here /esarchive/scratch/nperez/git/Flor/startR/Roberto_issueDepends.R

I have also unsuccessfully tried to use multiStart. In this case, I have add a line in the code of multiStart: .warning <- startR:::.warning since sourcing the files it was complaining there was no function .warning.

After that, I defined two paths:

repos <- list(v20200731 = list(name = 'v20200731', path =
                               paste0('/esarchive/exp/cesm1-1-cam5-cmip5/cmip6-dcppA-hindcast_i1p1/original_files/cmorfiles/DCPP/NCAR/CESM1-1-CAM5-CMIP5/dcppA-hindcast/$memb$/',
      mod,'/$var$/',grid,'/v20200731/$var$_',mod,
     '_CESM1-1-CAM5-CMIP5_dcppA-hindcast_s$sdate$-$memb$_',grid,'_$chunk$.nc',sep='')),
         list(v20200101 = list(name = 'v20200101', path =
                               paste0('/esarchive/exp/cesm1-1-cam5-cmip5/cmip6-dcppA-hindcast_i1p1/original_files/cmorfiles/DCPP/NCAR/CESM1-1-CAM5-CMIP5/dcppA-hindcast/$memb$/',mod,
    '/$var$/',grid,'/v20200101/$var$_',mod,
    '_CESM1-1-CAM5-CMIP5_dcppA-hindcast_s$sdate$-$memb$_',grid,'_$chunk$.nc',sep=''))))

and two sdate selectors:

   sdate = list(list(name = 'v20200731', sdate = indices(c(1,8))),
                          list(name = 'v20200101',  sdate = indices(c(1)))),

I think it would be good if you can take a look, first, to verify I am correctly understanding the issue, and second, to later discuss if Start() should deal with three dependencies or if multiStart is good to be used in this case.

I hope to talk to you soon.

Cheers,

Núria

Assignee
Assign to
Time tracking