Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • startR startR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 29
    • Issues 29
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 7
    • Merge requests 7
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • startRstartR
  • Issues
  • #121
Closed
Open
Issue created Nov 11, 2021 by aho@ahoMaintainer

Transform + chunk doesn't work depending on the spatial range

The transform + chunk works for one dataset but not two datasets sometimes (e.g., the script in #29 (comment 127495)). In this script, if chunk = list(latitude = 2), for example, it works well. But if chunk = list(longitude = 2), the length of longitude of exp and obs is not the same, so it fails.

library(easyVerification)
library(SpecsVerification)

lons.min <- 0
lons.max <- 40
lats.min <- 0
lats.max <- 40

exp <- Start(dat = '/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc',
             var = 'tas',
             sdate = paste0(2000:2010,'0101'),
             ensemble = 'all',
             time = indices(1),
             latitude = values(list(lats.min, lats.max)),
             longitude = values(list(lons.min, lons.max)),
             transform = CDORemapper,
             transform_params = list(grid = 'r360x181',
                                     method = 'con', crop = c(lons.min,lons.max,lats.min,lats.max)),
             transform_vars = c('latitude', 'longitude'),
             synonims = list(latitude=c('lat','latitude'),longitude=c('longitude','lon')),
             return_vars = list(latitude = 'dat',
                                longitude = 'dat',
                                time=c('sdate')),
             retrieve= F)

obs <- Start(dat = '/esarchive/recon/ecmwf/era5/monthly_mean/$var$_f1h/$var$_$sdate$.nc',
             var = 'tas',
             sdate = paste0(2000:2010,'02'),
             time = indices(1),
             latitude = values(list(lats.min, lats.max)),
             longitude = values(list(lons.min, lons.max)),
             transform = CDORemapper,
             transform_params = list(grid = 'r360x181',
                                     method = 'con', crop = c(lons.min,lons.max,lats.min,lats.max)),
             transform_vars = c('latitude', 'longitude'),
             synonims = list(latitude=c('lat','latitude'),longitude=c('longitude','lon')),
             return_vars = list(latitude = 'dat',
                                longitude = 'dat',
                                time=c('sdate')),
             merge_across_dims = TRUE,
             retrieve= F)


# Define function (applied on the essential 'target' dimensions)
compute_scores <- function(exp, obs) {
  my.prob<-c(1/3,2/3)
  my.FairRpss <- veriApply("FairRpss", fcst=exp, obs=obs,
                           prob=my.prob, tdim=1, ensdim=2, na.rm=TRUE)[[1]]
  dim(my.FairRpss) <- c(score = length(my.FairRpss))
  return(my.FairRpss)
}
step <- Step(compute_scores,
             target_dims = list(exp=c('sdate','ensemble'),
                                obs=c('sdate')),
             output_dims = list(my.FairRpss = 'score'),
             use_libraries = c('easyVerification',
                               'SpecsVerification'))

# workflow of operations
wf <- AddStep(list(exp, obs), step)

# works
res1 <- Compute(wf, chunks = list(latitude = 2))

# doesn't work
res2 <- Compute(wf, chunks = list(longitude = 2))

For res2, the dimensions of the chunked data are as below. The longitude is 11 in exp while 10 in obs.

exp
      dat       var     sdate  ensemble      time  latitude longitude 
        1         1        11        25         1        21        11 
obs
      dat       var     sdate      time  latitude longitude 
        1         1        11         1        21        10 

But for res1, the latitude is both 9.

Edited Nov 18, 2021 by aho
Assignee
Assign to
Time tracking