Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • multiApply multiApply
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 6
    • Issues 6
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Computational Earth SciencesComputational Earth Sciences
  • multiApplymultiApply
  • Issues
  • #12
Closed
Open
Issue created Sep 01, 2022 by aho@ahoMaintainer

Wrong result or error when the result dimension length of each chunk differs

The problem occurs when the output of the chunks Apply() loops over don't share the same dimension length. For example, input is an array with dimensions [sdate = 10, member = 3] and the target dimension is sdate. Apply() will applies function to each member, so there will be three chunks. If the three chunks all have output with length = 2, the final array will be [2, member = 3]. However, if the outputs of 1st and 2nd chunks have length = 2, but the 3rd one has length = 1, Apply() doesn't know how to merge them into one array. It either returns a wrong array and warning

In arrays_of_results[[component]][(1:prod(component_dims)) +  ... :
  number of items to replace is not a multiple of replacement length

or directly returns an error:

Error in arrays_of_results[[component]][(1:prod(component_dims)) + (m -  : 
  replacement has length zero

The problem is found in external/cstools#96 and also mentioned in #7 (comment 116762)

There are two solutions I can think of:
(1) If the dimension lengths are not the same, return a meaningful error message. It's reasonable that Apply() expects the same length of chunk outputs because we can be sure that the returned array has robust meaning (that is, the index 1 means the first time step, 33% percentile, etc.) and the results of all the chunks are aligned.

(2) Detect the largest length of the chunk output, and use NAs to fill the shorter output. So, take the example above, the final array will be [2, member = 3] and the value of [1, 3] is NA. A warning also needs to be returned so user can be aware of this situation.

@nperez What do you think? In my opinion, the 2nd option is more flexible. If it makes sense to you, I can fix the function by (2). Please let me know, thanks!

Best,
An-Chi

FYI @erifarov you may be interested in this issue, too

Assignee
Assign to
Time tracking