Wrong result or error when the result dimension length of each chunk differs
The problem occurs when the output of the chunks Apply() loops over don't share the same dimension length. For example, input is an array with dimensions [sdate = 10, member = 3] and the target dimension is sdate
. Apply() will applies function to each member, so there will be three chunks. If the three chunks all have output with length = 2, the final array will be [2, member = 3]. However, if the outputs of 1st and 2nd chunks have length = 2, but the 3rd one has length = 1, Apply() doesn't know how to merge them into one array. It either returns a wrong array and warning
In arrays_of_results[[component]][(1:prod(component_dims)) + ... :
number of items to replace is not a multiple of replacement length
or directly returns an error:
Error in arrays_of_results[[component]][(1:prod(component_dims)) + (m - :
replacement has length zero
The problem is found in external/cstools#96 and also mentioned in #7 (comment 116762)
There are two solutions I can think of:
(1) If the dimension lengths are not the same, return a meaningful error message. It's reasonable that Apply() expects the same length of chunk outputs because we can be sure that the returned array has robust meaning (that is, the index 1 means the first time step, 33% percentile, etc.) and the results of all the chunks are aligned.
(2) Detect the largest length of the chunk output, and use NAs to fill the shorter output. So, take the example above, the final array will be [2, member = 3] and the value of [1, 3] is NA. A warning also needs to be returned so user can be aware of this situation.
@nperez What do you think? In my opinion, the 2nd option is more flexible. If it makes sense to you, I can fix the function by (2). Please let me know, thanks!
Best,
An-Chi
FYI @erifarov you may be interested in this issue, too