I have included the following new parameters in RandomWalkTest (https://earth.bsc.es/gitlab/es/s2dv/-/blob/develop-RandomWalkTest/R/RandomWalkTest.R):
- N.eff: Effective sample size to be used in the statistical significance test. It can be NA (to use the length of the "time_dim" dimension) or an array with the same dimensions as "skill_A" except "time_dim" (for a particular N.eff to be used for each case). The default value is NA.
In case we want to compute the N.eff when using metric functions (like RPSS and RMSSS) Eno() has to be used outside RandomWalkTest() since .Eno() needs the data (e.g., obs) and RandomWalkTest() only gets skill_A and skill_B as inputs (but not the actual data).
- test.type: A character string indicating the type of significance test. It can be "two.sided.approx" (to assess whether forecaster A and forecaster B are significantly different in terms of skill with a two-sided test using the approximation of DelSole and Tippett, 2016), "two.sided" (to assess whether forecaster A and forecaster B are significantly different in terms of skill with a exact two-sided test), "greater" (to assess whether forecaster A shows significantly better skill than forecaster B with a one-sided test for negatively oriented scores), or "less" (to assess whether forecaster A shows significantly better skill than forecaster B with a one-sided test for positively oriented scores). The default value is "two.sided.approx".
I have left "two.sided.approx" as the default option for reproducibility with the previous version (it was the only possible test.type).
- alpha: A numeric of the significance level to be used in the statistical significance test. If it is a numeric, the significance will be returned. If NULL, the p-value will be returned instead. The default value is 0.05.
The p-value cannot be returned if test.type=="two.sided.approx". Also, this approximation is only for alpha==0.05. I have put some warnings (lines 128-135) in case the user asks for the p-value (i.e., alpha=NULL) or chooses alpha != 0.05.
Another thing: please see lines 162-167. An extra dimension is created in Apply. I think it is because target_dims of N.eff is NULL, but I am not sure. I have put those lines to remove that extra dimension, but you may know a better way of doing it :)
Please let me know if there is anything you want to discuss.
Best regards,
Carlos