Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • S s2dverification
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 65
    • Issues 65
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 7
    • Merge requests 7
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • s2dverification
  • Merge requests
  • !230

Correct Corr() p value.

  • Review changes

  • Download
  • Patches
  • Plain diff
Merged aho requested to merge develop-Corr_pval into master Oct 05, 2020
  • Overview 0
  • Commits 1
  • Pipelines 0
  • Changes 1

Hi @nperez

This is the fix for issue #244 (closed). When transforming s2dverification::Corr to s2dv::Corr, I found that Corr() and .Corr() have different ways of calculation for p-value. I corrected the one in Corr(). I leave the script I used to test below. The results of s2dverification::Corr, s2dverification::.Corr, and s2dv::Corr are consistent now.

# dat1
  set.seed(1)
  exp1 <- array(rnorm(240), dim = c(dataset = 1, member = 2, sdate = 5,
                                   ftime = 3, lat = 2, lon = 4))
  set.seed(2)
  obs1 <- array(rnorm(120), dim = c(dataset = 1, member = 1, sdate = 5,
                                   ftime = 3, lat = 2, lon = 4))

# .Corr (ensemble mean for exp first, then calculate correlation of exp and obs)
exp_sub <- Subset(exp1, c(1, 4, 5, 6), list(1, 1, 1, 1), drop = 'selected')
exp_sub <- s2dverification:::.aperm2(exp_sub, c(2,1))
obs_sub <- Subset(obs1, c(1, 2, 4, 5, 6), list(1, 1, 1, 1, 1), drop = 'selected')
res_sub <- .Corr(exp_sub, obs_sub)

res_sub
$corr
[1] -0.4850409

$p_val
[1] 0.2037891

$conf_low
[1] -0.9575415

$conf_high
[1] 0.6943717

# ensemble mean to exp
exp1_mean <- apply(exp1, c(1, 3, 4, 5, 6), mean)
exp1_mean <- s2dverification::Enlarge(exp1_mean, 6)
exp1_mean <- s2dverification:::.aperm2(exp1_mean, c(1, 6, 2, 3, 4, 5))
names(dim(exp1_mean))[2] <- 'member'

# corrected s2dverification (compare with .Corr)
res_mean <- Corr(exp1_mean, obs1, posloop = 2, poscor = 3)

res_mean[1, 1, , 1, 1, 1, 1]
[1]  0.6943717 -0.4850409 -0.9575415  0.2037891


# s2dv (compare with .Corr)
res_mean <- Corr(exp1_mean, obs1)

str(res_mean)  # see the first value in each list
List of 4
 $ corr      : num [1, 1, 1, 1:3, 1:2, 1:4] -0.485 -0.2287 0.0982 -0.3002 -0.5456 ...
 $ p.val     : num [1, 1, 1, 1:3, 1:2, 1:4] 0.204 0.356 0.438 0.312 0.171 ...
 $ conf.lower: num [1, 1, 1, 1:3, 1:2, 1:4] -0.958 -0.924 -0.858 -0.935 -0.964 ...
 $ conf.upper: num [1, 1, 1, 1:3, 1:2, 1:4] 0.694 0.819 0.902 0.792 0.649 ...
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: develop-Corr_pval