This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
reproducibility [2015/11/12 17:11] fmassonn |
reproducibility [2015/12/14 08:33] fmassonn [9 December 2015] |
||
---|---|---|---|
Line 145: | Line 145: | ||
===== Summary of monthly meetings ===== | ===== Summary of monthly meetings ===== | ||
- | ==== 13 May 2015 ==== | + | ===== 13 May 2015 ===== |
=== Agenda === | === Agenda === | ||
Line 180: | Line 180: | ||
''' | ''' | ||
- | ==== 17 June 2015 ==== | + | ===== 17 June 2015 ===== |
=== Points Discussed === | === Points Discussed === | ||
Line 217: | Line 217: | ||
* ''' | * ''' | ||
- | ==== 2 November 2015 ==== | + | ===== 2 November 2015 ===== |
=== Points Discussed === | === Points Discussed === | ||
Line 228: | Line 228: | ||
**Asif and François** launch the reproducibility experiment from stabilized **i06c** on Ithaca and MareNostrum | **Asif and François** launch the reproducibility experiment from stabilized **i06c** on Ithaca and MareNostrum | ||
- | ==== 12 November 2015 ==== | + | ===== 9 November 2015 ===== |
+ | Stabilization of **i06c**. The plots {{stabilization_GMST.pdf|here}} and {{stabilization_ice.pdf|here}} show the stabilization of one of the member of i06c that was extended to achieve equilibrium. It was decided that the run was now in a sufficiently stable climate to perform the second stream of reproducibility experiments. | ||
+ | |||
+ | ===== 12 November 2015 ===== | ||
There was a meeting involving François, Kim and Oriol, to discuss about the meaning and the use of compilation flags/ | There was a meeting involving François, Kim and Oriol, to discuss about the meaning and the use of compilation flags/ | ||
Line 238: | Line 241: | ||
**1) Reproducibility on Ithaca**. To isolate the effect of the decomposition of processors, we'll first run a reproducibility experiment on Ithaca, started from the equilibrated restart we have obtained after 60 years of simulation. We'll just change the domain decomposition (number and distribution). Since all other things will be equal by construction, | **1) Reproducibility on Ithaca**. To isolate the effect of the decomposition of processors, we'll first run a reproducibility experiment on Ithaca, started from the equilibrated restart we have obtained after 60 years of simulation. We'll just change the domain decomposition (number and distribution). Since all other things will be equal by construction, | ||
+ | * Can we risk this strategy given that we don't know when we won't have access to Ithaca anymore? | ||
* The reference decomposition is 72: (32+16+22) . What can be another decomposition? | * The reference decomposition is 72: (32+16+22) . What can be another decomposition? | ||
* Ideally, the compiler version, MPI and LAPACK versions, SZIP-HDF5-NetCDF-GRIB versions should also be freezed now, if we want to then run other experiments on other platforms. | * Ideally, the compiler version, MPI and LAPACK versions, SZIP-HDF5-NetCDF-GRIB versions should also be freezed now, if we want to then run other experiments on other platforms. | ||
Line 248: | Line 252: | ||
More about compilation options can be found {{https:// | More about compilation options can be found {{https:// | ||
+ | |||
+ | |||
+ | ===== 9 December 2015 ===== | ||
+ | We had a meeting with usual people + Klaus and Uwe (SMHI) who are also tracking this reproducibility issue and are interested in what we are doing. Please visit [[https:// | ||
+ | |||
+ | The discussions were quite rich, and here is the summary in a few bullet points | ||
+ | * We have to be extremely **careful** when saying things like " | ||
+ | * SMHI is mostly interested in understanding what are the configurations under which EC-Earth is reproducible, | ||
+ | * Assessing reproducibility of a whole system is different from assessing reproducibility of one particular variable (e.g., Antarctic sea ice extent in winter). A good point of the Barker et al. paper referenced above is that their test is multivariate, | ||
+ | |||
+ | The topic is becoming extremely complex, far-reaching and our team looking into the topic is growing every month. On the other hand it has been a long-standing issue (almost one year now) and we need to have insights for the next EC-Earth meeting and the upcoming CMIP6. Here is a suggestion as how to continue the work: this should be split in two tasks | ||
+ | * **Developer aspect** - Xavi Yepes is now looking in the bit-for-bit reproducibility issue with EC-Earth 3.2. and for short (3-month) runs. SMHI (Uwe, Klaus) and KNMI (Philippe Le Sager) are aware of this. He is making several tests: | ||
+ | - Changing the number of processors in NEMO, IFS, both. | ||
+ | - Setting optimization to -O2 or -O3 | ||
+ | - Setting the -fp-model to precise, strict, source | ||
+ | * **User aspect** - Asif, François will continue adopting the " | ||
+ | |||
+ | |||
+ |