User Tools

Site Tools


reproducibility

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
reproducibility [2016/02/05 11:18]
macosta
reproducibility [2017/11/10 14:03] (current)
fmassonn
Line 309: Line 309:
  
   * Determine the best method to quantify differences between runs   * Determine the best method to quantify differences between runs
-      * Propose a reference simulation which we can use to compare the rest of experiments. This reference simulation could be use in the future to check runs in new platforms, the inclusion of new modules, etc.+      * Propose a reference which we can use to compare the rest of experiments. This reference could be use in the future to check runs in new platforms, the inclusion of new modules, etc.
       * Use a statistical method to quantify the differences between runs and propose a minimum to achieve instead of bitwise precision in order to avoid critical restrictions in performance.       * Use a statistical method to quantify the differences between runs and propose a minimum to achieve instead of bitwise precision in order to avoid critical restrictions in performance.
       * Propose a method to know which of two simulations with valid results is the best. Some experiments using different compiler flags will obtain similar valid results (maybe with differences of only 1%). It would be convenient to know which obtain better results (quality of the simulation results).       * Propose a method to know which of two simulations with valid results is the best. Some experiments using different compiler flags will obtain similar valid results (maybe with differences of only 1%). It would be convenient to know which obtain better results (quality of the simulation results).
Line 316: Line 316:
   * If bit for bit reproducibility was achieved using ec-earth3.1, study how to obtain it using ec-earth3.2beta at least in a debug mode.   * If bit for bit reproducibility was achieved using ec-earth3.1, study how to obtain it using ec-earth3.2beta at least in a debug mode.
  
 +===== 27th of May 2016 =====
 +See the summarizing presentations of {{20160526_groupmeeting.pdf | François }} and {{20160526_EC-Earth3.2_MarioAcosta.pdf | Mario }}. A more general set of slides about climate-reproducibility is available {{ 20160526_EC-Earth3.1_FrancoisMassonnet.pdf | here }} and was also posted on the EC-Earth development portal issue {{https://dev.ec-earth.org/issues/207 | 207}}.
 +
 +Actions:
 +* Mario runs an experiment with **-fpe0** activated, on ECMWF.
 +* Mario/Oriol: Tests are to be made with libraries (NetCDF, GRIB, etc.) compiled with the same options and the same version of the code.
 +
 +===== 10th of November 2017 =====
 +Martin and François have worked to make the scripts testing the reproducibility more universal. These can now be found in the following gitlab project:
 +
 +https://earth.bsc.es/gitlab/fmassonnet/reproducibility.git
 +
 +A draft of the paper has been created:
 +
 +https://docs.google.com/document/d/1aMsdggygIGmbyiFmmEOEFIl6ZVe-EO7Jcd04B6ZP91A/edit
reproducibility.1454671083.txt.gz · Last modified: 2016/02/05 11:18 by macosta