User Tools

Site Tools


ec-earth_experiment_meeting

This is an old revision of the document!


23/03/2017

  • EC-Earth portal: Please have a look often at the EC-Earth and shaconemo portal, inform the others (or only Etienne and Javier) of relevant information. Also open issue there.
  • Domingo/Joan: Autosubmit new developments? bugfixes?
  • EC-Earth 3.2.2 released :
    • Progress about internal release into auto-ecearth3 ? Miguel / Mingu / Pablo
    • IFS compilation fpe0 : Miguel ?
    • ec2cmor integrated ? PA
    • progress to integrate all post processed variables Eleftheria (oce) ? Etienne (atm) ? Neven (ice) ? Laurent (oce + ice)? Javier V ?
  • PRIMAVERA simulations:
    • PRIMAVERA outclass (Etienne): three variable missing ? Update ?
    • PRIMAVERA spin-up status: 40 years spinup done on MN3? Eleftheria / Etienne ?
    • strategy ?
  • Computing hours:
    • RES account: Fini terrae: 400Mhours → to be used to : 1. run a seasonal prediction experiment with EC-Earth 3.2.1 for CMUG (deadline June) + APPLICATE + IMPREX + PREFACE ; 2. Perform small tests required by all members of the group - be parcimonious in the use of hours because we might come short as well + pas de haute resolution
    • ECMWF: 0.06% → to be used for PRIMAVERA spinup and high-resolution sea ice reanalysis - NOT ENOUGH HOURS - we'll request an extension once we have burnt the hours we have
  • Tuning of 3.2 (Martin, Eleftheria) - pas d'heures de calcul
  • Next computing hours applications (RES: Eleftheria, Omar, Neven, Javier, Martin, Prace: Virginie, Japanese computer?)
  • PRIMAVERA planning : Etienne : spinup ; Pablo : ctrl + historique ?
  • DCPP planning: distribution membres de prediction aux membres du groupe

01/03/2017

  • Communication: In order to avoid spamming from gitlab, Javier and Etienne will be tagged in the issues (instead of cp) and they would be in charged of spreading around important informations.
  • Ec-Earth SCRUM meeting has to take place every week and the attendants will send a summary of the important issues and decisions.
  • EC-Earth portal: Please have a look often at the EC-Earth and shaconemo portal, inform the others (or only Etienne and Javier) of relevant information. Also open issue there.
  • Domingo/Joan: Autosubmit new developments? bugfixes?
  • PRIMAVERA simulations:
    • PRIMAVERA ICs, share with partners? (Neven, Omar)
    • PRIMAVERA outclass (Etienne): three variable missing (nothing done since last meeting)
    • PRIMAVERA spin-up status: 2 years spinup done on MN3, working on 3.2.1
  • Status auto-ecearth release (3.2.??):
    • Ec-Earth 3.2.2 news?
    • ec2cmor integration update
    • progress to integrate all post processed variables
  • Computing hours:
    • pr1eaq: 77% until 14-Mar-2017 (1.1Mhours left) → long queuing time, issues, strategy?
    • RES account: Fini terrae: 400Mhours
    • ECMWF: 0.06% Not to use until marenostrum 3 closes
  • Tuning of 3.2 (Martin, Eleftheria)
  • DCPP initial conditions all ready? sea ice initial conditions (Neven), ocean and atmosphere ready
  • Next computing hours applications (RES: Eleftheria, Omar, Neven, Javier, Martin, Prace: Virginie, Japanese computer: Chloe to check the call)

10/01/2017

  • Communication: In order to avoid spamming from gitlab, Javier and Etienne will be tagged in the issues (instead of cp) and they would be in charged of spreading around important informations.
  • Ec-Earth SCRUM meeting has to take place every week and the attendants will send a summary of the important issues and decisions.
  • EC-Earth portal: Please have a look often at the EC-Earth and shaconemo portal, inform the others (or only Etienne and Javier) of relevant information. Also open issue there.
  • PRIMAVERA simulations:
    • PRIMAVERA ICs, share with partners? (Neven, Omar)
    • PRIMAVERA outclass (Etienne): three variable missing (nothing done since last meeting)
    • status: 2 years spinup done on MN3, working on 3.2.1 (will finish this week)
    • parallel 3.2.1_develop_primavera branch for PRIMAVERA only
  • Next auto-ecearth release (3.2.1a):
    • Progress towards new release: s2dverification, Earth Diag, merge with Primavera/trunk?(Miguel, Mario, Mingu)
    • changes to SPECS outclass which uses too much memory OR use other outclass
    • Ec-Earth 3.2.2 news?
    • ec2cmor integration update
    • progress to integrate all post processed variables
  • Computing hours:
    • pr1eaq: 39% until 14-Mar-2017 (3Mhours left)
    • RES account: ???
    • ECMWF: 0% Not to use until marenostrum 3 closes
  • Tuning of 3.2 (participation on the portal)
  • DCPP initial conditions all ready? sea ice initial conditions (François), ocean initial conditions (Chloé), atmospheric initial conditions (Omar, Martin, LP)
  • Domingo/Joan: Autosubmit new developments? bugfixes?
  • new PRACE project to be prepared in September by Virginie (http://www.prace-ri.eu/prace-project-access/); new K-computer application to be prepared in September by Virginie (http://www.hpci-office.jp/pages/e_h29_boshu); RES applications to be prepared by January by Martin, Chloé, Javier and Etienne (http://www.bsc.es/marenostrum-support-services/res)

27/10/2016

27/10/2016

  • Communication: In order to avoid spamming from gitlab, Javier and Chloé will be tagged in the issues (instead of cp) and they would be in charged of spreading around important informations.
  • Ec-Earth SCRUM meeting has to take place every week and the attendants will send a summary of the important issues and decisions.
  • EC-Earth portal: Please have a look often at the EC-Earth and shaconemo portal, inform the others (or only Chloé and Javier) of relevant information. Also open issue there.
  • PRIMAVERA simulations:
    1. PRIMAVERA ICs, share with partners? (Neven, Omar)
    2. Primavera branch (consortium): MACSP
    3. Status of the release for PRIMAVERA (Miguel, Mario, Mingu)
    4. PRIMAVERA outclass (Etienne): three variable missing
    5. testing suite status
    6. ec2cmor integration update
    7. plan to integrate all post processed variables
  • Mario/Miguel: Status of the release and development plan
  • Computing hours:
    • pr1eaq: 16% (775 kh)
    • RES account: ???
    • ECMWF: 42% 66 days lefts (21mSBU left).
  • Tuning of 3.2, any suggestion?
  • DCPP initial conditions all ready? sea ice initial conditions (François), ocean initial conditions (Chloé), atmospheric initial conditions (Omar, Martin, LP)
  • Domingo/Joan: Autosubmit new developments? bugfixes? Eleftheria - Testing of version 3.7.0?
  • EC-Earth development portal: any news about update to the EC-Earth 3.2 code (NEMO3.6 bugfixes)?
  • new PRACE project to be prepared in September by Virginie (http://www.prace-ri.eu/prace-project-access/); new K-computer application to be prepared in September by Virginie (http://www.hpci-office.jp/pages/e_h29_boshu); RES applications to be prepared by January by Martin, Chloé, Javier and Etienne (http://www.bsc.es/marenostrum-support-services/res)

10/10/2016

  • Communication: In order to avoid spamming from gitlab, Javier and Chloé will be tagged in the issues (instead of cp) and they would be in charged of spreading around important informations.
  • Ec-Earth SCRUM meeting has to take place every week and the attendants will send a summary of the important issues and decisions.
  • EC-Earth portal: Please have a look often at the EC-Earth and shaconemo portal, inform the others (or only Chloé and Javier) of relevant information. Also open issue there.
  • PRIMAVERA simulations:
    1. Primavera branch (consortium): MACSP
    2. Status of the release for PRIMAVERA (Miguel, Mario, Mingu)
    3. Post-processing
    4. PRIMAVERA outclass
    5. testing suite
    6. CMOR strategy: specs2cmor, ec2cmor
    7. Restart of current chunk
    8. PRIMAVERA ICs

14/09/2016

  • BSC visibility - need to contribute to the EC-Earth consortium - BSC was in charge of developing the HR configuration of EC-Earth 3.2 : https://dev.ec-earth.org/projects/cmip6/wiki - we need to be actor and not only users of EC-Earth.
  • All: From Paco VERY IMPORTANT check the activity of the EC-Earth development portal at least once a week, watch all those issues that might concern your work, update the issues in which you participate particularly if they haven't experienced any movement in the last two or three weeks
  • Issue traceability : auto-ecearth, auto-nemo, autosubmit - need to register which commit and which release hold the bug-fix for each issue opened in gitlab (many issues only contain a message stating that the issue was solved but where ?) - need to have a clear association between a commit and a bug-fix (some issues are said to be fixed in a develop branch and when the user switch to the develop branch many other bugs show up) - need to set a deadline to fix the issues (some remain open for more than 6 months) - need to set priorities on the issues
  • Domingo/Joan: Autosubmit new developments? bugfixes? Eleftheria - Testing of version 3.7.0?
  • Mario/Miguel: EC-Earth high resolution version progress? standard resolution additional features? Testing suite? Status of the templates - clean and clear enough to be distributed to the EC-Earth consortium? From Paco: Also go through the list of EC-Earth3.2 priorities prepared by CP for CES, plans for new releases; maybe explain what will be done during the SCRUM meetings
  • EC-Earth development portal: any news about update to the EC-Earth 3.2 code (NEMO3.6 bugfixes)?
  • Eleftheria/Chloé: Tuning of EC-Earth 3.2 in climate prediction mode - what is the performance? Has it been reported on the EC-Earth development portal (https://dev.ec-earth.org/issues/243)?
  • PRIMAVERA outclass status and tests: Everyone has reported on the EC-Earth development portal the variables he/she could not output? All the other variables are output properly without crash? New iodef.xml and IFS namelist uploaded to the EC-Earth development portal (https://dev.ec-earth.org/issues/254)? Need for PA (and priority with respect to other tasks) to update the CMORization tools to work on this PRIMAVERA outclass?
  • PRIMAVERA initial conditions all ready and working? EN4 - sea ice HadISST climatology with 3m Arctic and 1m Antarctic - Atm : soil moisture climatology
  • DCPP initial conditions all ready? sea ice initial conditions (François), ocean initial conditions (Chloé), atmospheric initial conditions (Omar, Martin, LP)
  • Computing hours: RES applications to be prepared by 13th September by Eleftheria, Neven and Valentina (http://www.bsc.es/marenostrum-support-services/res); new PRACE project to be prepared in September by Virginie (http://www.prace-ri.eu/prace-project-access/); new K-computer application to be prepared in September by Virginie (http://www.hpci-office.jp/pages/e_h29_boshu); RES applications to be prepared by January by Martin, Chloé, Javier and Etienne (http://www.bsc.es/marenostrum-support-services/res)
  • Status of ECMWF hours usage: Issue with high-resolution version fixed? Standard resolution experiments running at ECMWF? Have we increased our usage? From Paco: Note that almost nothing has been used this year and that all hours should be used before the end of December, using all the SBUs we have allocated will take several months

09/08/2016

  • Domingo/Joan : Autosubmit; from Paco: explain how people can perform an experiment using a specific revision from the SVN to contribute to the discussions in the portal, will this experiment run?
  • From Paco: how can people testing specific EC-Earth revisions from the SVN to contribute to the discussions in the portal produce the standard plots that Jost, Klaus and Uwe produce?
  • Miguel : EC-Earth high resolution status and standard resolution additional features to be tested - testing suite; from Paco: status and plan of the first official release and for the check and development of the templates
  • NEMO bugs - probable update to come for EC-Earth 3.2
  • PRIMAVERA outclass status and tests
  • PRIMAVERA initial conditions : EN4 - sea ice HadISST climatology with 3m Arctic and 1m Antarctic - Atm : soil moisture climatology
  • DCPP initial conditions : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions (Omar, Martin, LP)
  • Computing hours : RES applications to be prepared by 13th September by Eleftheria, Chloe, Neven and Valentina (http://www.bsc.es/marenostrum-support-services/res) ; new PRACE project to be prepared in September by Virginie (http://www.prace-ri.eu/prace-project-access/) ; new K-computer application to be prepared in September by Virginie (http://www.hpci-office.jp/pages/e_h29_boshu) ; RES applications to be prepared by January by Martin, Javier and Etienne (http://www.bsc.es/marenostrum-support-services/res)
  • From Paco: ECMWF has asked me if I'm sure we'll be able to use all the SBUs we still have allocated. These are a lot of hours and ECMWF penalizes projects that are not able to use their hours within the year. Is there a plan (with a backup) to use them?

20/07/2016

  • Domingo : Autosubmit
  • Mario : EC-Earth high resolution version status
  • Miguel : EC-Earth standard resolution additional features to be tested - testing suite
  • Mario : NEMO releases
  • SCRUM meetings auto-Ec-Earth / auto-NEMO
  • PRIMAVERA outclass status and tests
  • PRIMAVERA initial conditions : EN4 - sea ice HadISST climatology with 3m Arctic and 1m Antarctic - Atm : soil moisture climatology
  • Computing hours : PRACE HighResMIP_BSC rejected (12 millions CPU hours - not innovative enough) ; ECMWF special project submitted for 43 millions SBU (Virginie); new RES applications to be prepared by 13th September by Valentina, Javier and Martin (http://www.bsc.es/marenostrum-support-services/res) ; new PRACE project to be prepared by September by Virginie (http://www.prace-ri.eu/prace-project-access/) ; new K-computer application to be prepared by September by Virginie
  • Progress toward CMIP6 preparation : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions (Omar, Martin, LP)

20/06/2016

  • Domingo : Autosubmit
  • Javier : ocean_diagnostics
  • Mario : EC-Earth high resolution version status
  • Mario : EC-Earth standard resolution additional features to be tested
  • Mario : NEMO releases
  • Preparation for PRIMAVERA WP4 contribution (coupled ORCA12L75 - T1279L91), T1279L91 IFS only, waiting for IC files to do a first run (Etienne)
  • ESMValTool
  • Computing hours : ECMWF special project submitted for ?? millions SBU (Virginie); new RES applications to be prepared in September by Valentina, Javier and Martin (http://www.bsc.es/marenostrum-support-services/res) ; new PRACE project to be prepared by September by Virginie (http://www.prace-ri.eu/prace-project-access/) ; new K-computer application to be prepared by September by Virginie;
  • Progress toward CMIP6 preparation : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions (Omar, Martin, LP)

14/06/2016

  • Frequent Autosubmit releases - need for testers - strategy ?
  • Test suite before new release: longer run and deeper analysis?
  • More presence on the development portal
  • Testing and Tuning of EC-Earth 3.2beta low resolution
  • Issue with the standard-resolution EC-Earth 3.1 historical simulation
  • Computing hours : RES application submitted in May by Chloe, Neven, Eleftheria and Omar; new RES applications to be prepared in September by Valentina, Javier and Martin (http://www.bsc.es/marenostrum-support-services/res) ; new K-computer application to be prepared (Virginie); ECMWF special project to be prepared (Virginie) by the 30th June ; INCITE call ( https://proposals.doeleadershipcomputing.org/allocations/calls/incite2017) by the 24th June ? ; new PRACE project to be prepared by September by Virginie and Francois (http://www.prace-ri.eu/prace-project-access/) ?
  • Progress toward CMIP6 preparation : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions done
  • Preparation for PRIMAVERA WP4 contribution (coupled ORCA12L75 - T1279L91), T1279L91 IFS only, waiting for IC files to do a first run (Etienne)
  • ESMValTool

03/06/2016

  • Domingo : Autosubmit
  • Javier : ocean_diagnostics
  • Mario : EC-Earth high resolution version status
  • Mario : EC-Earth standard resolution additional features to be tested
  • Mario : NEMO releases ?
  • Test suite before new release: longer run and deeper analysis?
  • More presence on the development portal
  • Testing and Tuning of EC-Earth 3.2beta low resolution
  • Issue with the standard-resolution EC-Earth 3.1 historical simulation
  • Computing hours : RES application submitted in May by Chloe, Neven, Eleftheria and Omar; new RES applications to be prepared in September by Valentina, Javier and Martin (http://www.bsc.es/marenostrum-support-services/res) ; new K-computer application to be prepared (Virginie); ECMWF special project to be prepared (Virginie) by the 30th June ; INCITE call ( https://proposals.doeleadershipcomputing.org/allocations/calls/incite2017) by the 24th June ? ; new PRACE project to be prepared by September by Virginie and Francois (http://www.prace-ri.eu/prace-project-access/) ?
  • Progress toward CMIP6 preparation : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions done
  • Preparation for PRIMAVERA WP4 contribution (coupled ORCA12L75 - T1279L91), will start T1279L91 IFS only there are some requirements needed to start the runs (IC and configuration files) (Etienne)

19/04/2016

  • Domingo : Autosubmit
  • Mario : EC-Earth and NEMO releases to come - status + reproducibility
  • Testing and Tuning of EC-Earth 3.2beta low resolution
  • High resolution issues
  • HighResMIP - July ?
  • Computing hours : PRACE HighResMIP_BSC submitted for 12.000.000 CPUhours, RES application in May (Chloe, Neven, Eleftheria, Omar), new K-computer application to be prepared (Virginie), ECMWF special project to be prepared (Virginie)
  • Progress toward CMIP6 preparation : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions (Omar, Martin, LP)

16/03/2016

  • Domingo : Autosubmit
  • Mario : EC-Earth and NEMO releases to come - status
  • Computing hours : PRACE granted (5.000.000 CPUhours), RES Francois granted, RES Omar rejected, K-computer rejected, new PRACE application being prepared (Virginie), RES application in May (Chloe, Neven, Eleftheria, Omar), new K-computer application to be prepared (Virginie), ECMWF special project to be prepared (Virginie)
  • Progress toward CMIP6 preparation : sea ice initial conditions (Francois), ocean initial conditions (Chloe), atmospheric initial conditions (Omar, Martin, LP)

23/02/2016

  • Domingo : Autosubmit
  • Mario : EC-Earth and NEMO releases to come - status
  • Testing suite

CMIP6

  • HighResMIP : Javier
  • DCPP : list of responsibles
  • Computing hours : K-computer rejected - possibility to resubmit ; PRACE pending ; RES pending ; ECMWF special project in June

Initial conditions & Nudging files

Experiments

  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts : Neven
  • Neven : m063 for sea ice initial conditions - running
  • Omar : Nudged runs to develop attribution methodology - running
  • Javier : Nudged runs
  • Chloe : Tests for EC-Earth 3.2a - pending for initial conditions
  • Chloe : Land surface initial conditions sensitivity experiments - compilation issues
  • Eleftheria : i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3) waiting for atmospheric initial conditions
  • Martin: volcano experiments - running
  • François, Mario: reproducibility experiments (EC-Earth3.1b) - running
  • Francois : ORCA1 LIM3 tests for sea ice data assimilation
  • Valentina : historical run with EC-Earth3.1 - file access issues
  • Valentina : Test ocean nudging in oce-only mode - pending
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1 - pending
  • Eleftheria : ocean-only forecast 3 members with EC-Earth3.1 - pending numerical instability

09/02/2016

  • Mario : Ec-Earth and NEMO releases to come - status
  • Domingo : Autosubmit

Preparing HighResMIP

  • Marenostrum / ECMWF / K-computer
  • EN4
  • Responsibility for the run : Javier
  • Deadline : April 2017

Preparing DCPP

  • Marenostrum / ECMWF / K-computer
  • ORAS4 / ERA / Francois' reanalysis
  • Responsibility for the run

Initial conditions & Nudging files

Experiments

  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts : Neven
  • Volunteer to run a decadal experiment with EC-Earth3.1 : Omar
  • Neven : m063 for sea ice initial conditions - running
  • Omar : Nudged runs to develop attribution methodology - running
  • Javier : Nudged runs
  • Chloe : Tests for EC-Earth 3.2a - pending for initial conditions
  • Chloe : Land surface initial conditions sensitivity experiments - compilation issues
  • Eleftheria : i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3) waiting for atmospheric initial conditions
  • Martin: volcano experiments - running
  • François, Mario: reproducibility experiments (EC-Earth3.1b) - running
  • Francois : ORCA1 LIM3 tests for sea ice data assimilation
  • Valentina : historical run with EC-Earth3.1 - file access issues
  • Valentina : Test ocean nudging in oce-only mode - pending
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1 - pending
  • Eleftheria : ocean-only forecast 3 members with EC-Earth3.1 - pending numerical instability

11/01/2016

  • Testing suite before EC-Earth release - references : coupled = i00k ; atm-only = a02d ; oce-only = a02b
  • Storage space at marenostrum : transfer job during the experiment - default behaviour in Autosubmit
  • Permissions for IC and other shared files

Project

  • RES submission - 12th January 2015 (Omar, Francois)
  • next RES submission in May (Neven, Eleftheria, Chloe)
  • next RES submission in September (Valentina, Martin, Javier)

Initial conditions & Nudging files

  • Martin, Omar, LP : atmospheric initial conditions ?

Experiments

  • Neven : tuning experiments on marenostrum
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)
  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts
  • Volunteer to run a decadal experiment with EC-Earth3.1
  • Valentina : historical run with EC-Earth3.1
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1
  • Eleftheria to run 4 sensitivity experiments ocean-only forecast 3 members, atm-only forecast 3 members, coupled forecast 10 members, coupled historical 3 members, all with EC-Earth3.1

11/12/2015

Project

  • RES submission - 12th January 2015
  • RES report - 21st December 2015

Models

  • auto-ecearth status
  • aut-nemo status

Why do we still use auto-nemo for version 3.6?

  • Auto-nemo and NEMO stand-alone in auto-ecearth do not have excactly the same bathymetries and coast-lines
  • NEMO stand-alone with fixed forcing could be run with different namelist using auto-earth. This would fix problem of creating initial conditions with NEMO stand-alone which are used to run coupled experiments (e.g. in CMUG Tasks)

Initial conditions & Nudging files

  • Eleftheria, François, PA: download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • Eleftheria: ocean nudging files available ? (ORAS4, GLORYS, GLOSEA5, ORAP5)
  • Eleftheria, Chloe : ocean initial conditions ?
  • Martin, Omar, LP : atmospheric initial conditions ?
  • Neven, Francois : sea ice initial conditions ?

Other scientific issues

  • François, Martin, Omar: reproducibility
  • Public/private GIT repositories ?

Experiments

  • Neven : tuning experiments on marenostrum
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)
  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts
  • Volunteer to run a decadal experiment with EC-Earth3.1
  • Valentina : historical run with EC-Earth3.1
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1
  • Eleftheria to run 4 sensitivity experiments ocean-only forecast 3 members, atm-only forecast 3 members, coupled forecast 10 members, coupled historical 3 members, all with EC-Earth3.1

30/11/2015

Project reports

  • RES - 21st December 2015

Summary : On the best usage of our computing hours

Immediate measures
  • Register any new experiment to the new Experiment project - document the experiment there - bug/failures to be reported in the Auto-Earth/Autosumit/Oceanpp/Atmpp/s2dverification projects depending on the bug - who monitor the experiment should be registered on the Experiment project - progress status can be summarized in Experiment project when handing an experiment to someone else
  • Set permissions so that anyone can monitor this experiment
  • Set number of retrials to 0 when starting a new experiment
  • Adjust optimally the wall-clock time - wiki page setup by Eleftheria for reference
  • If hitting the wall-clock limit, check when was the last information written by the model
  • Monitor regularly the experiment - check the plot directory : status of jobs + job statistics + scientific validation
  • Kill an experiment which is unusually expensive or take an unusually long time to run
Short-term measures
  • New behaviour of retrials in autosubmit - distinctive retrial numbers for different types of jobs - default to 0
  • New development in Autosubmit : Inputs = 1. Period of time, 2. Job type, Outputs = 1. Number of jobs submitted, 2. Number of jobs running, 3. Number of jobs failed, 4. Number of jobs completed, 5. Total queuing time, 6. Total queuing time of failed jobs, 7. CPU time consumed, 8. CPU time consumed by failed jobs, 9. Reference of what is expected to be consumed, 10. Percentage of computing resources used.
  • New development in Autosubmit : automatic killing of an experiment which is unusually expensive or take unusually long to run
Long-term measures
  • Improve model stability - work on a few startdates/member examples from Omar's experiment where a large number of failures occurred

- Chloe to provide a selection of startdates/members

Priority on experiments

Initial conditions & Nudging files

  • Eleftheria, François, PA: download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • Eleftheria: ocean nudging files available ? (ORAS4, GLORYS, GLOSEA5, ORAP5)
  • Eleftheria, Chloe : ocean initial conditions ?
  • Martin, Omar, LP : atmospheric initial conditions ?
  • Neven, Francois : sea ice initial conditions ?

Other scientific issues

  • François, Martin, Omar: reproducibility
  • Public/private GIT repositories ?

Why do we still use auto-nemo for version 3.6?

  • Auto-nemo and NEMO stand-alone in auto-ecearth do not have excactly the same bathymetries and coast-lines
  • NEMO stand-alone with fixed forcing could be run with different namelist using auto-earth. This would fix problem of creating initial conditions with NEMO stand-alone which are used to run coupled experiments (e.g. in CMUG Tasks)

Experiments

  • Neven : tuning experiments on marenostrum
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)
  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts
  • Volunteer to run a decadal experiment with EC-Earth3.1
  • Valentina : historical run with EC-Earth3.1
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1
  • Eleftheria to run 4 sensitivity experiments ocean-only forecast 3 members, atm-only forecast 3 members, coupled forecast 10 members, coupled historical 3 members, all with EC-Earth3.1

12/11/2015

 * Virginie from skype

On the best usage of our computing hours

  • Set number of retrials to 0 + change behaviour in autosubmit
  • Adjust optimally the wall-clock time
  • Kill an experiment which is unusually expensive or take an unusually long time to run - automatic ?
  • If hitting the wall-clock limit, check when was the last information written by the model
  • Handing on the monitoring of experiments - Generic user to launch experiments ?
  • New development in Autosubmit : Inputs = 1. Period of time, 2. Job type, Outputs = 1. Number of jobs submitted, 2. Number of jobs running, 3. Number of jobs failed, 4. Number of jobs completed, 5. Total queuing time, 6. Total queuing time of failed jobs, 7. CPU time consumed, 8. CPU time consumed by failed jobs, 9. Reference of what is expected to be consumed, 10. Percentage of computing resources used.
  • Improve the model stability ? Approach ?

Available resources

  • Marenostrum
  • Ithaca
  • ECMWF

Initial conditions & Nudging files

  • Valentina & Omar : Initialization strategy for EC-Earth
  • Eleftheria, François, PA: download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • Eleftheria: ocean nudging files available ? (ORAS4, GLORYS, GLOSEA5, ORAP5)
  • Eleftheria, Chloe : ocean initial conditions ?
  • Martin, Omar, LP : atmospheric initial conditions ?
  • Neven, Francois : sea ice initial conditions ?

Other scientific issues

  • François, Martin, Omar: reproducibility

Experiments

  • Neven : tuning experiments on marenostrum
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)
  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts
  • Volunteer to run a decadal experiment with EC-Earth3.1
  • Valentina : historical run with EC-Earth3.1
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1
  • Eleftheria to run 4 sensitivity experiments ocean-only forecast 3 members, atm-only forecast 3 members, coupled forecast 10 members, coupled historical 3 members, all with EC-Earth3.1

26/10/2015

  • Anyone on skype?

Technical issues

  • Mingu : sharing our developments with EC-Earth consortium + run_script vs sim ?
  • Mingu & Javier : Progress with autosubmit ?
  • Asif : Status of autonemo and autoecearth ?
  • PA & Nico : Status of reformatting of model outputs and direct reading by Load ?
  • PA : OpenDaP
  • Asif : EC-Earth3.2
  • Asif : OpenIFS ?

Platforms

  • Oriol : ithaca
  • Asif & Eleftheria : cca - Mingu to illustrate the monitoring of computing ressources usage
  • Marenostrum usage

Initial conditions & Nudging files

  • Valentina & Omar : Initialization strategy for EC-Earth
  • Eleftheria, François, PA: download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • Eleftheria: ocean nudging files available ? (ORAS4, GLORYS, GLOSEA5, ORAP5)
  • Eleftheria, Chloe : ocean initial conditions ?
  • Martin, Omar, LP : atmospheric initial conditions ?
  • Neven, Francois : sea ice initial conditions ?

Other scientific issues

  • François, Martin, Omar: reproducibility

Experiments

  • Neven : tuning experiments on marenostrum
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)
  • Volunteer to run high resolution experiments ORCA025L75 with ORAP5 restarts
  • Volunteer to run a decadal experiment with EC-Earth3.1
  • Valentina : historical run with EC-Earth3.1
  • Valentina : Ocean nudged and Atm+Oce nudged experiments with EC-Earth3.1
  • Eleftheria to run 4 sensitivity experiments ocean-only forecast 3 members, atm-only forecast 3 members, coupled forecast 10 members, coupled historical 3 members, all with EC-Earth3.1

Resources and platforms

  • Project reports to be stored at /esnas/OLD/cfu/pub/reports/
  • Next RES report : ~5 page with two or three figures (LP)
  • RES (Jul-Oct 2015): 1.5 Mhours (Paco) used 38% until 28 Sep and and 0.3 Mhours (Neven) used 55% until 9 Sep; contact support if no more resources will be used; next proposal submitted by François proposing the SPECS and EUCLEIA experiments, no submission by Neven
  • ECMWF: resources 2015 originally 37 MSBUs, 20 Tb, all used, 15 MSBUs additional in August; possibility of using additional resources from the EC-Earth special project and AEMET resources
  • Japanese call ( http://www.hpci-office.jp/folders/e_invite ) proposal to be written by Virginie. Deadline : 2nd November
  • INCITE programme to use the Argonne BlueGene director's discretionary proposal granted 300 khours in Mira; next INCITE call should open in April 2016
  • PRACE-Tier 0 12th call <http://www.prace-ri.eu/prace-project-access/>, proposal to be written by Virginie
  • PRACE Tier 1: http://www.prace-ri.eu/deci-12-call proposal to be written by Virginie

28/09/2015

  • Anyone on skype?

New issues

  • Mingu: status of the strategy and tests for sending the modifications done at IC3 in the last two years to the EC-Earth3 development portal and to make the sim task as close as possible to the run script; outcome of the SMHI visit
  • DECI-13 submitted on 21 September, http://www.prace-ri.eu/deci-13-call by Chloé
  • Summary of RES user meeting 23 September
  • Reporting about autonemo and autoecearth
  • Reporting about the coupling of OpenIFS in EC-Earth

Pending issues

  • Decided to allow autosubmit to run python and R jobs
  • Javier, Mingu: update on autosubmit changes, including progress with new workflow definition solutions
  • Chloé: summarize the last SCRUM meetings (autosubmit and automodels)
  • Asif, Oriol T., Miguel, Xavi Yepes: summary of periodic meetings (at least once a week)
  • PA, Nico, Chloé: description of the person to hire to work on the reorganization of the model output (atmosphere and ocean), including modifications to the templates and avoiding the use of tar and gzip (hence removing MMO and MMA files); progress with the definition of the position following the tests of reading cmorised files through THREDDS
  • Eleftheria, François, Neven, Chloé, PA: update on download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • PA: update on OPeNDAP server and the ESG node with a presentation at the CFU talks and a description of the activity with a schedule on the wiki
  • Omar: EC-Earth3 initial conditions with coupled nudged atmosphere/ocean; need to use EMoP diagnostic online; give presentation at the CFU talks
  • Oriol: latest news about ithaca (schedule of access, incidences, nodes failed, etc); guarantee will be renewed on September the 15th
  • Asif: performance on cca
  • Eleftheria: GLOSEA5 experiments, including the analysis (comparison with ORA-S4, ORA-P5 and GLORYS with Neven, François, Virginie, with special attention to sea ice) of the monthly mean output; update on the optimisation of the python code for restart and monthly mean output interpolation; interpolate sea-ice restarts and monthly means for ocean nudging; involve Matthieu Chevallier whenever possible
  • François, Martin, Omar: update on the reproducibility testing of EC-Earth on any new platforms, experiments running on three platforms, Ithaca, ECMWF-CCA and MareNostrum3; check the EC-Earth forum and the wiki page

Experiments

  • Neven: tuning NEMO to generate ORCA025 sea-ice restarts, first NEMO3.3.1, then NEMO3.6
  • Paco: atmospheric initial conditions for 201411, 201111, 201208, 201205
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04r & m04o (T255-ORCA1-LIM2) done; m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)
  • Is anyone planning to run high resolution experiments ORCA025L75 with ORAP5 restarts ?
  • Is anyone (and when) planning to run a decadal experiment with EC-Earth3.1?
  • Historical run with EC-Earth3.1?

Resources and platforms

  • Prepare all end of project reports; make files available in /cfu/pub, for the RES aim at writing a ~5 page report with two or three figures per project (LP)
  • Ithaca: experiments need to be organized among CFU members to keep the machine as busy as possible; make sure the backup of /share and /home for CFU users is done
  • RES (Jul-Oct 2015): 1.5 Mhours (Paco) used 38% until 28 Sep and and 0.3 Mhours (Neven) used 55% until 9 Sep; contact support if no more resources will be used; next proposal submitted by François proposing the SPECS and EUCLEIA experiments, no submission by Neven
  • ECMWF: resources 2015 originally 37 MSBUs, 20 Tb, all used, 15 MSBUs additional in August; possibility of using additional resources from the EC-Earth special project and AEMET resources
  • PRACE-Tier 0 11th call <http://www.prace-ri.eu/prace-project-access/>, proposal submitted by Virginie with data pilot call included not granted; next call in September
  • INCITE programme to use the Argonne BlueGene director's discretionary proposal granted 300 khours in Mira; next INCITE call should open in April 2016

09/09/2015

  • Anyone on skype?

New issues

  • DECI-13 open, deadline 21 September, http://www.prace-ri.eu/deci-13-call/
  • RES user meeting 23 September
  • Asif: different grids used for IFS surface and pressure-level data; from LP “If it's actually cdo that is used to convert from spectral to gaussian, one can use either sp2gp or sp2gpl. I suspect we probably use the first one now. The second option should return a lower Gaussian grid resolution”
  • Chunks missing after transfer: Is this still happening? Is there a checksum mechanism in place when transferring the files from the HPC to the nas?

Pending issues

  • Chloé: summarize the last SCRUM meetings (autosubmit and automodels)
  • Asif, Oriol T., Miguel, Xavi Yepes: summary of periodic meetings (at least once a week)
  • PA, Nico, Chloé: description of the person to hire to work on the reorganization of the model output (atmosphere and ocean), including modifications to the templates and avoiding the use of tar and gzip (hence removing MMO and MMA files); progress with the definition of the position following the tests of reading cmorised files through THREDDS
  • Eleftheria, François, Neven, Chloé, PA: update on download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • PA: update on OPeNDAP server and the ESG node with a presentation at the CFU talks and a description of the activity with a schedule on the wiki
  • Omar: EC-Earth3 initial conditions with coupled nudged atmosphere/ocean; need to use EMoP diagnostic online; give presentation at the CFU talks
  • Oriol: latest news about ithaca (schedule of access, incidences, nodes failed, etc)
    • 39/48 nodes working
    • Guarantee will be renewed on September the 15th
  • All: look at the CMIP6 wiki page and provide comments
  • Javier, Mingu: update on autosubmit changes, including progress with new workflow definition solutions
  • Paco, Asif: performance problems on cca; update on request for additional resources submitted to ECMWF
  • Mingu, Javier, Asif: status of the strategy and tests for sending the modifications done at IC3 in the last two years to the EC-Earth3 development portal and to make the sim task as close as possible to the run script; things will be discussed thoroughly at a meeting with Martin and Uwe
  • Mingu: use of localpost, introduction on templates; make sure how to use compatible output selection; make sure that by default the diagnostics are automatically called several times through the experiment to monitor what the experiment is doing
  • Eleftheria: GLOSEA5 experiments, including the analysis (comparison with ORA-S4, ORA-P5 and GLORYS with Neven, François, Virginie, with special attention to sea ice) of the monthly mean output; update on the optimisation of the python code for restart and monthly mean output interpolation; interpolate sea-ice restarts and monthly means for ocean nudging; involve Matthieu Chevallier whenever possible
  • François, Martin, Omar: update on the reproducibility testing of EC-Earth on any new platforms, experiments running on three platforms, Ithaca, ECMWF-CCA and MareNostrum3; check the EC-Earth forum and the wiki page
  • Omar, François, Asif: adaptation of EMoP to the templates to use the appropriate output; perform the same with ECMean; make sure how to use compatible output selection; make sure that by default the diagnostics are called several times through the experiment to monitor what the experiment is doing; when and by whom will the limitations to ECMean be sorted out; the EMoP documentation on the wiki is not very explicit and has problems with the formatting in dokuwiki, an additional effort is needed

file:///cfu/diagnostics/emop_obellprat/a01q/20000101/fc0/time_series/a01q/atmosphere/index.html file:///cfu/diagnostics/emop_obellprat/a01q/20000101/fc0/time_series/a01q/ocean/index.html

Experiments

  • Neven: tuning NEMO to generate ORCA025 sea-ice restarts, first NEMO3.3.1, then NEMO3.6
  • Paco: atmospheric initial conditions for 201411, 201111, 201208, 201205
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04r & m04o (T255-ORCA1-LIM2) done; m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)

Resources and platforms

  • Prepare all end of project reports; make files available in /cfu/pub, for the RES aim at writing a ~5 page report with two or three figures per project (LP)
  • Ithaca: experiments need to be organized among CFU members to keep the machine as busy as possible; make sure the backup of /share and /home for CFU users is done
  • RES (Jul-Oct 2015): 1.5 Mhours (Paco) used 37% until 9 Sep and and 0.3 Mhours (Neven) used 55% until 9 Sep; next proposal to be submitted by François proposing the SPECS and EUCLEIA experiments, Neven's plans?; next application deadline 15 September 2015, 11:00 - CET
  • ECMWF: resources 2015 originally 37 MSBUs, 20 Tb, all used, 15 MSBUs additional in August; possibility of using additional resources from the EC-Earth special project and AEMET resources
  • PRACE-Tier 0 11th call <http://www.prace-ri.eu/prace-project-access/>, proposal submitted by Virginie with data pilot call included not granted; next call in September
  • INCITE programme to use the Argonne BlueGene director's discretionary proposal granted 300 khours in Mira; next INCITE call should open in April 2016
  • PRACE Tier 1: volunteers to prepare a proposal for a call foreseen in Feb to start the runs in Sept, check regularly with Jorge Rodríguez (BSC PRACE contact) http://www.prace-ri.eu/deci-12-call

13/08/2015

  • Anyone on skype?

New issues

  • RES user meeting 23 September
  • Different grids used for IFS surface and pressure-level data
  • Chloé: summarize the last SCRUM meetings (autosubmit and automodels)
  • All: load of ithaca; the planned use of ithaca is Martin (maybe still running volcano experiments), Eleftheria with i05m, Neven/François planning some sea-ice experiments to start in late July, Asif planning reproducibility experiments; however, the machine is EMPTY too often, which is unacceptable; what has been the response to my email to the CFU in July? has there been an exchange of messages among those interested to prioritise the simulations?
  • Asif, Oriol T., Miguel: periodic meetings (once a week)
  • Asif, Mingu: transfer issues from this list to the autosubmit and automodels tickets
  • PA, Nico, Chloé: need to hire a person to work on the reorganization of the output, including modification of the templates (see entry below describing the needs); progress with the definition of the position
  • Eleftheria, François, Neven, Chloé, PA: update on download of ocean reanalyses (GloSea5, ORA-S4, ORA-P5 and GLORYS2v3)
  • PA: update on OPeNDAP server and the ESG node with a presentation at the CFU talks and a description of the activity with a schedule on the wiki
  • Omar: EC-Earth3 initial conditions with coupled nudged atmosphere/ocean; need to use EMoP diagnostic online; give presentation at the CFU talks

Pending issues

  • Oriol: latest news about ithaca (schedule of access, incidences, nodes failed, etc)
    • 39/48 nodes working
    • Guarantee will be renewed on September the 15th
  • All: look at the CMIP6 wiki page and provide comments; the documentation for DECK, HiresMIP and ScenarioMIP is still missing
  • Javier, Mingu: update on autosubmit changes, including progress with new workflow definition solutions
  • Asif, Paco: performance problems on cca; update on request for additional resources submitted to ECMWF
  • Oriol, Paco: solution for external people to work with autosubmit and fat nodes at the BSC
  • Mingu, Asif (from the automodel scrum): status of sending the modifications done at IC3 in the last two years to the EC-Earth3 development portal and to make the sim task as close as possible to the run script; things need to be discussed thoroughly in a meeting with Martin and Uwe
  • François, Martin, Omar: update on the reproducibility testing of EC-Earth on any new platforms, experiments running on three platforms, Ithaca, ECMWF-CCA and MareNostrum3; check the EC-Earth forum <https://dev.ec-earth.org/boards/10/topics/162?page=2>; check wiki page
  • PA, Asif, Nico, Chloé: plan to reorganise the model output; the objective is to read it directly with Load() (i.e. to skip atmospheric diagnostics) using the CMOR files as main reference (avoid tar and gzip files); need to keep the MMA files until all variables are cmorised as they contain many more variables than those that are cmorised; diagnostics from MMO files to be performed online; PA is currently converting the fields to CMOR (offline), Nico needs to check if he can read the examples (with one file per chunk to avoid having to wait until the end of the experiment to concatenate them, which has problems) with Load(); use OPeNDAP to access the data instead of concatenating the individual chunk files; nccf_atm_monthly will be replaced later by its equivalent based on CMOR rather than MMA when PA has written it
  • Martin, Virginie, Asif: use of localpost, introduction on templates; make sure how to use compatible output selection; make sure that by default the diagnostics are called several times through the experiment to monitor what the experiment is doing
  • Omar, François, Asif: adaptation of EMoP (https://earth.bsc.es/wiki/doku.php?id=models:emo_p&s[]=emop) to the templates to use the appropriate output; perform the same with ECMean (https://earth.bsc.es/wiki/doku.php?id=models:ecmean&s[]=ecmean); make sure how to use compatible output selection; make sure that by default the diagnostics are called several times through the experiment to monitor what the experiment is doing
  • Eleftheria: GLOSEA5 experiments, including the analysis (comparison with ORA-S4, ORA-P5 and GLORYS with Neven, François, Virginie, with special attention to sea ice) of the monthly mean output; update on the optimisation of the python code for restart and monthly mean output interpolation; interpolate sea-ice restarts and monthly means for ocean nudging; involve Matthieu Chevallier whenever possible

Experiments

  • Neven: tuning NEMO to generate ORCA025 sea-ice restarts, first NEMO3.3.1, then NEMO3.6
  • Paco: atmospheric initial conditions for 201411, 201111, 201208, 201205
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3)
  • Omar: series of land surface initialization experiments with EC-Earth-v3.1b: m04r & m04o (T255-ORCA1-LIM2) done; m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin: volcano experiments
  • François, Asif: reproducibility experiments (EC-Earth3.1b)

Resources and platforms

  • Prepare all end of project reports; make files available in /cfu/pub, for the RES aim at writing a ~5 page report with two or three figures per project (LP)
  • Ithaca: experiments need to be organized among CFU members to keep the machine as busy as possible; make sure the backup of /share and /home for CFU users is done
  • RES: 1.5 Mhours (Paco) and 0.3 (Neven) for Jul-Oct 2015, used 27% until 13 Aug (below the expected 35%); next proposal to be submitted by François proposing the SPECS and EUCLEIA experiments, Neven's plans?; next application deadline 15 September 2015
  • ECMWF: resources 2015 37 MSBUs, 20 Tb, all used; additional 15.4 million SBUs requested; possibility of using additional resources from the EC-Earth special project and AEMET resources
  • PRACE-Tier 0 11th call <http://www.prace-ri.eu/prace-project-access/>, proposal submitted by Virginie with data pilot call included not granted; next call in September
  • INCITE programme to use the Argonne BlueGene director's discretionary proposal granted 300 khours in Mira; next INCITE call should open in April 2016
  • PRACE Tier 1: volunteers to prepare a proposal for a call foreseen in Feb to start the runs in Sept, check regularly with Jorge Rodríguez (BSC PRACE contact) http://www.prace-ri.eu/deci-12-call

09/07/2015

New issues

  • Performance problems on cca
  • Solutions for external people to work with autosubmit and the serial machines when we are at the BSC
  • Chloé: summarize the last SCRUM meetings (autosubmit and automodels)
  • Oriol: explain the situation of ithaca (schedule of access, incidences, etc)
  • Martin: CMIP6 contribution
  • Javier, Mingu: update on autosubmit changes, including new workflow definition solutions

Pending issues

  • Asif, Mingu: transfer issues from the SCRUM list to the experiment meeting agenda in the wiki
  • Is the version of autoecearth and autonemo with pluggins (in autosubmit3) released? At the last experiment meeting, 8 experiments were still to be done for autoecearth3. Any news? Have people still worked with autosubmit2.5?
  • Mingu, Asif: status of sending the modifications done at IC3 in the last two years to the EC-Earth3 development portal and to make the sim task as close as possible to the run script; merge request #175 done in r2208 but issue not closed yet because the implemented solution is based in $config variable, although Uwe F. says that the $config variable feature has been introduced quite recently, so it's use is not really established yet; things need to be discussed thoroughly in a meeting with Martin and Uwe
  • Asif: status of EC-Earth3.1d(e) to produce a clean version for the plugin development (plugins coordinated by Asif, who maintains the plugins for auto-ecearth2, auto-ecearth3 and auto-nemo)
  • François, Martin, Omar: update on the reproducibility testing of EC-Earth on any new platforms, experiments running on three platforms, Ithaca, ECMWF-CCA and MareNostrum3, including with different number of processors on the same platform and reproducing the same experiment with the same conditions to see if gives the same results, Asif to inform about this one; check the EC-Earth forum <https://dev.ec-earth.org/boards/10/topics/162?page=2>; a wiki page is now available <http://ic3.cat/wikicfu/index.php/Reproducibility> but needs to be transferred to the new wiki
  • Pierre-Antoine, Asif, Nico, Chloé: in the plan to reorganise the model output the objective is to read it directly with Load() (to skip atmospheric diagnostics), use the CMOR files as main reference, need to keep the MMA files until all variables are cmorised as they contain many more variables than those that are cmorised, diagnostics from MMO files to be performed online, include basic plotting to monitor the experiments; PA is currently converting the fields to CMOR (offline) but keeping the MMA files, Nico needs to check if he can read the examples (with one file per chunk to avoid having to wait until the end of the experiment to concatenate them, which has problems) with Load(); use OPeNDAP to access the data instead of concatenating the individual chunk files
  • Martin, Virginie: use of localpost; nccf_atm_monthly will be replaced later by its equivalent based on CMOR rather than MMA when PA has written it; plot is supposed to run on stargate also but it is not written yet
  • Omar, François, Asif: EMoP (http://ic3.cat/wikicfu/index.php/EMoP), adaptation of the templates to get the appropriate output, perform the same with ECMean, make sure that by default the user doesn't call the diagnostics at the end of the experiment, but several times through the experiment to monitor what is being done. Merge information to new Wiki.
  • Asif: latest news about EC-Earth3.2 (https://dev.ec-earth.org/projects/ecearth3/roadmap)
  • Eleftheria: GLOSEA5 experiments, including the analysis (comparison with ORA-S4, ORA-P5 and GLORYS with Neven, François, Virginie, with special attention to sea ice) of the monthly mean output, restart interpolation (more machines needed), update on the optimisation of the python code: interpolation of restarts still ongoing, optimization from Francesco
  • Pierre-Antoine: update on OPeNDAP/ESG node; link to the OPeNDAP activities and the choice of a database strategy at BSC-ES; add a schedule to the plan
  • Omar, Virginie: EC-Earth3 initial conditions with coupled nudged atmosphere/ocean. Update visualization and give presentation at a internal talk (LP).
  • François, Neven, Eleftheria, Chloé, Virginie: download and assess ORAP5; download GLORYS2v3 (longer reanalysis than GLORYS2v1 but worse sea ice), needs to be tested and compared with GLOSEA5, ORAP5

Experiments

  • Virginie m04q (EC-Earth-v3.1b and ORCA025L75-LIM3): long forced ocean run for tuning; far too few sea ice stopped → new experiment to be created by Neven with different namelist parameters (pstar and hiccrit); need autosubmit3 and pluggins to develop further the templates and reduce the scratch space usage
  • Virginie i01j: issue with 20131101 start date for which random chunks had disappeared from cfunas; unable to narrow down the origin of this issue; has rerun; still no atmospheric initial conditions for 201411, 201111, 201208, 201205
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3): it is equivalent to i00k but with EC-Earth3.1-LIM3 in its low resolution configuration; at first 4 start dates and 5-year length have been submitted but the objective is to extend it to start dates every 1st November from 1960 to 2014 with 10-year forecast length; start dates every 1st February, 1st May and 1st August with 1-year forecast length
  • Omar's series of land surface initialization experiments with EC-Earth-v3.1b: m04r & m04o (T255-ORCA1-LIM2) done; m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin's volcano experiments
  • François' reproducibility experiments (EC-Earth3.1b)
  • Two seasonal hindcasts with new nudged initial conditions (1993-2009, T255-ORCA1): does the forecast improve with initial states from coupled assimilation?

Resources and platforms

  • Prepare all end of project reports; make files available in /cfu/pub, for the RES aim at writing a ~5 page report with two or three figures per project (LP)
  • Ithaca: experiments need to be organized among CFU members to keep the machine as busy as possible; backup of /share and /home for CFU users
  • RES: previous projects, Paco 106% used, Neven ?; two proposals granted, 1.5 Mhours and 0.3 for Jul-Oct 2015, used 1% until 9 Jul; next proposal to be submitted by François proposing the SPECS and EUCLEIA experiments; next application deadline mid September 2015
  • ECMWF: resources 2015 37 MSBUs, 20 Tb, all used; possibility of using additional resources from the EC-Earth special project and AEMET resources; request more SBUs from user support for 2015
  • PRACE-Tier 0 11th call <http://www.prace-ri.eu/prace-project-access/>, proposal submitted by Virginie with data pilot call included
  • INCITE programme to use the Argonne (Omar) BlueGene https://www.olcf.ornl.gov/2014/04/17/2015-incite-call-for-proposals/ director's discretionary proposal submitted
  • PRACE Tier 1: volunteers to prepare a proposal for a call foreseen in Feb to start the runs in Sept, check regularly with Jorge Rodríguez (BSC PRACE contact) http://www.prace-ri.eu/deci-12-call

27/04/2015

  • François?

New issues

  • Asif, Mingu: transfer issues from the SCRUM list to the experiment meeting agenda in the wiki
  • From Virginie: Why isn't the new version of ecearth/nemo with the pluggin released yet? This was supposed to be done in a few days only, which have now turned to be months. Many experiments are waiting for this version to be started, with the result that all new experiments are on hold and the machines underused. This can not last. Response from Asif: layer-0 testing (with autosubmit3 develop branch made available by Mingu in mid Jan) a week ago and afterwards started with adopting the latest autosubmit3 develop branch/changes; faced a couple of issues (https://gitlab.cfu.local/cfu/autosubmit/issues/23 a few of them have already been sorted last week while a couple of them still left to sort). Moreover under layer-1 or layer-2 testing; specs2cmor and atm&oce nudging (together first thereafter separately as Omar suggested) is on the way along with couple of technical issues (I have discussed with PA and Omar how to sort them respectively). In total; there are 13 test experiments need to be run as per reference to the designed testing procedure for releasing auto-ecearth3's new tag but still I have opened/created/tested 5 experiments and remaining 8 will be opened this week. Response from Virginie : What is the solution for CFUers to start working again ? Shall we post-pone the official release and meanwhile CFUers start their experiments and developments with autosubmit 2.5 ? And when your tests are done, Asif upgrades with the new developments ?
  • Chloé: summarize the last SCRUM meeting
  • Oriol: explain the priorities and the maintenance activity on ithaca
  • CMIP6 contribution and EC-Earth meeting
  • Mingu, Asif: status of sending the modifications done at IC3 in the last two years to the EC-Earth3 development portal and to make the sim task as close as possible to the run script; merge request #175 done in r2208 but issue not closed yet because the implemented solution is based in $config variable, although Uwe F. says that the $config variable feature has been introduced quite recently, so it's use is not really established yet; there are still some inconsistencies that need to be sorted out; for the plan for the run script, the idea is to finally use the simple script they provide and transform our .sim template on a wrapper on top of it; both things need to be discussed thoroughly in a meeting with Martin and Uwe.
  • Javier, Mingu: inform on 1) agreement on new experiment naming, 2) how to enrich the database describing the experiments, 3) role of cylc (there is an IS-ENES2 deliverable this year that requires dealing with the issue)

Pending issues

  • Asif: status of EC-Earth3.1d(e) to produce a clean version for the plugin development (plugins coordinated by Asif, who maintains the plugins for auto-ecearth2, auto-ecearth3 and auto-nemo)
  • François, Martin, Omar: update on the reproducibility testing of EC-Earth on any new platforms, experiments running on three platforms, Ithaca, ECMWF-CCA and MareNostrum3, including with different number of processors on the same platform and reproducing the same experiment with the same conditions to see if gives the same results, Asif to inform about this one; check the EC-Earth forum <https://dev.ec-earth.org/boards/10/topics/162?page=2>. A wiki page is now available <http://ic3.cat/wikicfu/index.php/Reproducibility>
  • François, Omar, Asif: problems with sea-ice output, not written some times (randomly, although linked to the number of processors in LIM); sea ice is written in the HR Runs (m04p, m04s) so it seems to happen only with the former version of EC-Earth (3.0.1); the time-step in these runs was per default 1200s, but in some chunks 600s, yet ice seems to have been written in both cases
  • Pierre-Antoine, Asif, Nico, Chloé: in the plan to reorganise the model output the objective is to read it directly with Load() (to skip atmospheric diagnostics), use the CMOR files as main reference, need to keep the MMA files until all variables are cmorised as they contain many more variables than those that are cmorised, diagnostics from MMO files to be performed online, include basic plotting to monitor the experiments; PA is currently converting the fields to CMOR (offline) but keeping the MMA files, Nico needs to check if he can read the examples (with one file per chunk to avoid having to wait until the end of the experiment to concatenate them, which has problems) with Load(), consider the use of OPeNDAP to access the data instead of concatenating the individual chunk files
  • Virginie: development of localpost and plot jobs to be added after local transfer - localpost runs ocean_pp + nccf_atm_monthly on stargate - debugging in progress. nccf_atm_monthly will be replaced later by its equivalent based on CMOR rather than MMA when PA has written it. plot is supposed to run on stargate also but it is not written yet.
  • Omar, François, Asif: EMoP (http://ic3.cat/wikicfu/index.php/EMoP), adaptation of the templates to get the appropriate output, perform the same with ECMean, make sure that by default the user doesn't call the diagnostics at the end of the experiment, but several times through the experiment to monitor what is being done
  • Asif: latest news about EC-Earth3.2beta (https://dev.ec-earth.org/projects/ecearth3/roadmap)
  • Eleftheria: update on EC-Earth tuning at ECMWF; don't wait until the meeting to update everyone on the tuning effort (Eleftheria: there is nothing done yet, waiting for Torben and Jost to coordinate the tuning and update about the experiments to be done)
  • Eleftheria: GLOSEA5 experiments, including the analysis (comparison with ORA-S4, ORA-P5 and GLORYS with Neven, François, Virginie, with special attention to sea ice; meeting planned for 7 May) of the monthly mean output, restart interpolation (more machines needed), update on the optimisation of the python code: interpolation of restarts still ongoing, optimization from Francesco has a bug, test experiments working on both marenostrum and ECMWF, waiting for tagged version of autosubmit to run experiment in ECMWF
  • Pierre-Antoine: update on OPeNDAP/ESG node; link to the OPeNDAP activities and the choice of a database strategy at BSC-ES; add a schedule to the plan
  • Omar, Virginie: EC-Earth3 initial conditions with coupled nudged atmosphere/ocean <file:///cfu/scratch/lpcaron/EC_EARTH/i05k/outputs/diagnoce/i05k/index.html>
  • Download and assess ORAP5: meeting on 7 May with Francois, Neven, Eleftheria, Chloé, Virginie
  • Download GLORYS2v3 : longer reanalysis than GLORYS2v1 but worse sea ice, needs to be tested and compared with GLOSEA5, ORAP5

Experiments

  • Virginie m04q (EC-Earth-v3.1b and ORCA025L75-LIM3): long forced ocean run for tuning; far too few sea ice stopped → new experiment to be created by Neven with different namelist parmeters (pstar and hiccrit); need autosubmit3 and pluggins to develop further the templates and reduce the scratch space usage
  • Virginie i01j: issue with 20131101 start date for which random chunks had disappeared from cfunas; unable to narrow down the origin of this issue - has rerun - Still no atmospheric initial conditions for 201411, 201111, 201208, 201205
  • Eleftheria i05m (EC-Earth-v3.1b and T255-ORCA1-LIM3): it is equivalent to i00k but with EC-Earth3.1-LIM3 in its low resolution configuration; at first 4 start dates and 5-year length have been submitted but the objective is to extend it to start dates every 1st November from 1960 to 2014 with 10-year forecast length; start dates every 1st February, 1st May and 1st August with 1-year forecast length
  • Omar's series of land surface initialization experiments with EC-Earth-v3.1b: m04r & m04o (T255-ORCA1-LIM2) done; m04s (still pending for November) & m04p (T511-ORCA025-LIM2)
  • Martin's volcano experiments; running on ithaca, third eruption almost finished
  • François' reproducibility experiments i06c and e00x (EC-Earth3.1b)
  • Danila's completion of her forecast experiments; waiting for ithaca to be free
  • Volunteer to run two seasonal hindcasts on new nudged initial conditions (1993-2009, T255-ORCA1). Does the forecast improve with initial states from coupled assimilation? Neven ?

Resources and platforms

  • Prepare all end of project reports; make files available in /cfu/pub, for the RES aim at writing a ~5 page report with two or three figures per project
  • Ithaca: UNDERUSED, only François running; experiments need to be organized among CFU members to keep the machine as busy as possible; backup of /share and /home for CFU users
  • RES: UNDERUSED, 53% used, expected 47%, no info about the use of Neven's project; two proposals granted, 2.1 Mhours and 0.3 for Mar-Jun 2015, used 36% until 10 Apr; other CFU members to prepare and submit independent proposals under their name, particularly proposing the SPECS and EUCLEIA experiments; next application deadline 19 May 2015, a fourth period for the activity to be requested, a volunteer needed to fill in the forms
  • ECMWF: UNDERUSED; resources 2015 37 MSBUs, 20 Tb; possibility of using additional resources from the EC-Earth special project and AEMET resources; request more SBUs from user support for 2015
  • PRACE-Tier 0 11th call <http://www.prace-ri.eu/prace-project-access/>, proposal submitted by Virginie with data pilot call included
  • INCITE programme to use the Argonne (Omar) BlueGene https://www.olcf.ornl.gov/2014/04/17/2015-incite-call-for-proposals/ deadline 27 June; Omar to request director's discretionary proposal
  • PRACE Tier 1: volunteers to prepare a proposal for a call foreseen in Feb to start the runs in Sept, check regularly with Jorge Rodríguez (BSC PRACE contact) http://www.prace-ri.eu/deci-12-call
  • GENCI: http://www.genci.fr/fr/content/utiliser-les-moyens-genci Martin requested but no possibility for foreign institutes to apply for GENCI proposals, although there were potential possibilities for proposals out of the GENCI procedures, waiting for a response…

Long-term pending issues

General

  • Consider the testing of CERFACS' atlas called CAMI (based on NCAR-NCL, report on V1.0 http://pantar.cerfacs.fr/globc/publication/technicalreport/2008/camiv1.0_user_guide.pdf)
  • Asif to prepare T511 output files for Jost to assess the seasonal hindcast climate
  • Virginie/Lauriane to provide feedback on the use of autosubmit with CNRM-CM
  • Models: testing the architectural definitions and migrate the sim script to a way of working similar to the run script like in EC-Earth (Oriol, Mingu, Asif), now to be considered only in Autosubmit3; Asif to find out about SMHI's plans to develop ecconf (include ensembles for instance); think about the idea of fetching svn branches in Autosubmit 2.5, where svn branches come from the trunk plus a merge with some needed changes to run with autosubmit (Asif to check with Uwe if they are ready to migrate to something like git and, if not, if IC3 can make its developments on svn, find out about their policy); get bits of code from templates into functions: this is already possible, but Oriol to contact Uwe to decide if the SMHI strategy is appropriate for us (sourcing)
  • New EC-Earth3 initial conditions with coupled nudged atmosphere/ocean: Omar to work with Virginie <file:///cfu/scratch/lpcaron/EC_EARTH/i05k/outputs/diagnoce/i05k/index.html>
  • Update on the EC-Earth3 tuning exercise, standard and high resolution (Asif, Paco, Chloé): rn_ebb parameter impact
  • Implementation of pp for EC-Earth3 (LP): adapt them to ORCA025 http://sansone.to.isac.cnr.it/ecearth/diag/timeseries/diagnoce/tom8/
  • Implementation of the PI RK scripts for EC-Earth3 (Asif): http://ic3.cat/wikicfu/index.php/PI3
  • CFUers volunteering for maintaining up-to-date sea ice, ocean and atmospheric initial conditions. (Request from Virginie and François). Important for participation in near-real time intercomparison studies such as the sea ice outlook.
ec-earth_experiment_meeting.1490096431.txt.gz · Last modified: 2017/03/21 11:40 by vguemas