Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • autosubmit autosubmit
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 338
    • Issues 338
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 21
    • Merge requests 21
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • autosubmitautosubmit
  • Merge requests
  • !160

Developing issue #511. Added jobs structure persistence.

  • Review changes

  • Download
  • Patches
  • Plain diff
Merged wuruchi requested to merge issue511 into master Apr 02, 2020
  • Overview 1
  • Commits 2
  • Pipelines 0
  • Changes 5

Added job structure persistence to job_list update_geneaology().

When job_list generation is called and transitive reduction is executed, Autosubmit will try to save the resulting dependency structure to reuse it in following calls and avoid doing transitive reduction repeatedly.

The persistence database is saved in /esarchive/autosubmit/expid/pkl/ as structure_expid.db.

If the method detects that the experiment structure has changed, it will recreate the table that stores the structure.

Currently, this method is automatic and does not require any flag from the user. After testing, the overhead it adds the first time it creates the persistence file is non-noticeable but the benefit is remarkable, so I decided to include it as automatic. Also, verification is rigorous.

Edited Apr 02, 2020 by wuruchi
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: issue511