Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • SUNSET SUNSET
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 43
    • Issues 43
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 12
    • Merge requests 12
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Terraform modules
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • SUNSETSUNSET
  • Merge requests
  • !175

Draft: Provenance for the Loading and Aggregation module + rocrate provenance when executing jobs with autosubmit

  • Review changes

  • Download
  • Patches
  • Plain diff
Open apuiggro requested to merge Dev-Provenance into master Dec 04, 2024
  • Overview 7
  • Commits 11
  • Pipelines 31
  • Changes 68
  • Provenance feature for Loading and Aggregation modules: By specifying Provenance: yes in the Analysis section of the recipe, JSON files will be generated and stored in out_dir/outputs/provenance during the workflow execution. These files contain a comprehensive description of the transformations applied to the data using the METACLIP ontologies. As a result, the transformations can be interactively visualized with the METACLIP interpreter.

  • When running multiple jobs with autosubmit, setting the parameter auto_conf$provenance to True triggers the creation of the file rocrate.yml. This file is later used by autosubmit to create a zip archive with ro-crate-metadata.json at the top level. This JSON file includes all the provenance details related to the autosubmit execution. Additionally, it contains the provenance of each individual job, described with METACLIP, as well as the recipes used. For more information: https://earth.bsc.es/gitlab/es/sunset/-/tree/Dev-Provenance/use_cases/ex1_4_provenance_autosubmit.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: Dev-Provenance