Draft: Pyflow generator

This merge request adds a new autosubmit generate subcommand to Autosubmit.

The usage is as follows:

usage: autosubmit generate [-h] [-engine {pyflow}] expid [args]

For example:

autosubmit generate a000 --engine pyflow --server=localhost --deploy

The command above will dynamically load the autosubmit.generators.pyflow and call functions to parse the options passed for the generator and to generate the workflow. At the moment it reads the configuration of a000 using autosubmitconfigparser. Pending:

• Move from autosubmitconfigparser to values stored by Autosubmit (i.e. skip this dependency if AS already has this info)
  • Not needed. Other commands call it too.
• Remove the NetworkX dependency from the PyFlow generator, and use the data from Autosubmit databases
  • This means we will assume users have executed autosubmit create first, and only then autosubmit generate (sounds OK to me)
• Add support to Splits
• Make sure that environment variables exported in Autosubmit are exported in ecFlow too (see discussion in comments)
• Make sure we are reducing transitive dependencies accordingly (AS should do it, instead of the PyFlow generator)
• Decide who/how will handle the scripts; we could:
  • Produce the final job template in Autosubmit, and include it in ecFlow as a script (ecFlow will add header/tail scripts)
  • Make all variables provided by Autosubmit for scripts, and make them available in ecFlow via envVars, PyFlow Variable, etc.
  • Combine these two approaches, or think in another approach?
• Verify that the execution of climatedt workflow (vanilla workflow) produces the same data in ecFlow5.8 & AS4
• Handle variables in templates and extra files (e.g. namelists) - see version 2 of the climatedt workflow
• Write tests
• Write docs
• Write changelog

Thanks to @mcastril for pointing to previous issues and to parts of the code where we already have the graph expanded, and with the proper dependencies. And also for suggesting to re-use the Autosubmit generated data, and benefiting of the structure produced by autosubmit create in a meeting this morning 🙇

Draft for now, but hopefully not for too long. Watch this space!

-Bruno