Autosubmit GUI Next Iteration Discussion
@mcastril @kserrade @dbeltran @pechevar
In this issue, we take a look at design and architecture ideas for the next iteration of Autosubmit GUI. We also analyze and discuss design and architecture choices made in other platforms that play a similar role than what we currently plan for Autosubmit GUI.
You can also comment, suggest your own contributions, doubts, etc.
Cylc-8
Architecture
Starting at Cylc-8 Architecture we have that:
They adopt a Hub Proxy model inspired by JupiterHub
, where we have a central Hub that launches a Web Proxy service that will handle the requests of the users. Then, when a user tries to interact with his experiment or workflow, the Hub spawns a UI Server under that user's account. In Autosubmit terms, it would be as if the Hub is opening a terminal logged in as the user and setting it ready to launch Autosubmit commands, but without the terminal.
We have that this model effectively allows the Hub to run a UI Server as if it were the user and in that way access the user's files, modify them, launch commands, etc. However, the actions of the UI Server are restricted to what we, the developers, intend.
The UI Server needs to communicate with the workflow manager in some way, they (Cylc-8 team) achieve this by using ZeroMQ, which is a messaging library that allows for efficient communication between systems over a network.
For our purposes, ZeroMQ could allow us to implement communication between our UI Server and the Autosubmit instance that it has to spawn to (for example) change the status of a job. The request will be launched from the UI Server* and received by the Autosubmit CLI and then we capture the result and send it to the UI Server. Remember that through all these operations, the UI Server has made it possible to work as if it were the user.
Important: So far we have defined that there is 1 Hub and 1 Web Proxy, but we have 1 UI Server spawned for each user that tries to use the GUI. Furthermore, this UI Server can access the user's files and launch commands as if it were the user.
The Web Proxy plays another important role, it redirects the user requests from the GUI to the user's specific URL that shows the information retrieved by the UI Server specific to that user.
Comments about Autosubmit GUI
Our current implementation is centralized, there is a single API Server that works with specific requests and it cannot execute commands as if it were a specific user, only as webadmin
(the user that is currently running the server in bscesweb04
). There are ways to make this work though.
On the other hand, we have this decentralized system where a Hub spawns UI Servers for each user. Currently, I do not see any big barrier that would make Autosubmit GUI not work under this model, for example, Autosubmit API can keep working as it is and it would be just another resource that the UI Server requests information from. Moreover, it seems to suit the interaction needs we have planned for the next iteration of Autosubmit GUI. The only significant bump in the road is to get familiar with the tools involved in the creation of this infrastructure.
Ideas for Autosubmit
Looking at the Cylc-7 documentation, I found a couple of things that could be useful and not prohibitively hard to implement:
-
For Autosubmit GUI: In job information, we not only show the minutes it has been on the queue but when exactly it was submitted, it started, and finished. We could also include the last line of the log of that job, but this would require a change of paradigm in the logging system, which is something already in the backlog.
-
For Autosubmit: Cycl has a function scan that lists all the Cylc instances running on that machines. We can implement something similar so the users can easily check if some Autosubmit workflow is accidentally (or not) running on the background.