Database runid update
Hi @dbeltran, the criteria for an experiment to be considered a new run of itself, is if you run the create
command, or if using the setstatus
you change certain amount of jobs or some of the main jobs, right?
We have an idea for the testing suite to generate some plots of the performance to see if there has been any long term changes in it, but to compare them we would need that the setstatus
to not be considered as a new run in any case, since it is usually used in the tests to check the fixes of failing jobs. The create
is always used at the same time for all experiments, so it would generate a new run for all experiments and a drop (due to issues with the cluster or changes to the code for example) would be reflected at the same point (marked by the runid) in all the plots. Also it would be better for the performance comparison that we already have implemented (right now, sometimes I need to update some experiments manually on the report since they had used a setstatus
).
It might be possible to make that the setstatus
keeps its considerations as of generating a new run like it is now for normal experiments but that it doesn't generate a new run for test experiments (expid starts with t
)?
FYI: @mcastril (might be updated alongside #827 (closed), since it is not urgent in any way).