Change job state handling in tests

We want to avoid printing "FAILURE" in the function wait_for_job
as a new use case may result in a job not starting in a timely
fashion and NOT be an error. So change "FAILURE" in wait_for_job
to "WARNING" and add checks for function errors in the tests
as needed (most places already check and log errors).
There were also many cases where "FAILURE would be printed by
wait_for_job, but the job would not have a non-zero exit code
and those are now fixed.