The initial motivation of this commit was to ensure that the qunit
test suite doesn't stop when an error is thrown in a test, which
could happen if the error was thrown "sufficiently close to the
end of the test". Indeed, the "unhandledrejection" event being
async, it was sometimes triggered after the end of the test, when
the service registry was already reset, and the check of the
presence of the error service was wrong, so the error event wasn't
default prevented (e.g. await makeView(...) and the view crashes
at render time).
This led us to rework in more depth the way we deal with errors in
tests. Here are a few behaviors we want (probably not exhaustive):
- an error in a test must never end the suite (executed in py)
- an error in a test must always make the test fail, except if the
error is expected in the scenario, which one must be able to
state
- a test must always wait for potential unhandledrejection events
to be triggered before ending.
- ideally, we don't want to have to deal with unhandledrejection
in each test throwing an error (in order to prevent the suite to
stop)
To achieve this, we come with the following solution. We introduce
a new assertion method, "expectToThrow" which allows to state that
during the test, we expect errors to be thrown. It takes a list of
error messages that will be compared at the end of the test with
the errors that have been thrown during the test. If they differ,
a qunit failure is pushed and the test fails. If an error occurs
in a test and "expectToThrow" hasn't been called, qunit is directly
informed of the error and a failing assertion is done, make the
test fail as well.
If the error service isn't available in the test environment, we
apply the logic above when an "error" or and "unhandledrejection"
event is thrown. If the error service is available, we wrap the
default handler (typically the one that handles everything that
hasn't been handled by specific handlers, like tracebacks) and if
we get to it, we apply the logic above. This means that one must
call "expectToThrow" if
- the error service isn't deployed, or
- the thrown error is handled by the default handler, because
it is something like a traceback (errors like UserError,
ValidationError are graciously handled by the RPCErrorHandler)
and thus never reach the default handler.
In all cases, we prevent default the event such that the error
doesn't make the python test end.
Finally, to ensure that "undhandledrejection" events are handled
before the test ends, we wait, in the qunit lib, for a setTimeout
before ending the test, which ensures that all such events have
been dispatched.
closesodoo/odoo#137120
Related: odoo/enterprise#48211
Signed-off-by: Samuel Degueldre (sad) <sad@odoo.com>
Aaron Bohy
2f9c4979fa