The old path would just return e.g. `Array(2)` for an array of size 2,
which is not very useful when trying to see if an array's contents are
relevant to an issue.
Turns out arrays are serialized pretty much exactly like objects
without a subtype, just with property names being indices instead of
keys. So add a case which formats such remoteobjects as
array-literal-ish.
closesodoo/odoo#46056
X-original-commit: 6d5639024fa0f7501efe836e5571cd45bb46b123
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Since Chrome version 80, the V8 javascript engine is using an new
pointer compression feature [1]. This features tries to reserve more the
4GiB of memory. As a consequence, during HttpCase tests, odoo tries to
launch chrome headless in a subprocess but fails because of the
memory-limit-soft.
With this commit, the Chrome browser is spawned in a forked process
that removes the memory limit.
Also a new Chrome CLI switch is used to prevent crash reports to be sent
to google.
[1] https://v8.dev/blog/v8-release-80closesodoo/odoo#45859
X-original-commit: 726d9c50720c2d3c9e617e998ed0dcd12272a271
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
During an HttpCase, when the Chrome browser starts, it may happens that
the websocket is unreachable. In such a case, the stop method is called
and tries to stop the browser gracefully by sending a `Browser.close`
through the websocket ... which is not reachable.
The result is a confusing traceback. With this commit, nothing is sent
is the websocket is not available.
Moreover, the `sigxcpu_handler` attribute is used in the stop method but
not yet declared. Fixed by moving it before calling the _chrome_start
method.
X-original-commit: 0d156fd3bc114b40680ca5aa0456e0f21a5b8cff
In some cases, the registry might be updated during a test step
(creating custom models/fields, for example). In those cases, there
should be an explicit call to `reset_changes` on the registry to make
sure that the next test class starts with a registry that is consistent
with the database state.
Co-Authored-By: Raphaël Collet <rco@odoo.com>
When revamping the message fetching from the headless browser, since
stacktraces are available (by default) on console.error and
console.warning events I assumed it could / would be useful to show
them in the Python-level log. And they *were* quite useful during the
original fixing stage.
However they turn out not to be very useful day-to-day:
* they add a lot of noise and lead to the error message itself being
lost in a big block of red / logging.error
* when a tour fails (which is the vast majority of the failures) the
JS stacktrace always points to the same location in the tour
manager (the one which goes "this step never succeeded") which is
completely useless
* aside from being bundled, normal JS code (where the stacktrace could
be useful) doesn't generally use console.warn or console.error, it's
going to straight blow up with an exception in which case we always
get a stacktrace
Leave the stacktrace formatting for console.trace as that's pretty
much the only point of using this instead of console.log.
closesodoo/odoo#44343
X-original-commit: 34500853f39c9557852cb83cbc274f59d803922f
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
* remove misleading documentation about a "test failed" message, in
13.0 any uncaught exception or console.error will cause the current
test to be interpreted as failed
* fix tour manager to console.error its step and not add a second
useless error message
* fix menu tester to try and display the failure cause on failure
* improve qunit's test reporter to print the number of tests failed in
case of test suite failure
Should make test failures in tours a bit clearer.
closesodoo/odoo#44296
X-original-commit: 78121b68d099b16f2d775a7a8a963a2a0f474843
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Sink handling of JS logging, exceptions and websocket timeouts so
calls other than _wait_code_ok handle them somewhat properly: the
issue fixed by odoo/odoo#41231 passed because it occurred during
module loading, which happens during initial page loading (browser_js
> navigate_to > _websocket_wait_event), which ignored logs (and
exceptions though here it's a console.error log), and as a result
reported no failure (and would simply miss that specific test as well
as every test following it).
Also since ChromeBrowser treats console.error as an exception,
important messages should be logged atomically. Merge two consecutive
console.error into a single one at the loading of modules so we don't
just get an exception "error while loading foo.bar" without any of the
useful details.
That ChromeBrowser treats console.error as exception is also why the
new method gets a flag (to suppress this behaviour): in the case of
two console.error, upon encountering the first it's treated as an
error so we try to take a screenshot, which goes through the messages
in order to get the screenshot response, which encounters the second
console.error, which gets treated as an exception, which hides the
first error.
Instead, screenshotting (and more generally _websocket_wait_id) should
treat console.error as a regular logging call, probably.
Also run JS tests in debug=assets for easier debugging (ha!) and
improve formatting of exception object when receiving an exception:
* if we can get a description on an `exception` remote object just
print that, it's formatted to show the exception type, message &
traceback
* otherwise format the garbage that is an "ExceptionDetails" object
Some code modify return of res.users().context_get, but this is a
cached method so this will unexpectedly affects totally unrelated code.
For example, changing the company with the company switcher could add
`allowed_company_ids` inside the cache, then it will be cached until the
server is restarted, even if we change company again inbetween.
Added test failed with:
"NotImplementedError: '__setitem__' not supported on frozendict"
on the line with `User = User.with_context(context)` where User already
contained `allowed_company_ids` in its context.
note:
in this forward-port, context_get is also changed to return frozendict
and prevent being able to have an unexpected issue by code that modify
context_get returns.
opw-2158340
closes#42465closesodoo/odoo#42723
X-original-commit: 5d69885c1cd6921b3de00aae7e0ed6fff243ff95
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Sort of but not really, this commit fixes a special case in which
launching a --test-file of a file with at least two SavepointCases would
create a postgresql deadlock and it would be impossible to terminate the
Odoo process without sending a SIGKILL or waiting for the lock to
timeout.
This was introduced at #39368 and happens because of the way that
unittests unwraps suites, to keep it short, when it unwraps the custom
OdooSuite class internally, it ends up with a vanilla TestSuite with
which to run the different test cases, and since #39368 depends on the
overrides added to OdooSuite to function, the class cleanups are not
triggered at the end of a test class (rollback, cache cleanups, env
reset, registry reset, etc.).
The fix is to manually unwrap the suite of tests to keep OdooSuite as
the suite with which to call the tests, which was already done for
--test-enable (although for different reasons, --test-tags?) which is
why --test-enable didn't have any problems.
This commit also fixes a typo I found on the backport, which meant
classCleanups were not being executed if the setUpClass failed, but it
had no effect on classCleanups during tearDownClass.
Task-ID 2160398
Depends on #43135closesodoo/odoo#43296
X-original-commit: 7a5ded7d40afc29043d356b5dece0dbe1fbd5ab3
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
An http-port provided on the command line (may also have been an issue
for config files, didn't check) would not be taken in account anymore,
because `odoo.tests.common` would be imported during the import of
`odoo` itself (when loading odoo.service.server), itself importing
`odoo.tools.config` leading to a default configuration being set up.
* remove `odoo.tests.common.PORT`, `config['http_port']` should be
used always
* defer the import of odoo.tests.common by moving it inside
load_test_file
* stop generating default configs
closesodoo/odoo#43283
X-original-commit: 45871f498ea4cf3ada719692e69cd413883ab442
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
In the SSF, o2m updates get initialized with no values (1, id,
{}). The record data is only fetched when the "record" gets updated
explicitly from which updates will hopefully get properly tracked &
saved.
However if the "record" was first initialized through an onchange
values which are updated by the onchange (diverging from the db) those
would not get tracked and thus wouldn't get saved when the record is
saved.
* use more specific placeholder (None) for "o2m records to update but
we don't have values yet"
* once we have values, always store them as an update-tracking dict
* if we don't have values yet for an o2m and an onchange is trying to
write to it, initialize with values from database first (might
eventually be a good idea to initialize upfront though there's the
question of what happens for default values and recursive views)
* mark anything coming back from the onchange and differing from local
values as changed (so they get sent out on save)
* properly reify parent values for onchange instead of sending them
as-is
* the evaluation context for contexts (and domains) needs properly
formatted values so use `_values_to_save` to get them, however it
cares about neither required-ing nor filtering out e.g. unmodified
fields, therefore add an awful toggle to handle this
Task 2150302
Probably todo in the future:
* better UI for change-tracking dict, should have "snapshot"
support (to freeze / discard previous changes)
* cleanup save, it's unclear that it properly resets the form
closesodoo/odoo#43235
X-original-commit: 6c99fe3ffec3aeb8d210e3b943ccef81d62d44ce
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When running multiple odoo on the same machine (in order to test
multiple things concurrently, etc...) it's easily enough to start them
on 8069, 8070, 8071, ... and that *seems* to work, but then they start
walking on one another and *losing* chromes entirely, so you end up
with hundreds of chrome processes: right now with 2 Odoo running tests
my machine is at 365 Chrome processes, 550 process and 3180 threads
total (should be ~160 and ~700 with 2 chrome processes at most).
Instead, ask the OS to ask for a devtools port, then close the socket
and pass that to the child process. There's a race of sort for the
instant between closing the socket and Chrome reopening it, but:
* the window is very short
* we got a random port from the ephemeral range, it's possible
somebody else gets it inbetween but unlikely (the 50% on a birthday
attack is 200 for Linux's ephemeral range, it's a somewhat lower 150
for the IANA range used by BSDs and Windows)
Sadly chrome doesn't stop if it can't bind to the port it's given.
closesodoo/odoo#42102
X-original-commit: 897ded1cb8b675024d75c6975ca171dbefdfc333
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in application tests or force them
to write custom quick and dirty tools and asserts. Purpose of this merge
is therefore to move tools classes and mocks to mail directly and use them
in various sub modules.
SPECIFICATIONS
Have class, mocks, tools and asserts available in mail so that all modules
below from mail can use them.
Including
* mock mail gateway in a clean way: mock server connection, email building
and sending;
* allow to simulate errors while sending emails to test corner cases;
* provide tools to insert emails in mail gateway;
* mock mail application to check record creation (message, notifications,
mails, ...);
* mock bus notification;
* provide clearer assert methods for bus and mail notifications;
* provide clearer emails sending and content methods;
* provide a with_user tool context manager for tests allowing to quickly
change current user given a login;
Most of those tools, asserts and mocks come from test_mail/tests/common.py.
They have been partially rewritten to be easier to use or to perform tests
more cleanly.
Future commits will gradually update existing tests in test_mail, test_mass
mailing and test_mail_full.
LINKS
Task ID 2068986
PR #38070
This commit takes advantages of the features added in the parent commit
to have better/cleaner tests.
closesodoo/odoo#39368
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
This commit partially backports bpo-24412, which allows the definition of
class cleanups (addClassCleanup) and module cleanups (omitted),
similar to instance cleanups (addCleanup).
This is useful for tests that override unittest's setUpClass and
could crash during its execution: If this happens, it is possible that a
bunch of crap is left in the database or even worse, the cursor becomes
completely fucked; Thanks to the addClassCleanup, we can undo the damage
done by the setUpClass.
Another benefit is that it is called unconditionally after tearDownClass
is called, so it can also be called as a replacement and/or safer
tearDownClass.
The unittest signature of `assertRaises` allows a named parameter
`msg`[1]. The message is displayed in case of assertion failure if
`assertRaises` is used as a context manager (but not when giving it a
callable).
The Odoo implementation did not implement this behavior, despite it
being expected by multiple odoo tests.
Fix Odoo version to be in line with the method we're overriding /
shadowing.
[1] https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertRaisesclosesodoo/odoo#36719
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
`is_false` relies on no small part on equality tests between the input
triplets and either `TRUE_LEAF` or `FALSE_LEAF`. While these are
defined as tuples, RPC domains will always be lists (as neither
XML-RPC nor JSON have tuples, and their arrays deserialize to Python
lists).
This is an issue, because tuple and list never compare equal. As a
result, while the in / not in predicates can succeed, the TRUE_LEAF /
FALSE_LEAF never will, and thus domains which contain either and might
shortcut (avoid a query entirely) will always go through the entire
process.
Fix by having domain normalization also ensure all triplets are
tuples: that's the first thing `is_false` does, it should never cause
issues and could fix / improve / shortcut other routines.
Note: Also implements TRUE_LEAF and FALSE_LEAF handling in the
SSF's modifiers evaluator. And fixes the ValueError to work
correctly if it breaks on a tuple / dict.
closesodoo/odoo#39706
X-original-commit: a44f008b918ee66742f7e943b00e4ee3d7fe2b78
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this commit, console.error were catched by browser_js
in order to log them but only value was used on received object,
which is correct only when the received object is text.
With owl arrival, error object may be logged. This
commit add the ability to manage logged object, fallbacking on
a complete representation of object if object is not an error or
description is empty.
closesodoo/odoo#39592
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When loading a page on an existing starting database, registry is not
fully loaded causing potential error when trying to access model
existing in database (views, menitem, ...) since model added in last
loaded module does not exist in registry.
Thus, executing browser js test may lead to errors when executed during
an update on a database with other modules installed.
HTTPCase should be executed post_install to ensure that registry is
fully loaded to avoid this problem.
Since HTTPCase are slower than other test, it is also a good idea to
execute them at the end, in order to prioritize fast fail.
With this commit, a warning is isued if such a test class is tagged to
run at install time.
While at it, remove deprecated at_install and post_install helpers and
remove the deprecated phantom_js alias.
closesodoo/odoo#39462
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
In some case, a browser_js test with login "None" following a browser_js
test with a defined login could result in the second test being executed
with the previous user.
This was caused by a race condition, a request response comming back
to chrome just after browser clear, restoring the old cookie.
(All odoo request have the set_cookie flag set in order to refresh
cookie timeout)
The solution here is to check one more time for cookie in authenticate,
but also to remove HTTPCase session from session_store. This will
only be effective when calling browser_js without login in the same
HTTPCase .
closesodoo/odoo#39525
X-original-commit: 218db53b9da1573dc18610678de3b7f763531c60
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
m2ms are internally represented as "6" commands, however in domains
it's possible to compare an m2m value to a list of ids (to
investigate: whether this is an artifact of internal webclient repr or
part of the real contract).
Add a workaround in SSF modifier computation to convert the m2m
command storage to a simple ids list.
A better fix would probably have been to represent the m2m as a list
of ids internally (and only convert on load / save) however it not
completely trivial as it has to be done recursively in order to
properly handle an m2m inside an o2m. So it's a complete change of the
internal data model (which should probably go alongide more
fundamental changes e.g. properly handling parent refs, etc...)
Also add very minor support for widgets (mostly so it's possible to
set widget=many2many on an o2m field).
closesodoo/odoo#39467
X-original-commit: 927979beff9e5d4f6c88078862779eaa88f4e07d
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The nightly clickall runbot fail due to a crash of the underlying chrome
browser used to run the test suite. The problem is related to the
resources the browser is using. We leverage the problem by starting one
dedicated browser per app.
closesodoo/odoo#39190
X-original-commit: 228e57d980839bd2b4e9123bf26f8a64de2ae6ab
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The form proxy will raise an error if a required field is not filled,
even when the web client would let it pass through. Indeed, the web
client (correctly) assumes that a user cannot fill in an invisible
field and that something will probably be done on the server to handle
that (e.g. an override of the write/create or simply a command-handling
code by the ORM). Since invisible fields do not even get a widget
instaciated for them, no validation takes place whatsoever.
Since it is sometimes (for obscure reasons) necessary to include all
kinds of invisible fields in tree view (e.g. to get correct values for
related stored fields, somehow), it is possible that required fields
in a view are in fact not really required since they are ignored
by the web client and then correctly handled/modified server-side.
This commit adapts the test proxies to behave similarly to the web client
(since that is, after all, the goal of these proxies) and to ignore
invisible fields when validating views.
closesodoo/odoo#37432
Signed-off-by: Damien Bouvy (dbo) <dbo@odoo.com>
Because the sub-record values would not necessarily get fetched
ever (whether default or stored), the computation of modifiers might
blow up if it relied on one of the un-fetched un-specified fields.
One such situation is trying to create a partner with child partners
if base_address_city is installed: the module adds a readonly attr
predicated upon the parent_id, without explicitly providing such the
field would be missing from the O2M record's values.
Closes#37176closesodoo/odoo#37452
X-original-commit: 184d1b69eac2b78c72228c81c5c6cf8cc4a56eb4
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The following issue occurs nondeterministically: a test fails when
trying to flush pending updates with a user id that does not exist.
This only happens with `SavepointCase` tests.
Here is the explanation: the method `cr.savepoint()` retrieves an
environment to flush pending updates, and the chosen environment has a
user that was created in another test, and thus no longer exists. The
problem is that `Environments.envs` contains environments that are no
longer supposed to be used.
The solution consists in, after the test, discarding the environments
that were not present before the test.
closesodoo/odoo#37341
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
In some cases, the first request of a test is made before the
proper setting of the cookie. In that case, Odoo picks a new session,
leaving the request unauthenticated.
closesodoo/odoo#35780
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
PURPOSE
Test frontend and UI tools of eLearning.
SPECIFICATIONS
Purpose: allow video auto play without any required user action. It allows to
have test tours using youtube autoplay feature like future website slides
tours (spoil, spoil).
LINKS
Task ID 1937768
When an HttpCase browser_js test is unable to join a request thread, a
warning is logged.
To avoid mergebot failures, this commit changes the warning to a log
info.
This problem was introduced by the refactor in this commit: 3ca788f55closesodoo/odoo#36022
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
During an HttpCase, when a browser_js test is finished, cookies are
removed to ensure that the next call will start on a clean base. From
time to times, when an HttpCase have multiple tests methods that call
browser_js, the user session of the previous test is being used.
With this commit, the session cookie is explicitely deleted and should
prevent that kind of problem. By the way, the browser's cache is also
cleared.
When issuing the first json command, it happens that the chrome process
has crashed. As a consequence, a log entry appears with saying that it
could not connect to chrome debugger.
After some investigations, it appeared that it was only happening with
google-chrome versions 75.0.3770.90 and 74.0.3729.169.
This error message was seen on the chrome stderr:
listp->slotinfo[cnt].gen <= GL(dl_tls_generation).
As a conclusion, it seems that we were hit by this issue:
https://github.com/GoogleChrome/puppeteer/issues/2207
While this commit does not fix the google-chrome issue, it adds a new
verification to check if google-chrome process is running before trying
to issue the json command. If it's not case, an error message is logged
with the error code.
After a browser_js, http requests threads are joined. If one of them
doesn't finish gracefully, a dumpstacks occurs.
As the dumpstacks call is in a loop, it can quickly polute logs. Even
more, a the dumptack may include threads that were not yet processed and
that will be joined in a future loop.
With this commit, dumpstacks will be called once and for all at the end
of the method, if at least one thread is remaining.
Furthermore, before this commit a sleep of 0.5 sec occured at most ten
times for each thread before considering it as lost. It means that each
thread is benefiting of the cumumulated time of the previous ones.
That's not fair.
With this commit, a default timeout of 10 sec is used for all threads.
The original warning is kept for each remaining request.
* disable background networking like GoogleUrlTracker ...
see https://codereview.chromium.org/3312014
* disable backgrounding occluded windows is a CLI swith that was
specifically written for tests to avoid non deterministic behavior. To
avoid test flakiness, it should come with disable renderer-backgrounding
and disable background-throttling as stated here:
https://github.com/smooth-code/jest-puppeteer/issues/137
* disable breakpad is used to disable crash reporting, the difference
with disable crash-reporter is not clear.
see https://peter.sh/experiments/chromium-command-line-switches/
* disable defaults-apps, prevent installation of default apps on the
first run
* disable dev-shm usage that may cause crashes
Undeterministic failures "Cannot connect to chrome"
can occurs in two scenarios:
-One time on a build, occasinnaly. This seems to be linked to chrome version.
runbots using version 71 don't have this problem when it occurs on runbot
with version 74 and 75.
-For all (36) attemps of the build to connect to chrome debugger,
this is exeptionnal but add a lot of noise by adding this type of
failure on any runbot.
Used chrome versions at this time:
r11-r22 -> 71.0.3578.98
r23-r24 -> 74.0.3729.169
r25-r28 -> 75.0.3770.90
All 'Could not connect' failure by runbot by day
day r13 r20 r24 r25 r26 r27 r28
07-15 36 4 2
07-16 6 1 1
07-17 36 2 1 2 2
07-18 2 39 1 2
07-19 1 3 1 3
07-20 1
07-21
07-22 2 1 4 1
07-23 1 1 1
07-24 4 3 2 1
07-25 2 5 2
07-26 3 3 1 1
07-27
07-28
07-29 36 1 1 1
07-30 3 1
07-31 1 3 2 1
08-01 3 1 1
08-02 3 1 1 1
08-03 1
08-04
08-05 1 5 4 1
08-06 1
08-07 2 1 2
08-08 2 2 2 1
08-09 2 2
08-10
08-11 1
08-12 2 1 4 3
08-13 1 2 1 2 72
This branch is the combination of several optimizations in the ORM:
* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;
* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;
* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);
* make method `modified` take advantage of inverse fields to inverse
dependencies;
* filter records by evaluating a domain on records in Python;
* a computed field with `readonly=False` behaves like a normal field
with an onchange method;
* computed fields are computed in superuser mode by default.
Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.
closesodoo/odoo#35659
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
Xavier Morel