Commit Graph
293 Commits
Author SHA1 Message Date
Xavier Morel 1af688d534 [FIX] core: flushing around HTTP calls
Explicit flushing of the cursor had been implemented in the "legacy"
`url_open` helper, but it was missing when going through `requests`
directly (via the `opener`), or when performing XML-RPC calls.

Fix that:

* extend `requests.Session` and `xmlrpc.client.Transport` so they take
  a cursor
* move the setup of the XML-RPC clients to `setUp` so they can *get*
  the cursor
* move the rest of `HttpCase.__init__` to `setUpClass` and drop the
  override entirely
* remove the now-redundant flush in `url_open` (as it's done by the
  `opener`)

closes odoo/odoo#77359

X-original-commit: c7bdae34690f31960d1fa95b5c3b4052540f5d07
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-09-29 05:48:42 +00:00
Nicolas Bayet 4813f42997 [IMP] mail,*: replace jinja with qweb
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment

By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).

There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).

We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.

To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.

This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
  (for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
  in order to see only one at once
- a floating select input to switch visibility of a particular logical
  branching

Task-27033

X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
2021-09-28 23:42:54 +00:00
Xavier-Do 8a7990735d [IMP] tests: autoretry mechanism for staging
This pr proposes an auto retry mechanism for tests. This shouldn't
impact normal testing: tests are not supposed to fail, but the growing
number of tests and pull requests can lead to some bottleneck when a
staging fails because of a random error. This mechanism should help to
reduce splits/the need to retry a failed pr.

This branch have been tested with the nightly multi build, creating 40
identical build without test-tags to disable know random errors.
This multi build is used to detect test failing randomly, this is an
excellent candidate to detect the effect of the retry.
On average, with the current base of this pull request, there is between
 10 en 15 failures over 40 build.
With the auto retry mechanism, only 1 build failed over 40 builds since
the same error was triggered twice.
This is simply because with the retry mechanism, an error that has a
probability of p to fail randomly will still have a probability of p² to
 fail with the retry mechanism. A error that occurs 10% of the time
 should only appear 1% of the time with one retry.  In most of the case,
  the retry is sucessfull: https://runbot.odoo.com/runbot/build/10053257

The current solution to allow to enable this mechanism only in some
cases (staging) is to check an environment variable
"ODOO_TEST_FAILURE_RETRIES" that defines a number of retry.
This will allow to retry more than once if an error still occurs to ofen
 with the autoretry.

The mechanism will run multiple time the same test on the same
test_case, meaning that some modification on self may impact the second
execution. The following code is an example of how this could be
problematic, but also a good example to test the auto-retry mechanism.

```python

class TestRetry(HttpCase):
    def test_fail(self):
        self.t = getattr(self, 't', 0) + 1
        if True or self.t == 1:
            import logging
            _logger = logging.getLogger('test_a')

            with self.assertLogs(level="ERROR"):
                _logger.error("This shouldn't be log at all")

            with mute_logger('test_a'):
                _logger.error("This shouldn't be logged (mute)")

            _logger.error("This should be log")
```

As we can see here the error logs are also managed, and emit at a lower level the first time, butany log higher than 25 will make the test "failed" and the autoretry mechanism will be triggered. The second time, everything is logged normally. We also need to replace Traceback by _Traceback to avoid being catched by runbot Traceback detection regexes.

The inspiration here commes from the assertLogs, that replace all handlers. The mute_logger had to be adapted to use the same strategy, so that quite_logger won't detect logs catched by mute_logger or assertLogs.

closes odoo/odoo#76336

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-09-24 13:59:56 +00:00
Christophe Monniez 2eaaa0614d [IMP] tests: improve subtest failure message
When a subtest fails, the failure message maybe a bit cryptic like
`Fail: Subtest (login=admin)`

With this commit, the parent test case and test method are displayed
too.

closes odoo/odoo#76046

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-09-07 05:47:17 +00:00
Raphael ColletandXavier Dollé 1595c0ee27 [REF] core: replace thread-local "envs" by cursor-bound "transaction"
Refactor the Environments object into a Transaction object, which is
bound to one cursor, and is no longer shared among several cursors.

The following methods/properties have been changed:
 - Environment.envs no longer works (because of the design change);
 - Environment.manage() is deprecated (no longer useful);
 - Environment.reset() is now an instance method;
 - env.clear_upon_failure() is deprecated in favor of cr.savepoint().

closes odoo/odoo#75598

Related: odoo/enterprise#20451
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
2021-09-03 15:45:46 +00:00
Raphael Collet 765ec7837c [REF] core: add methods flush() and clear() on cursor
This deprecates the ugly and inconvenient functions flush_env(),
clear_env(), and avoids explicit calls to precommit.run().

Part-of: odoo/odoo#75598
2021-09-03 15:45:46 +00:00
Rémy Voet (ryv) 0e3dcf091c [IMP] base: add warning if Chrome not installed
For test tours.
2021-08-24 14:13:33 +00:00
Martin Trigaux 15692f3948 [IMP] *: merge ir.model.data helpers
remove _get_id and _get_object_reference that were one liner to
_xmlid_lookup
2021-08-10 13:49:05 +02:00
Martin Trigaux c7bac3dee0 [IMP] *: make ir.model.data helper private
No reason to interfact with them directly in RPC
2021-08-10 13:49:04 +02:00
Samuel Degueldre 2d3dbe9217 [IMP] tests: allow browser_js ready code to be a promise and await it
closes odoo/odoo#74789

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2021-08-06 12:33:54 +00:00
wan 5bc37418bf [IMP] account: warn when the sequence format changed
Explain the situation when a manual change has been done to the sequence
of `account.move`. This was needed because some user didn't realize that
they changed the sequence, and when they realized it, it had polluted
multiple numbers after that.

closes odoo/odoo#74326

X-original-commit: 56f7afb253da6560cddae121f5b6cdab3e3b1c6e
Signed-off-by: Laurent Smet <smetl@users.noreply.github.com>
Signed-off-by: William André (wan) <wan@odoo.com>
2021-07-28 10:25:04 +00:00
Thibault Delavallée 777d0d316e [IMP] website_event(_track): reorganize tests
Purpose is to merge some tests, clarify naming and reorder files according
to their content.

Task-2577079
PR odoo#72411
2021-07-07 08:27:54 +00:00
Xavier Morel 438c6efdd7 [IMP] core: JS test runner
Emit JS warnings as Python warnings, otherwise it's a pain in the ass
to identify the t-raw messages (they get lost in the INFO spam).

Also just log exceptions when looking for specific messages instead of
re-raising them: if the pipe from chrome is full of exception reports,
the runner may blow its stack as it looks for the screenshot message:
on the first it takes a screenshot, then looks for the screenshot
message, finds an exception, takes a screenshot, looks for a
screenshot message, finds an exception, takes a screenshot, looks for
a s...
2021-06-29 05:34:16 +00:00
Xavier Morel ded278b9c2 [FIX] core: have HttpCase automatically set the base url
While HttpCase did set `web.base.url` before starting a browser, in
the non-browser test cases (or cases which would mix browser and
non-browser) it would not do so.

This is an issue when installing the database with one http-port and
running tests with an other e.g. after duplicating the database (or
even not duplicating it) in order to run multiple test instances
concurrently, which requires using different http ports.

Tests would then see the base url generated during installation,
embedding the port used at installation, and would break weirdly (at
best exploding due to not finding any server to bind to, and at worst
making request on the wrong instance entirely). Simply updating the
base url during setup seems to fix most of the tests.

Notes:

* Some tests (e.g. survey) don't flush() their create/update before
  calling `start_tour` or `browser_js`, the implicit flush because of
  the ICP handled the issue. Perform an explicit flush of base
  (similar to `url_open`) to ensure they keep working correctly.
* `url_join` should handle absolute URIs correctly, it does imply
  slightly different semantics in case the `base_url` has a non-empty
  path, but that seems like a very limited risk (and possibly
  convenient to boot).
* `payment` needed a fix because the vagaries of the MRO led to the
  extra parameter internally used by the thing to be passed to
  `HttpCase`'s `setUpClass`, which would not expect it.

closes odoo/odoo#72645

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-06-25 09:13:49 +00:00
Romain Derie 5a7e6d0247 [IMP] website: add tests - redirect 301 in case if trailing slash
This commit add tests to ensure every dispatch flows redirect to a 301 if the
requested URL has a trailing slash.
It also ensures the URL params are not lost in the process.

  - Basic controller: `/my/?a=b`
  - Website Pages: `/my-page/?a=b`
  - Basic controller with language: `/fr_BE/my/?a=b`
  - Website Pages with language: `/fr_BE/my-page/?a=b`
  - Homepage with language (special case/controller): `/fr_BE/?a=b`

opw-2505818
opw-2513575

closes odoo/odoo#71065

Community: https://github.com/odoo/odoo/pull/71065
Enterprise: https://github.com/odoo/enterprise/pull/18615
Related: odoo/enterprise#18615
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
2021-06-03 12:07:32 +00:00
Xavier-Do 4444475ef4 [ADD] base, core: built-in profiling tool in Odoo
This commit adds tooling to profile performance and save execution by
saving stack traces and queries to a file/database in specific format.

----------
Collectors
----------

For now, three different profiling modes (aka Collectors) are available
even if a last once should be introduced by @Gorash to profile qweb
execution.

- SQLCollector (or 'sql'): Saves the current stack trace and the query
every time Cursor.execute() is called. Any query executed on the thread
will be collected, no matter the cursor.

- PeriodicCollector (or 'traces_async'): Saves the stack trace every
'interval' seconds using a parallel thread to profile the caller thread.
The python implementation was optimized to minimize impact on
performance while remaining portable and easy to enable/disable
inside a odoo execution. Higher the frequency (lower the interval),
more impactful the profiling will become on the execution and increase
memory usage. From last experiments, 1ms looks to be a good minimum for
short executions.

- SyncCollector (or 'traces_sync'): Saves the stack trace every function
call/return. This collector is obviously quite impactful on performance
and can quickly overload the memory for long executions, but this is
quite useful to understand the precise path followed by some short
executions. Any time related information will be almost irrelevant with this
collector.

A base Collector defining minimal collectors features can easily be
extended to create custom collectors if needed.

----------------
Profiler & Usage
----------------

Collectors are not supposed to be used by themselves, but should be
given to a Profiler. The Profiler will synchronize collectors starts and
stop, and manage saving them to a file of in a ir_profile in the
database.

Exemple of usage:
```
    with Profiler():
        do_stuff()
```

This simple example will use the default collectors (sql and
traces_async) and save them to the database. The database is defined
automatically from current_thread 'dbname' if available.

Example of usage:
```
    with Profiler(collectors=['sql'], db=False, path=/home/user/logs/do_stuff_profile/{time}):
        do_stuff()
```

This more complex example disable the default behavior consisting
to save to the database, gives a path where the profile will be saved
and specify to only use the 'sql' collector. Note that
collectors=[SQLCollector()] would have the same behavior since
Collectors can be either a Collector instance or a string describing the
desired collector. This allows to define custom params for the
collectors and use custom collectors if needed.

Note that it is always possible to get results after execution without
saving it since they are available on the profiler.

```
    with Profiler(collectors=['sql'], db=False) as p:
        do_stuff()
    print(len([None for entry in p.collectors[0].entries if ...]))
```

Profiler will also save the stack below the profiler start point, and
collectors will only collect the part of the stack over this stack.
This is a good way to reduce collectors CPU and memory usage.

Collected entries will be saved as follows:

```
    [{
        'start': 2.0,
        'context': {},
        'stack': [
            ['path_to_file', lno, 'func_name', 'line_content'],
            ...
        ],
    },
    ...
    ]
```
SQLCollector will add three additional keys on each entry:
- query      (query without parameters)
- full_query (mogrified query with parameters)
- time       (the 'exact' execution time of the query)

----------------
ExecutionContext
----------------

A last tool, ExecutionContext, allows to define some context on some block of code:

Example of usage:
```
    def process_modules(modules)
        for module in modules:
          with ExecutionContext(module=module): # note the 'not linter frienldy but still convenient' 2 spaces indentation
            do_stuff(module):
```

This context will automatically be added in the stack as a virtual frame between
process_modules and do_stuff in order to split do_stuff from one single frame to
one frame per module.

----------
Speedscope
----------
The saved data are in a simple json format easy to analyze, but can't be visualized in
speedscope as they are. A utility class `Speedscope` can be used to generate a format
readable by speedscope. The used format is actually the format defined by speedscope,
meaning that all features should be available using it.

The output format is evented, meaning that we need to transform a list of samples
(a list of stack) to a list of event (going in/out a frame).
This is the main task of the Speedscope, as well as combining samples from different
sources, to display SQLCollector and PeriodicCollector results mixed together.

When stored on an ir_profile, the default speedscope generation can easily be generated
with the speedscope computed field.

This class can be used as it is but will mainly be useful for the next commit.

Special thanks to @rco-odoo for the in depth review and @Gorash for support.
2021-06-02 07:47:48 +00:00
Ivan Yelizariev 691fa54cc8 [IMP] core: clarify docs about config --test-tags
Technical name is not added to test_tags since https://github.com/odoo/odoo/commit/95b4f2ab4b5698ab3a28c9c35ac8da6fb6def983

at_install tag is added by default since introducing @tagged decorator: https://github.com/odoo/odoo/commit/b356b190338e3ee032b9e3a7f670f76468965006

Clarify how special tags at_install/post_install work.

Also, add dots for @tagged doc, because otherwise we have a mess in sphinx docs.

---

task-2431630

closes odoo/odoo#71329

X-original-commit: e4bf1e7dca2a9ef6dabf3f201d086ae77ff993d9
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2021-05-27 11:32:05 +00:00
Raphael Collet 960cd72774 [REF] core: add method get_currency_field() on Monetary to retrieve it
This replaces the setup of the attribute 'currency_field', which depends
on the presence of other fields on the model, and may therefore vary
from one registry to another.  This is necessary to make monetary fields
shareable across registries.
2021-05-03 12:33:29 +00:00
oco-odoo ed1205ae55 [FIX] base: form view emulator: bidirectional definition of 'in' and 'not in'
Doing domains like [('something_ids', 'in', 4)] crashed the emulator because it was trying to check something was 'in 4', instead of reversing the check (as real form views do).

closes odoo/odoo#68988

X-original-commit: eb14583442f2729182ffb8ec7b069e2505df3577
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-04-08 14:57:28 +00:00
8cc066173d [IMP] *: Improve assets management
This commit changes the way assets are declared in Odoo modules.

Before: assets were declared in template files. Template bundles were
generated from primary templates, so technically any qweb template could
have been called as an asset bundle, with the 't-call-assets' directive.

Being standard qweb templates, they had access to standard HTML tags
(script, link, with or without raw scripts or style definition), qweb
directives (t-call, t-raw, etc.) and could be inherited by other
templates.

Now: assets are defined in the module's manifest and generated by the
't-call-assets' directive.

More information on the new system can be found on the updated user
documentation (see the "JavaScript Reference" section).

Task: 2352566

Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
2021-03-31 13:57:17 +02:00
Julien CastiauxandRaphaël Collet 9a8cf18ddb [IMP] core: New 'capture_trigger' test helper
When testing cron triggers, it is common to get the newly created
triggers in order to validate they exist and are scheduled are the right
moment.

This new helper is a context manager that capture all triggers or
triggers created for a specific cron. The created triggers are
accessible via the context's object `records` attribute.

The various tests have been updated so they use that new helper.

closes odoo/odoo#68163

Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
Co-authored-by: Raphaël Collet <rco@odoo.com>
2021-03-24 09:29:38 +00:00
Alvaro Fuentes 8aa4f42442 [IMP] tests: add test_sequence order for tests
We have some tests in odoo/upgrade that are sensitive to the order on
which they are executed. Specifically: IntegrityCase tests need to be
run after all UpgradeCase tests across all Odoo modules.

To support this we implemented a sorting mechanism for tests based on
the test_sequence class attribute. This is intended to be used by meta
cases, not by individual tests.

closes odoo/odoo#66521

Related: odoo/upgrade#2184
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2021-02-19 12:51:00 +00:00
Christophe Monniez 9fce3f83cb [IMP] tests: add data_dir support in test_module_operations
In some situations, like during tests on runbot, the data-dir location
may vary.

With this commit, the `data-dir` CLI argument is added to the
test_module_operations script.

closes odoo/odoo#66492

X-original-commit: 781c91784a0a997967a0152dca2c59a0cb2bab74
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
2021-02-18 16:09:03 +00:00
c9c7198b53 [IMP] tours: make chrome request a websocket port from the OS
This should be a smarter and properly reliable version of #42071: in
that, the runner requests a port, closes it, and gives the port to
Chrome. However this apparently turns out to be less reliable than
hoped for and the port we just released can immediately be picked up
by somebody else (the original PR assumed the allocation of ephemeral
ports would be random or FIFO but that may not be the case, especially
inside containers).

This uses the same technique of requesting port 0 so the OS allocates
one, but it's Chrome requesting & immediately connecting so there
should be no race condition possible, and we keep the property that as
long as ephemeral ports are available Chrome will be able to open one
without conflicts or overlaps.

This leaves the issue of *retrieving* the port chrome got. Thankfully
it turns out we use a custom user-data-dir in which case Chrome writes
the port it got to `$DATA_DIR/DevToolsActivePort`[0]. Despite the
file's name it *also* contains the path for the devtools endpoint so
we need to only read the first line (rather than be able to read and
intify the entire thing).

Wait up to 10s before giving up entirely, and wait 100ms between each
check for the file's existence: on my machine without significant load
the file appears after 80 to 150ms, waiting up to 90ms seems ok (it's
not like we're in a super hurry as tours tend to be pretty long).

Other paths explored before moc used his eyes and brain and found out
about DevToolsActivePort:

* Chrome prints ws URL on the stderr, however because we don't know
  how much garbage Chrome might send there we need to send it to a
  continuous sink otherwise Chrome *might* end up blocking on its
  stderr because we're not reading from it. This turns out to be a bit
  of a mess of processes or additional threads.
* xdo suggested we check what ports Chrome listens on using something
  like netstat/ss (turns out `psutil` has support for that OOTB),
  which worked great except on WSL (where it didn't work at all), and
  the future-proofness was a bit questionable as Chrome might add
  other servers in the future.
* fme suggested using socket activation support[1] and passing in the
  port we'd opened without closing it, which would really have been
  ideal, however it turns out it was removed a few months later when
  chrome added pipes support[2], which was a pain to realize as chrome
  doesn't exactly do any useful error reporting (so unknown options
  just disappear into a void to be never seen or heard of ever).
* And while the pipes system[3] has *serious* positive attributes
  (even lower initialization overhead, we could remove the websocket
  dependency, also avoids wasting sockets though that's not too much
  of an issue here) it would require rewriting a lot more than just
  the initialization as it uses its own logical protocol
  (NUL-terminated JSON). TBF most of the messaging stuff is properly
  contained into just a few `_websocket` methods but still...

[0] https://bugs.chromium.org/p/chromium/issues/detail?id=624837#c4
[1] https://bugs.chromium.org/p/chromium/issues/detail?id=624837
[2] https://chromium-review.googlesource.com/c/chromium/src/+/954405/3#message-ab7415a7db7b94787300d987216e9ce60db47bc2
[3] https://chromium-review.googlesource.com/c/chromium/src/+/954405/3

opw-2378464

closes odoo/odoo#65195

X-original-commit: b679d97a83f1ac898ee0916e92a5b440702bdcdf
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
Co-authored-by: Christophe Monniez <moc@odoo.com>
2021-01-28 10:08:18 +00:00
Thibault Delavallée 42b8a15e5e [FIX][IMP] test: improve usage of new_test_user tool
Allow a False email as valid valid. Generate an email only if not given but
keep void values.

Ensure company_id / company_ids match, notably to ease creation of users
in a multi company test environment.

LINKS

Task ID-2421795
COM PR odoo/odoo#63677

X-original-commit: 21999058800957a08c5e129c527afe6b84812a8a
2020-12-23 15:16:12 +00:00
std-odoo 8c89694151 [FIX] test_event_full: fix the registration testing tour
Bug
===
Sometimes, the registration testing tour failed.

The bug can be semi-deterministic if we add a "sleep(1)" in the endpoint
"/event/<event>/track".

Reason
======
The reason for that is the service worker. It will pre-fetch all the
links in the page (see "prefetch-pages"), so for the "Online Reveal"
we will pre-fetch ~100 pages... If the server is slow, it can cause
issues.

If one endpoint takes some time, all other HTTP requests done by the
service worker will be waiting for it.

So, at the end of the testing tour, the service worker will continue to
make HTTP requests (because it makes the request sequentially) and so
some threads will still be created after the tour.

Even if "_wait_remaining_requests" is called to wait those threads, as
the service worker is still running, it will still continue to make HTTP
requests, creating new threads...

Fix
===
The solution to this issue is to kill the service workers of the browser
when we stop the tour before waiting for the end of the "HTTP request
threads".

Task 2381066

closes odoo/odoo#62827

X-original-commit: 6d08408a33880221b28f0f8a81d699825d927a48
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2020-12-03 13:50:52 +00:00
Raphael Collet 1398b6b44c [IMP] tests: deprecate SavepointCase
closes odoo/odoo#62031

Related: odoo/enterprise#14872
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-11-24 13:23:32 +00:00
Raphael Collet 7f2e168c02 [IMP] tests: merge TransactionCase and SavepointCase
The now unique class behaves as the former class `SavepointCase`.  It is
now up to the developer to use `setUp` or `setUpClass` for preparing the
tests.
2020-11-24 13:23:32 +00:00
Xavier ALT af2eaae941 [FIX] core: add UPDATE as valid many2many command in SSF
This commit fix the "Unsupported M2M command 1" raised `Form` helper.

The `onchange()` method will in fact emit UPDATE command for many2many
fields when the value submitted an the one in database has changed
(this is the case for example for nested m2m in form views)

OPW-2044631

closes odoo/odoo#59943

X-original-commit: 76bd8208a416cefab6becff9f2d7204c7d196201
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Xavier ALT <xavieralt@users.noreply.github.com>
2020-10-14 08:23:22 +00:00
Nicolas Martinelli 481d1393e7 [FIX] base_setup, website, tests: remove Gengo references
The Gengo modules were removed with:
https://github.com/odoo/odoo/commit/b38b72e456a
https://github.com/odoo/odoo/commit/9b1f0962baa

But there are still references to it.

opw-2349904

closes odoo/odoo#58938

X-original-commit: a30edba504aa1b62aa02092112a55bca65cf9a0c
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
2020-10-01 17:34:27 +00:00
Christophe Monniez 103d8f012e [IMP] tests: use concat demuxer to encode frames
When the sceencast argument is used to produce a video file of failing
tests, the framerate is computed with a KISS average.
This results in a misleading video flow. For example, if a tour step is
stuck, the average could show a smooth transition instead of the reality.

With this commit, the real duration of frames are used to produce the
video. The result is a more realistic video flow.

The configuration text file used by the ffmpeg concat demuxer is kept
alongside with the video file so that it can be used for other purposes
(e.g.: parse the durations to be used by a video player on runbot).

closes odoo/odoo#58280

X-original-commit: 58eb6a4565c024077557fa2de83fdf831e5ef026
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2020-09-22 19:41:09 +00:00
Xavier Morel 2da6bc2730 [FIX] base: virtual group fields on users with new default/onchange
* override `onchange` in res.users in order to properly generate and
  send the reified group fields alongside the rest: while default_get
  sets them up, those fields then get stripped by the onchange
  machinery as they don't actually exist on the model
* add a test to check for it
* fix SSF in relation with the new default/onchange system
  - because fields may not actually exist on the record,
    `record_to_values` needs to `read` the record data rather than
    directly access the recordset: the recordset likely will not
    know about fake fields
  - after the initial onchange has run the form must be filled with
    falsy values in case the onchange has not sent defaults for
    everything
  - on creation, all fields in the form should be considered modified

Task 2341153

closes odoo/odoo#58152

X-original-commit: f8658f229180a595d69cd6daefeded082c7f64ea
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-09-21 14:53:29 +00:00
Raphael Collet e8f7cfd00a [FIX] core: cursor hooks API and implementation
Python 3.8 changed the equality rules for bound methods to be based on
the *identity* of the receiver (`__self__`) rather than its *equality*.
This means that in 3.7, methods from different instances will compare
(and hash) equal, thereby landing in the same map "slot", but that isn't
the case in 3.8.

While it's usually not relevant, it's an issue for `GroupCalls` which is
indexed by a function: in 3.7, that being a method from recordsets
comparing equal will deduplicate them, but not anymore in 3.8, leading
to duplicated callbacks (exactly the thing GroupCalls aims to avoid).

Also, the API of `GroupCalls` turned out to be unusual and weird.  The
bug above is fixed by using a plain list for callbacks, thereby avoiding
comparisons between registered functions.  The API is now:

    callbacks.add(func)     # add func to callbacks
    callbacks.run()         # run all callbacks in addition order
    callbacks.clear()       # remove all callbacks

In order to handle aggregated data, the `callbacks` object provides a
dictionary `callbacks.data` that any callback function can freely use.
For the sake of consistency, the `callbacks.data` dict is automatically
cleared upon execution of callbacks.

Discovered by @william-andre

Related to odoo#56583

References:

* https://bugs.python.org/issue1617161
* python/cpython#7848
* https://docs.python.org/3/whatsnew/changelog.html#python-3-8-0-alpha-1
  (no direct link because individual entries are not linkable, look for
  bpo-1617161)

X-original-commit: d4b2e9224839aed8fc160ebe5a89e0f7d4c6a5bb
2020-09-03 14:29:39 +00:00
Thibault Delavallée 1404af789c [REF] various: use helper to create tests users
PURPOSE

Lessen use of mail-specific calls and variables

SPECIFICATIONS

Use mail_new_test_user tool in tests, lessening use of mail-specific context
keys in tests.

LINKS

Task ID-2326281 (context keys use cleaning)
PR odoo/odoo#56631
PR odoo/enterprise#12707

X-original-commit: 910559c092dc7dfa00b91339a5423e16cd4668e1
2020-08-28 07:59:23 +00:00
Luis González 4debe9f1fa [FIX] tests: Handle attrs containing numbers instead of bool
The Form class already handles cases for attrs that contain boolean
values, e.g.:
`attrs="{'readonly': True}"`

But it doesnt for integers, e.g.:
`attrs="{'readonly': 1}"`

This commit changes the expected non-domain value from boolean to
integer, because both are valid cases and the former is a subset of the
latter.

[1] https://github.com/odoo/odoo/blob/b3d4938ba6b1/addons/repair/views/repair_views.xml#L54

closes odoo/odoo#56612

X-original-commit: 782534a429f10e7b114c838687d38aa4080a920c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Luis González [Vauxoo] <luisg123v@users.noreply.github.com>
2020-08-26 15:36:43 +00:00
Raphael Collet 488e334fc8 [REF] core: combine default_get() with first onchange()
When creating a new record, the client calls `default_get()`, completes
the returned values with `False`, and calls `onchange()` to apply the
onchange to the defaults.

Optimize the double round-trip by integrating `default_get()` inside
`onchange()` for the first call.  The method is called with an empty
list of fields, and usually no field values, except for records in a
one2many field.  In this case, `onchange()` does both steps above.

Task 2261084
2020-08-20 13:39:04 +00:00
Xavier Morel a3ec322993 [IMP] core: tests reporting
* remove useless OdooTestRunner
* don't log results & time per-file, log a module-level tally instead
* add number of tests to post-test results
* generate a single test suite per module (see note)
* use the previous item to split out the at_install test-running in
  two steps: generating the suite for the module then running that
  suite, this way for modules which have no test, or for
  which all tests have been deselected by test tags, we can avoid some
  of the setup necessary to prepare for running tests but possibly
  quite expensive (e.g. `setup_models`)

Note: single test suite per module

I wanted to stop creating a test result for (essentially) every file
in the module, however because of the class-level ``addCleanup``, a
TestResult can't be reused by independent suites:

In order to run class-level cleanup, the test suite checks between
tests if the test it's *preparing* to run is in the same class as the
last test it ran, and if not applies the class-level cleanup.

The problem is that the "previous test class" is stored on the result
object, which is never cleaned up, and the "between tests" check is
really performed *before each test*.

This means when reusing results across suites it will run the
class-level cleanup at the end of one suite and immediately at the
start of the next, which will cause issues if class-level cleanups are
not idempotent (thankfully ``TestTestCursor`` has a non-idempotent
``tearDownClass` which let me discover the error).

Possible fixes are:

* don't reuse results
* clear the relevant states / attributes between suites
* put individual suites in a Big Suite for running

The latter seems simpler: just create a single suite for the entire
odoo-level module instead of creating one suite per test module.

Note to the note: the case of nested suite is taken in account, the
"end of suite" cleanup only runs at the end of the top-level suite, so
technically we don't have to unwrap suites for *that* purpose, we're
doing so in order to filter the test cases inside the suites. But
maybe we could integrate this feature to the suites themselves...

closes odoo/odoo#55185

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-08-19 14:08:21 +00:00
Xavier Morel c1c43bbe38 [REM] core: assertion reports
That's a not-very-useful subset of OdooTestResult, so:

* make results merge-able (aka add ability to update a result with the
  contents of another)
* remove support for test data files, and transmission of the
  assertion report thing through the data-files loading
* replace "legitimate" uses of assertion report by test result
* have run_unit_tests manipulate and return a result instead of weird
  flags & ternaries
2020-08-19 14:08:12 +00:00
Xavier Morel ec8a64a85c [REF] core: move testing-related functions to odoo/tests submodules
Attempts to clean up odoo/module and odoo/service a tad, they still
invoke testing-related utilities but are more logical in what
they *contain*.
2020-08-19 07:28:44 +00:00
Xavier Morel dccbf425a1 [FIX] core: ensure we create a new session for each tour
The browser itself would get mostly cleaned up between tours, but the
session object would not get cleaned, and apparently in some cases
that could lead to an incoherent session: a tour would add data to the
session which the next tour (logging in as a different user) would
not (fully) override, leading to a session inconsistency and a Session
Expired exception during the tour.

Fix by not storing the session on the test object, the session is
created during authentication then set on the opener & browser.
2020-08-14 21:20:47 +00:00
Xavier Morel 950d962d95 [IMP] core: add env to various auth methods
Allows accessing various keys, especially whether this is an
interactive login or not.

Also have the xml-rpc `login` delegate to `authenticate` instead of
having its own half-assed implementation.

And remove some dead code: as far as I can tell, Session.authenticate
is never called with a uid.
2020-08-14 21:20:47 +00:00
Laurent Smet 34d53ebc87 [REF] account: Add generic 'AccountTestInvoicingHttpCommon' class
Thanks to this setup, we will be able to run the tours independently to any demo data.
To do so, the HttpSavepointCase is born to do the same as HttpCase but with a setUpClass.

--task: 2290120
2020-08-06 11:20:27 +00:00
Xavier Morel b3761a8cba [FIX] core: fix handling of removals in sub-o2m
Issue is specifically in the case of an onchange removing a record in
an o2m in an o2m (so a sub-o2m) if loading an *existing* record in the
SSF: since the server pretty much only returns a REMOVE_ALL followed
by the records to keep or create, conserving the removal information
requires diffing the value currently stored in the form and the result
fo the onchange.

Diff which was properly done for top-level o2ms, but not for the ones
below that (apparently forgot this bit when improving support for
nested o2ms earlier this year).

X-original-commit: 177d009541cca589c6ebb85fc051b893ec001536
2020-07-20 11:42:58 +00:00
Raphael Collet dd8a6b8a82 [ADD] test_new_api: tests on queries made by create with computed fields
closes odoo/odoo#54209

Related: odoo/upgrade#1464
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-07-08 11:58:33 +00:00
Xavier Dubuc 5afc70faef [FIX] mail: fix open thread in discuss
See also odoo/enterprise#11684
task-2282273

closes odoo/odoo#54141

Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2020-07-06 14:23:17 +00:00
fja-odoo 7630ef3388 [IMP] website: add tests for website_visitor
website visitor testing had some flaws.

task-2079873

closes odoo/odoo#52281

X-original-commit: 4e5f049e48f06eb4be2a9972d06167e68bba1d61
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
2020-06-02 16:08:43 +00:00
Raphael Collet 020536142c [IMP] core: better tests on queries made by search 2020-05-20 07:56:14 +00:00
Xavier Morel fbfd36aabb [FIX] core: fix _cleanup_from_default
Technically removing the entire thing if it's not one of the special
cases is a form of cleanup I guess, but that seems a bit brutal and
counter-productive. So that function should *probably* return the
input value if it's not a type which requires special processing.

closes odoo/odoo#51531

X-original-commit: b42994c87dce12c5ba22943f6cbe00240fe43138
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-05-19 11:08:20 +00:00
Xavier Morel ac3cb21836 [FIX] SSF: handle o2m nesting
The SSF had pretty much punted on nested o2m as that's... not usually
a concern because few people are insane enough to put o2ms in their
o2ms in their views. And also because even if somebody did, sometimes
nothing would break because it turns out to be pretty difficult to
actually get into *using* those things.

MRP managed to do it though, and repro-ing required copying over parts
of mrp.production and then understanding why it didn't fail:

* obviously needs an o2m (f1) which contains an o2m (f2), in the
  view (an invisible tree inside a visible tree)
* needs an onchange which somehow updates the sub-o2m
* needs to actually trigger an onchange on the root form for f1,
  meaning f1 must be a dependency of a compute field or something (here
  I just marked every damn field as on_change as the optimisation of
  "don't call onchange when there's no need to" doesn't matter)

Also needs to be working on an existing record with existing
lines *and sublines* as the issue occurs with records to update.

The issue here is that `_onchange_values` would clean up f1 e.g. send
nothing for unmodified entries, and only send modified fields
otherwise, but it would only do so for the toplevel, meaning the
sub-level would not go through this step, and could send UPDATE
commands with an `id` field (set to the original value but
still). This would then proceed to blow up while loading the record,
as id fields are not writeable.

The fix is to perform `_onchange_values` recursively. Do that using a
separate helper in order to avoid blowing up on override and whatnot,
or faffling about with weird branching to get the "default" values in
case they're not provided, the root function can get all the relevant
bits and call the helper with them, then the helper does that setup
internally and calls itself directly.

An other issue I stumbled upon when investigating is a similar problem
on *save*, due to an implementation detail of the SSF: UPDATE commands
are fetched lazily.

`_values_to_save` took care of "hydrating" all update commands (and
validating and filtering them) of modified o2m fields, but as it would
not do so recursively a modified f2 would not get properly hydrated
and filtered, and could try to write `None` onto existing records.

closes odoo/odoo#51350

X-original-commit: 3dfb4cfad849936748cc6a5b8f712e100461a86f
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-05-15 14:53:20 +00:00
Christophe Monniez 277aa01112 [FIX] tests: wait for Chrome tab to appear
From time to time (frequently on Mac OS), the Chrome tests are failing
because there is no tab found in the browser instance.

It seems that in some circumstances, the browser is started but the tab
takes some times to appear, for that reason, the tab information is not
available in the first json commands.

With this commit, the json command is issued multiple times with an
increasing delay until the desired key is found in the json answer, or
until the timeout is reached.

closes odoo/odoo#50784

X-original-commit: 4b12643abc6b8ba34f14e468e8d7760bc2db385b
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2020-05-06 17:18:07 +00:00