100 Commits
Author SHA1 Message Date
Xavier Morel 7f5f296396 [REM] base: broken method Partner._email_send
The underlying `odoo.tools.email_send` was removed in
82de620424 (merged in 14.5) but this
callsite was missed, this method has been broken ever since.

I really want to remove this method directly, but technically it's an
expensive no-op if called on a recordset of partners without emails
set (or an empty recordset), so instead make it trigger a warning &
remove in master.

closes odoo/odoo#159177

X-original-commit: 457a4be9d6bb1e15b996a5a14468f9b0d88e0b0f
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2024-03-26 08:20:25 +00:00
Xavier Morel 429a37b395 [FIX] base: allow finding states by display_name
That is how states are (might be?) exported, so it should be possible
to find them the same way.

closes odoo/odoo#158984

Task-id: 3644762
X-original-commit: 26df8e2858d8bcaa11dfae68be328388b1983745
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2024-03-25 08:33:16 +00:00
Xavier Morel bc81230311 [FIX] core: skip marking completed tours as failed, restore had_failure
Succeeds #140464

Misunderstood `had_failure` and should not have reused it, its goal is
to avoid eagerly aborting some JS tests -- specifically the unit test
suites -- while still logging errors normally (useful when watching
interactively, or for the runbot's own reporting).

So the *checks* added on `had_failure` should in fact be checks on
`_result.exception()`, and as it turns out on `_result.done()`: if a
tour is already marked as successful we can't fail it either.

So we should not, we should log an error (to notify the caller /
runbot) and then bail. While #140464 did improve some things, we could
still lose legit errors and get pages of unhelpful `InvalidStateError`
if a tour would succeed *then* failures would occur, as the guard only
checked that the tour had already failed.

closes odoo/odoo#141815

X-original-commit: 87fcf66203dcbd281470285d648365d33d0e2fca
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-11-13 14:38:40 +00:00
xmo-odoo fc7804b14e [FIX] onboarding: dependencies
When onboarding was updated to override web assets in 518a4e4c43 it should also have been updated to *depend* on web.

closes odoo/odoo#141474

X-original-commit: 76907e3b34d8e33943024772e373fd0c3228b1f8
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-11-08 04:09:56 +00:00
Xavier Morel 2143ae7a9f [IMP] core: skip on marking tour as failed if it's already failed
If the web client gets into a bad state and piles on errors (or
exceptions) this leads to a mess with hundreds of kilobytes of
warnings being logged as we try to set an already failed tour to
failed (and also take screenshots), as in
http://runbot142.odoo.com/runbot/static/build/53098209-master/logs/test_only.txt

Also update `has_failure` to directly depend on an exception being
set, as that should be a more reliable indicator. And much like the
case of an error log skip trying to set the tour to failed in
`_set_exception` if it's already failed. Although in that case do log
the traceback as an `error`.

closes odoo/odoo#140879

X-original-commit: 96c501036a6dcb216e82e0dfa42b59a9b917d742
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-11-04 05:35:42 +00:00
Xavier Morel f6e01d6b53 [FIX] l10n_sa_edi: markup usage
`''.join` would strip the `Markup` out, so the markup would appear
literally in the result instead of being interpreted as HTML.

Also switch to using dict-style formatting, it's a touch clearer (and
shorter) for this case, since `e` is already a dict (hopefully).

closes odoo/odoo#140496

X-original-commit: 4d88e1df4deb5f1aab23ca117f6b6261babec3f7
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-31 15:54:29 +00:00
Xavier Morel 090b721f7e [FIX] mass_mailing: markup usage
- using a string to join `Markup`s is useless, that just strips out
  the `Markup`
- the entire toplevel `Markup` can be formatted in one shot

closes odoo/odoo#140428

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-31 10:34:00 +00:00
Xavier Morel 781dcdf396 [IMP] core: remove prefetch on Module during loading
If upgrading a database across an addition of a new field to
ir.module.module (which is uncommon but does happen), the field
prefetching would try to load the field before the database schema had
been upgraded, leading to a loading error.

Since we *only* want / need the module's name, we can `search_fetch`
to preload just the field we need, and avoid ancillary
prefetching. It's a bit of an unnecessary optimisation compared to
just turning prefetching off, but it's also simpler (shorter) here
so...

closes odoo/odoo#140010

X-original-commit: c0102ca5dc3507c39f5ae6406fd962419fb09c6e
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-27 08:36:17 +00:00
Xavier Morel cb7837470d [FIX] mail: upgrade of existing composers
When upgrading mail across the addition of the linked record's company
and mail alias, if the `composer.model` is not part of a module that's
already loaded (which is very likely) the compute will blow up when it
tries to look up the model in the env.

Add that check to the condition.

closes odoo/odoo#139845

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-26 21:08:19 +00:00
Xavier Morel 04f75728fd [FIX] *: restore gettext and unlink lints
Unwittingly broken by the removal of `__implements__` in
a6d601dc4e, these lints don't run
correctly with the old pylint, allowing new errors to creep in since.

closes odoo/odoo#139605

X-original-commit: 99cec73f585c8759142a707b06934d9c23aa5478
Related: odoo/enterprise#49473
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-26 13:41:19 +00:00
Xavier Morel 39be91b48e [IMP] core: clean up the ACL tracebacks some
Currently if a record is not in the cache it raises a `CacheMiss` the
handling of which leads to loading the record from the database (or
something).

If an issue occurs during that loading, because the loading is an
`except` scope the cache miss gets linked with the new one via

> During handling of the above exception, another exception occurred:

This is both noise (the cache miss is not actually relevant) and
misleading, because the wording makes it look like an unrelated error
occurred during handling.

- move the loading of the record out of the `except` to limit the
  scope of the `KeyError` and avoid "inheriting" it
- try to `from None` a few specific errors to remove implicit linkage,
  as the new error should have all relevant information, the key error
  / cache miss is just an implementation detail

closes odoo/odoo#139680

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-25 19:37:31 +00:00
Xavier Morel 9ebbfdac73 [FIX] *: incorrect translations markings
Fixes a large number of cases where strings are translated then
formatted, instead of letting `_()` do the formatting internally,
which allows it to recover from incorrect translations (missing,
broken, or extra placeholders).

Also

- removes translation markers entirely when there's nothing to
  translate e.g. `_("%s - %s")` is not useful
- fixes a few messes which lead to only partial translatability
  (DRY is generally a bad idea when translations are involved, even
  more so when you don't make the variable part translatable)
- fixes a few nearby issues noticed at the same time
- replaces a few `"%s"` by `%r`, which should automatically quote
  strings relatively appropriately
- fixes translated strings which use `\` to escape a newline (in order
  to fill-paragraph): `\` escapes only the newline, if the
  continuation string is indented this results in a bunch of spaces
  ending in the string to translate, which is pretty garbage for the
  translator, using implicit concatenation works much better

Note: some of the updates revert f-string parameters to %, because
babel (2.9) apparently has trouble with f-strings and blows up trying
to extract them.

Not in scope:

Helping translators fix translatable strings e.g. any translation
string with more than one placeholder probably should use keyword
placeholders

- Provides more context / data to the translator to make sense of the
  sentence.
- Allows reordering the translated terms, which can be necessary
  depending on the sentence and language.

closes odoo/odoo#139314

Related: odoo/enterprise#49311
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-23 16:45:09 +00:00
Xavier Morel 57984131f7 [REM] web: pwa registration success log
odoo/odoo#136944 removed the log from website_event_track, but despite
noting the discontinuity I missed that *a new instance of the log line
had been added in 16.1* and that is likely why 16.1 was suddently
spammed with that: website_event_track (likely) only installs its PWA
when opening an event on website, but since odoo/enterprise#35322 the
web client tries to install its PWA *every time it's loaded*, which
can be up to once per test for qunit tests (if they need a webclient).

In odoo/odoo#133560 this registration was moved from enterprise to
community, so needs to be nuked here as well.

Follows the removal of this log line in odoo/enterprise#48831.

closes odoo/odoo#138469

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-12 11:20:36 +00:00
Xavier Morel 9d3de1190b [FIX] core: make model decorators work better with decorator 5
Decorator 5 changed the default decoration method from a transparent
exec-ing to wrapper functions. This makes the decorators visible to
the profiler, and breaks one of the profiler tests as the stack traces
now differ between using decorator 4 and decorator 5. Amongst other
concerns, this is an issue because debian bookworm has updated
decorator to 5 (.1.1), and the next ubuntu LTS (which should be 24.04
hopefully codenamed Nefarious Nematode) will do the same (Ubuntu has
been providing decorator 5 since 23.04).

5.1 added a `decoratorx` function which corresponds to the old
exec-based `decorator`, however it doesn't have a `decorate`
version. So we have to flag the wrappers, instead of decorate-ing the
original method with them. This seems to have the same semantics so
why we were using `decorate` is not entirely clear why we were not
doing that previously (neither
b1e83fd7b8 nor #25383 really provide
explanations). Possibly because pylint is a dum-dum and requires
ignoring one of its rules?

Implement in 14 since it doesn't hurt even though the test in question
does not exist yet.

closes odoo/odoo#137323

X-original-commit: 52e92904eca3cdeaee734b0a83e2d3fb463e8b56
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-03 06:16:49 +00:00
Xavier Morel 5dd8159e13 [REM] website_event_track: pwa registration success log
Not very useful to the average user, and can fill up logs during
tests (mostly 16.1 onwards for some reason but might as well square
up everything).

closes odoo/odoo#137149

X-original-commit: 4461d90e534d7568e6b676913c959d9d083f88fc
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-02 08:03:45 +00:00
Xavier Morel 6c59eea421 [REM] base: use of PyOpenSSLContext in mail server
The mail server mostly doesn't use the intrinsic features of
`PyOpenSSLContext`, instead it pretty much just uses the underlying
`OpenSSL.SSL.Context`, just through the wrapper.

The only thing it actually uses from `PyOpenSSLContext` is
`load_cert_chain`, and since we are using a keyfile and are not using
a password this is equivalent to *two* function calls. Just perform
those two calls directly, remove all the indirections, and remove the
unnecessary import.

Bonus content: since 2.0 `load_cert_chain` reraises the inner errors
as `ssl.SSLError` which we don't handle, so we avoid this extra issue.

This was discovered because from 2.0.0 to 2.0.4 the
`contrib.pyopenssl` module was marked as deprecated (it was
undeprecated in 2.0.5) but regardless its use is an unnecessary
complication here.

closes odoo/odoo#137198

X-original-commit: e534bbed78a80d1ba0c8edd22e039e5cfb50e613
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-02 06:54:42 +00:00
Xavier Morel beb0c25a94 [IMP] core, bus: future werkzeug compatibility fixes
- `Response.charset` is deprecated since 2.3, `Response.set_cookie`
  accesses the currently-extent internal `_charset` directly until
  this too gets removed in Werkzeug 3.0. Add a `_charset` to
  `FutureResponse` so this does not crash.
- Bytes response headers are deprecated since 2.3, and will get
  removed in 3.0, passing bytes in websocket is completely unnecessary
  happenstance which is trivially fixed.

closes odoo/odoo#137145

X-original-commit: 66e3040d4ad9e03aabefebd4799f699ad30fca45
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-02 06:54:39 +00:00
Xavier Morel a92e88d547 [IMP] base: make tests less sensitive to babel / CLDR updates
Babel's named / implicit formats (short, medium, long, ...) come from
the CLDR, which can get tuned as debates get settled, cultures shift,
etc... as a result using these formats can break tests on any babel or
even CLDR release (technically nothing stops distros from updating
their bundled babel with new CLDR data).

b77eb98bbf6bc937efb7941802c0794ae3622094 previously did some
mitigation of this issue, but even if they're not yet in distros
further Babel updates (e.g. 2.12) already affect some of the patterns
we're using.

Proactively mitigate this issue more by only using explicit datetime
patterns (and time patterns, date is fine because it retrieves the
pattern from the lang rather than default to babel patterns) in the
date/time formatting tests. This means only specific terms still vary,
and those should be a lot more stable / reliable than e.g. futzing
with separators and minor formatting issues.

closes odoo/odoo#137181

X-original-commit: 3547e980428cc1dd11c8e82446e7d7f824b9f6f8
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-02 05:56:06 +00:00
Xavier Morel e2752a04ff [FIX] safe_eval: 3.11 compatibility
Complement on 1e35315399 (#112450):
alongside the split between forwards and backwards jump we missed that
3.11 has a specialized version of each for the `is None` and `is not
None` cases. A use of that was added in standard in 16.5 (#120446) but
more generally it makes sense that server actions would support
conditional tests against `None`, probably...

closes odoo/odoo#137099

X-original-commit: 3227ae45fb79cd08a102aecb484e4f0a4f2597c1
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-29 13:11:58 +00:00
Xavier Morel 62d3e1a1a3 [FIX] crm: incorrect field name as expression source
PR #105873 added computed indexes on the `mobile` and `phone` commits
for the benefit of `phone_mobile_search`, however it created both
indexes to work on the `phone` field as source, rather than whichever
field they were named after and partial for. As a result, the index
has no effect when searching on `mobile`.

closes odoo/odoo#135878

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-21 08:59:47 +00:00
Xavier Morel 4fec630081 [FIX] base: correctly check debug mode when formatting access errors
Intent of checking `group_no_one` was always to query the advanced
info / debug mode, however when the semantics of group_no_one got
changed in 31518bc09b this site was
missed, and now always displays "advanced" errors for internal
users. Which was not the intent.

Also since we're printing `display_name` and some of them annoyingly
hook onto context variables to show extended information, reset the
context to the user's default in order to avoid such
extended-formatting `display_name`.

Also fix "debug mode" in `TestIRRuleFeedback`, which has been broken
since time immemorial (likely as long as the group_no_one semantics
changed).

closes odoo/odoo#135940

X-original-commit: 9d3ffa6540c07e97b7160167756edd8e71f2308a
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-20 06:16:47 +00:00
Xavier Morel 6177b04ce5 [FIX] core: remove unnecessary ast.unparse
Only available in Python 3.9, and for now at least Odoo remains
committed to supporting Python 3.8.

And it's not actually necessary: `literal_eval` supports ast node
input, because internally it just `ast.parse`s the input then
evaluates it, giving it an AST node just skips the parse step.

closes odoo/odoo#135302

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-13 13:49:38 +00:00
Xavier Morel 885aef7b14 [ADD] base: support for breakpoint() in qweb
The breakpoint() builtin was added in [Python 3.7] ([PEP 553]). It is
conveniently everything-agnostic, can be configured (including
disabled) via an envvar (`PYTHONBREAKPOINT`) and
code (`sys.breakpointhook`), and defaults to pdb (`pdb.set_trace`).

This is more flexible as it allows using arbitrary callables without
special casing, which mostly allows using less common
debuggers (e.g. IDE debug servers / hooks).

As using an empty string for the debugger name was not allowed
previously, co-opt that to invoke `breakpoint`. And deprecate the old
style.

[Python 3.7]: https://docs.python.org/3/whatsnew/3.7.html#pep-553-built-in-breakpoint
[PEP 553]: https://peps.python.org/pep-0553/

closes odoo/odoo#134842

Related: odoo/documentation#5800
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-11 10:29:02 +00:00
Xavier Morel 46178afee0 [IMP] core: precompute (company_id parent_of $x)
There are cases where the result of `check_company_domain_parent_of`
is used in in a loop, leading to the parent_of relation being
recomputed once or even multiple times per iteration during SQL
evaluation.

Computing the parent relationship turns out to be fairly expensive in
worst case scenarios (e.g. lots of companies), so while precomputing
doesn't save much for a 1:1 situation (though it does make the job of
the expressions compiler a bit simpler), the ability to compute it
just once instead of say 140 times does make a huge difference in
e.g. some report renderings.

Nota: apparently `_check_company_domain` can be called with a string
because lol, so there's a special case for that.

closes odoo/odoo#133530

X-original-commit: 2f9ae135c9dd7cb09fa83fda7a9b304993cb3edc
Related: odoo/enterprise#46518
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-04 08:03:36 +00:00
Xavier Morel 08518bbad3 [FIX] account: index account codes
Account codes are very commonly searched by prefix. This field used to
be implicitly indexed thanks to a unique constraint on `(code,
company_id)`, however the constraint was removed in #125642 rather
than converted to an index.

For the issue at hand `code` has a much higher discrimination power
than `company_id`, however it might make sense to also index
`company_id`, or to add compound indexes for `code, company_id` and /
or `company_id, code` for other work loads.

X-original-commit: 45c579908d90f6275b7d255d59c6555e8a57b959
Part-of: odoo/odoo#133530
2023-09-04 08:03:36 +00:00
Xavier Morel 8d06889ec3 [FIX] base: encoding guessing of html module descriptions
I missed a critical issue in #133708: various users had discovered
they could already fix description issues by adding an XML declaration
to their document which is very cool (though technically not really
valid).

What is a lot less cool is that lxml gets *extremely* unhappy when
asked to parse *strings* with an encoding declaration, raising a
ValueError, so the purported fix breaks on any module which does that,
which seems to include a lot of OCA modules.

Gate the encoding guessing by bailing if the document has an XML
declaration, in which case we just assume the author knows what
they're doing and we leave them alone. For extra safety, check the
encoding declaration in ascii and utf16. Could also have checked for
BOMs, but lxml seems to not care about them overly much (in fact it
seems to prefer them decoded which is odd).

Also same as non-utf8 descriptions, mark XML declarations as
deprecated (because it's a hack to make UTF8 descriptions work which
is not necessary anymore).

closes odoo/odoo#133968

Reported-by: @rezak400
X-original-commit: fd353d7d0104431208b91603431e41ef4a6e54bb
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-01 16:46:03 +00:00
Xavier Morel dd4b3c3a66 [IMP] web: simplify waitForCondition
`waitForCondition` is written in a rather old style with a recursive `setTimeout` to handle the delay / debounce between checks. The code is rather hard to follow, and unnecessarily so: the code can be rewritten as an iterative async function which is a lot simpler.

Important change: `waitForCondition` now immediately checks for the condition instead of waiting `interval` milliseconds before doing so.

This is because the conventional usage is for it to be preceded by an `await triggerClick`, which includes an `await
waitForNextAnimationFrame()`. As a result there are good odds the click has already been processed by the time `waitForCondition` runs.

closes odoo/odoo#133645

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-01 15:09:49 +00:00
Xavier Morel 431b4e69e3 [FIX] base: correctly parse utf8 html module descriptions
Apparently `lxml.html.document_fromstring` (and possibly other
`lxml.html` loaders) parses byte-strings as latin1 regardless of their
actual encoding, maybe because python2, maybe because there's a super
legacy html4 parser underlying it.

Either way that means ever since loading
`static/description/index.html` files was added 10 years
ago (4bf6a7ea4c) `_get_desc` has been
loading these files in latin1 rather than the utf8 most people would
expect.

Add an explicit decoding phase to try and load html description files
in UTF8. Fall back to latin1 in case there are description files which
are genuinely in latin1, or even just some random-ass broken stuff
which very much isn't utf8 (the extended-ascii encodings -- of which
latin1 is one -- will happily accept and mangle any input as every
byte value is valid, utf8 is a lot more structured).

Closes #127846

closes odoo/odoo#133859

X-original-commit: 4dbc3b00e587f3d64cfd964a685f2bddd1b499ad
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-01 11:39:56 +00:00
Xavier Morel 2da41ec07b [IMP] core: don't log cache being nuked during test teardown
Improves on #119813 (595aa24843): the
commit splits the ORM cache into several, and adds a log entry when
invalidating caches (both individual and all).

To keep tests isolated and coherent (and also make performance tests
usable), the test framework has to clear all caches between tests to
ensure they don't affect one another. This adds a line of log
to *every* test, pointing into the guts of the test framework.

Since the clearing is willful, unconditional, and not bypassable, the
log line has essentially no value, it just adds tremendous amounts of
noise to the logs.

Fix by muting the registry logger specifically when clearing the cache
in the test suite (there is currently no dedicated cache logger, if
there ever is mute that instead).

closes odoo/odoo#133811

X-original-commit: 52371bac7d4fadbeab139e4cd044cdc6b1005299
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-01 05:53:42 +00:00
Xavier Morel 5dbdf7ca69 [IMP] core: reintroduce warning on watch
Was removed during the watch refactoring of #111422, present to avoid
people merging `watch=True`.

closes odoo/odoo#132727

X-original-commit: d2b33743e1b6faef82b24eb1e30b142e6b072290
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-08-23 07:40:34 +02:00
Xavier Morel f96988cbe5 [IMP] core: improve test request blocker
Better tag selection
====================

The block was originally hooked onto the `external` tag under the
assumption that this was the tag used to allow external requests.

It's not, it's the tag for the "external" test suite, which is only
one of the test suites allowed to perform external calls. Hook again
onto `standard`, this might have some false positives (allow requests
which we'd rather block), but it should have a lot less false
negatives (block requests we want to allow).

Make Overrides Easier
=====================

Currently `_request_handler` raises a regular `ConnectionError`, this
is an issue because it passes some requests through, which could
themselves trigger genuine `ConnectionError`.

When overriding `_request_handler` to implement fallbacks for bespoke
URL mocks, these two cases need to be distinguishable as overrides
likely want to handle blocked requests, not actual failures.

Therefore `_request_handler` should a dedicated exception. This
exception should be a subclass of `ConnectionError`, so that blocked
requests are treated as regular connection failures by normal Odoo
code.

Class-scope
===========

Originally `_request_handler` was scoped on the instance with the idea
that it'd be a `mock` object, which individual tests could
`configure_mock`. This turned out not to work correctly, because
`Mock.side_effect` does not receive a `self`, hence the `Session` was
inaccessible and it was not possible to passthrough local requests.

While I moved to a regular `lambda` (because a direct method didn't
work either), I forgot to remove the `request_mock` attribute, and
didn't think that the block could now be lifted up to the class
scope. Doing this, requests performed during a "standard" test case's
`setUpClass` are now also blocked, which they very much should be.

closes odoo/odoo#128977

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-07-19 13:13:32 +02:00
Xavier Morel 2a92852d5c [FIX] web_editor: suppress exceptions in get_video_thumbnail
Error responses are already ignored, but connection errors would blow
up the entire thing, which breaks `test_resequence_images` (and
possibly others).

Part-of: odoo/odoo#128497
2023-07-15 12:00:11 +02:00
Xavier Morel 0892923bcf [FIX] mass_mailing: test_mailing_controller
Add a handler for the fake URL we track, otherwise after visiting the
short URL we get redirected and the request blows up.

Part-of: odoo/odoo#128497
2023-07-15 12:00:11 +02:00
Xavier Morel 4111f02fb3 [ADD] core: block requests calls during tests
Non-external tests should not be performing requests to external
websites or services.

Add a mock to handle such requests:

- if the test is not tagged external
- and the request is not to localhost
- and the request is not to `file:` (it's possible to install a
  FileAdapter in a session to resolve file URLs)
- raise a connection error (and log, both with some details of the
  blocked request for debugging)

The mock is layered on the lowest possible level (`Session.send`), so
tests can either:

- reconfigure the mock to handle cases differently (the mock is
  re-created on every test)
- layer their own mock at a higher level of the library

Eventually we might also built-in a routing / dispatch mechanism so
it's easier to declare external services you want to mock, somewhat
similar to what's available client-side.

Whitelist `file:` because e.g. zeep performs `file:` request, using a
bespoke adapter installed in its session.

Part-of: odoo/odoo#128497
2023-07-15 12:00:11 +02:00
Xavier Morel 6533c081fb [FIX] mail: resilience of get_link_preview_from_html
`get_link_preview_from_html` was coded to avoid fetching the entire page content when it only needs page data, however it tries to decode the entire input buffer even though the fetch window might cut off a codepoint in two.

Fix by stripping out anything which follows the `</head>` tag, which might contain partial codepoints.

Also add a few more improvements:

- avoid visiting the entire buffer if we parsed more than 16k (as unlikely as that is), even when offsetted `find` returns an offset from the start of the buffer
- don't decode upfront, as `html.fromstring` will decode just fine internally (better really as it has decoding fallbacks)

closes odoo/odoo#128240

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-07-13 00:46:21 +02:00
Xavier Morel ce1a4a2ee0 [IMP] base: deprecate global rules
ir.model.access rules without group will now log a warning

Part-of: odoo/odoo#125216
2023-07-11 22:33:47 +02:00
Xavier Morel 489566d4bb [FIX] website: don't error on inaccessible model
When deleting a page, `search_url_dependencies` tries to trawl through
all models to see if they might have a link to the page being deleted.

However if the user invoking that function does not have access to a
model with an HTML field, it raises an error. Given pages are managed
by website designers which are *not administrators* there is no reason
to believe the current user has access to every model in the
database (not that even admins do these days). Since
`search_url_dependencies` is a best-effort search anyway, just ignore
any model to which the current user doesn't have access.

An alternative would be to do the search in sudo mode, but that
doesn't seem necessary, and could even be problematic if a match is
found:

- it might leak information the user should not access (because the
  record name is returned, as well as the model & field names)
- it will trigger further access errors (because links to problematic
  records are provided, on which the user might want to click)

closes odoo/odoo#127973

X-original-commit: 47a69a186bc415bf61a523da9686ca96c0beb74e
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2023-07-10 18:16:31 +02:00
Xavier Morel 074a91a300 [FIX] test_mail: mutlicompany systray test
odoo/odoo#122354 added this test but didn't handle that other models
might trigger systray activities.

On June 12th, the test started failing because calendar has a "Pricing
Discussion" demo calendar event on the 12th of every month. It
probably would have also failed on the 3rd and 22nd which both have
demo meetings for the admin.

Fix by looking up specifically activities of the test model we're
concerned with.

closes odoo/odoo#124684

X-original-commit: 54dce61ef8fa97e18f0cf53d1f098af55e1210e9
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-06-13 10:13:34 +02:00
Xavier Morel 70f131626c [IMP] account_tax_python: simplify code, remove context
- remove context injection added 6 years ago
  (2dab340717) but apparently never
  used, if it's needed in the future it would be much cleaner to
  extract the context(s) into a method and allow overriding that
- filter taxes just once, rather than filter then re-add
- also keeps recordset order which is probably useless but can't hurt

closes odoo/odoo#123651

Signed-off-by: William André (wan) <wan@odoo.com>
2023-06-06 14:48:27 +02:00
Xavier Morel 2b30be424f [FIX] core: content tab discovery in headful mode
It's not clear when and how this happened but apparently "headful"
chrome has a built-in background_page for hangouts which appears
before the `about:blank` page in the list of targets, and possibly
appears before the `about:blank` page has opened at all.

odoo/odoo#111422 was tested with chromium which apparently doesn't
have this feature either (or does it?), which probably contributes to
having no idea when it appears.

This feature also doesn't respond to `--disable-extensions`, despite
its url marking it as one:

    chrome-extension://nkeimhogjdpnpccoofpliimaahmaaome/background.html

The result was that the tour runner would hook onto the hangouts
target and try to load pages, which it would reject with
`net::ERR_ABORTED`, hence the tours just getting stuck.

Fix by improving the heuristic to find a content page: look for a
target of type `page`, and with the url `about:blank`, rather than
just take whichever tab target is listed first. Requires modifying
`stop` as it can now be called after we've started the browser, but
before we've created the websocket connection.

Also move `--no-first-run` from the headless to the default switches
to avoid Chrome's migration & default browser popup, apparently it
doesn't cause Chromium grief anymore (???). If this turns out to be a
concern, add a condition on the `executable` or something.

closes odoo/odoo#122117

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-05-23 18:02:21 +02:00
Xavier Morel ce77b9f673 [IMP] core: avoid searching chrome binary once per browse_js
Delegate the property's resolution to a top-level function,
cached (with a single slot). This is slightly less efficient than
using some sort of global set once, but it's much easier.

Unlikely to be a huge issue in the face of *starting an entire
browser*, but may as well skip it.

closes odoo/odoo#111422

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-05-19 10:18:48 +02:00
Xavier Morel e6fd0ad5ee [IMP] core: cleanup browser API
It doesn't need to store so much crap as attributes, and the data flow
of the init can be made more explicit.

- have setup functions return stuff, which can be set on `self` by the
  `__init__`, making the dataflow more explicit
- maintain a high-level `Popen` object instead of moving a `pid` around
- pass items like browser size as parameters instead of attributes
- remove redundant attributes like `screencasts_frame_dir` (/ convert
  to properties)

Part-of: odoo/odoo#111422
2023-05-19 10:18:48 +02:00
Xavier Morel 2b0d9fa6a9 [ADD] core: ability to run tours in a regular (headful) browser
Chrome's screencast / remote view (in the remote devtools) is
apparently *designed* for touch emulation /
simulation (https://crbug.com/1410433), which is not convenient when
trying to debug mouse-bound issues in a tour (e.g. drag&drop
problems).

However we can "simply" run the thing in a normal browser, on a watch
basis, and shut down the entire thing after each tour-call. This also
obviates the need for complicated cleanup steps to try and isolate
tours (as we can just delete the profile the browser created),
although it is somewhat less efficient as we keep starting and
stopping the browser.

It does simplify the result of merging `clear` and `stop` (into stop).

The switches setup does have a few foibles:

- `--no-first-run` causes tours to not run at all on my machine, in a
  non-headless browser, so it's left just for headless
- also moved a bunch of other flags which seem to be mostly for
  automated annoyances to headless only
- left extensions for now, not entirely sure which is the right one,
  since we're running with fresh profiles I would assume there's no
  extensions anyway

Part-of: odoo/odoo#111422
2023-05-19 10:18:47 +02:00
Xavier Morel faabf62a39 [FIX] account_peppol: incorrect view extension
`account_peppol` hooks its addition of `option_peppol` in the invoice
sending form to `option_send_by_post`.

However `option_send_by_post` is added by `snailmail_account` which
`account_peppol` does *not* depend on. And while it's likely
`snailmail_account` has long been installed when `account_peppol` gets
installed there's no actual guarantee, and `snailmail` could even have
been uninstalled.

Which is exactly the issue, when uninstalling `snailmail_account` or
any of its dependencies which don't uninstall `account_peppol`
(`snailmail`, `iap_mail`, `iap`) the "send & print" (send invoice)
wizard is broken.

Fix by re-hooking the view extension on `option_send_mail` instead,
that is installed by an actual dependency of `account_peppol`, and
part of the view which `account_peppol` actually extends
(`account.account_move_send_form`).

closes odoo/odoo#121522

Related: odoo/enterprise#41124
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-05-16 15:55:38 +02:00
Xavier Morel d9a3206a16 [FIX] hr_holidays: cancel leaves when they get deleted
When validating leaves they generate a bunch of ancillary
objects (calendar events, resource calendar leaves).

When deleting leaves, those don't get removed, which is an issue when
uninstalling then reinstalling the module: because leaves use the
resource calendar to check availability, and how many working days the
leave covers, and because leaves must cover at least 1 working day
it's possible for the leftover calendar leaves to prevent creating hr
leaves.

Solve the problem by canceling the leave when it is deleted. This
should not be too much of an issue when deleting a leave normally as
`_unlink_if_correct_states` prevents deleting validated
leaves.

Sending the notifications is suppressed during uninstallation (seems
like most people involved would be aware the "time off" module is
getting removed), but for the case of a manual deletion it is probably
useful, especially as notifications are only sent for leaves in
`validate` and `validate1` states.

Part-of: odoo/odoo#121522
2023-05-16 15:55:38 +02:00
Xavier Morel 84ad30338d [FIX] hr_contract: demo data to fix uninstall
On install, `hr_work_entry_contract` only associates work entries to
contacts which are open or closed.

However during its execution (?) `hr_work_entry_contract` generates
entries associated with `Mitchell Admin Contract`, which is a draft
contract. As a result, when uninstalling then reinstalling
`hr_work_entry_contract` it is not able to re-associate the entries to
the contract, and thus can't reinstate the `required=True` on
`HrWorkEntry.contract_id` either, which is a "reinstallation failure"
on the CI.

A simple solution is to create the contract closed, though it would
also be a good idea to not be able to create work entries associated
with a draft contract either, maybe?

X-original-commit: 6b7f3f6ec40eb819dd1d94a94722610a63d5697a
Part-of: odoo/odoo#121522
2023-05-16 15:55:38 +02:00
Xavier Morel 7c78bb5e5c [FIX] base: uninstallation of project
Since the "dirty flag" refactoring of
384fda2c2a
`IrModelFields._prepare_update` did not cope well with fields missing
from the python-side models, which can during uninstallation for
custom fields (possibly because the script loads the registry
incorrectly, not entirely clear).

Because of a custom field created by worksheet linking to it, the
removal of the `project.task` table would fail, making the
reinstallation of project fail to restore several constraints.

This case was actually handled correctly just a few lines above when
trying to resolve field dependencies, both record and field would be
checked for their presence before actually trying to use them.

Getting the model from the registry / environment has not been noticed
to break uninstallations, but might as well do that too so everything
lines up, and just in case.

X-original-commit: 05aca6ee4ce795b85b90430efd761f0cd3a6d5a3
Part-of: odoo/odoo#121522
2023-05-16 15:55:37 +02:00
Xavier Morel 390384ecbb [FIX] core: handle recursion error when resolving stored fields
Issue discovered in the uninstall (and reinstall) of sale_project: a
dump has ~100 tasks, when reinstalling `sale_line_id` has to be
initialised, this is done by marking `sale_line_id` on all extant
tasks as to-recompute, which triggers their computation on the next
`flush`.

Because it's a recursive field, `Field.recompute` ensures only one
record at a time gets recomputed (as there could be cross-dependencies
in the recorset which protection would prevent from resolving).

As the field computation runs, it accesses itself, which triggers a
cache miss, which triggers a `_fetch_field` (to get the currently
stored value), this calls `_read`, which flushes the field we're
trying to read.

The problem here is that for efficiency the cache miss will look for
all records in the cache without a value for the
field (`_in_cache_without`) and try to `fetch` on them as well. This
means rather than not doing anything in flush, we're going to
`Field.recompute` on all records except the one selected the first
time around, which repeats the cycle until there is no more additional
record found in `_in_cache_without`, which could trigger the next
round of `recompute`, and the entire thing unwinds, and we probably
perform a ton of unnecessary additional `compute_value`.

Except that doesn't even happen, because the process from one compute
to the next takes 12~13 stack frames, which given the default
recursion limit of 1000 gives a hard limit of 76 fields before hitting
a RecursionError. As this is less than 100, a recursion error [is what
we get](https://runbot.odoo.com/runbot/build/31726625).

In 15.2, this was fixed by only expanding the fetch on non-recursive
fields, pessimizing recursive
fields (5c2511115b14299516fce4aa3737a62faaf5b653). Test-wise this only
impacted mail performances and in a relatively minor manner.

In 16.0, the mail tests actually match already (so that part was
skipped by the cherrypicking) however this impacts the knowledge perf
tests much more significantly e.g. `test_article_creation_multi_roots`
gets +9 queries when creating 10 top-level articles, which is a bit
much.

So use an alternative which is ugly as hell but which I didn't
consider for 15.2 (may want to backport it one day if the current fix
is an issue): catch the recursion error and use the existing
fallback (of fetching just the requested record's field without
expanding the recordset).

This likely makes for a pretty inefficient situation in the original
case as we're certainly going to hit the recursion limit repeatedly,
but that still fixes the issue, and it avoids deoptimising cases which
fall short of the recursion limit (resolving under 60 records or
so).

Plus despite creating giant stacks we might actually get good
efficiency as we're going to hit recursion limits repeatedly but
that's pure python, once we fall below the limit we can resolve
everything at once with a single SQL query (or something along those
lines).

X-original-commit: 9e71094582ec4c9b719431e77538da8f91ffa9e3
Part-of: odoo/odoo#121522
2023-05-16 15:55:37 +02:00
Xavier Morel 95fa9fadb8 [FIX] base, crm: uninstallation
Uninstallation does not cope well with `setup_models` being performed
unconditionally as those will dramatically alter registry states, and
resurrect computes which the uninstallation has disabled: rather than
try to update registry models in-place (which is rather fraught) the
uninstallation deletes the columns, tables, and `ir.*` reflection
records and only after all of that is done does it reset the registry.

This means while it does fix up the registry caches (`field_depends`
and `field_triggers`) as it goes, resetting those may cause the
recomputation of fields whose columns have been deleted, possibly
based on dependencies whose columns have also been deleted.

As such these kinds of manipulations should either be performed in
`@ondelete` methods which don't get executed during uninstallation, or
they should be gated behind an uninstallation check.

In crm the latter is necessary, as `ondelete` runs before `unlink`
actually executes, and the registry reset would run too early (and
unnecessarily).

In base, only the latter is possible as we're not in `unlink` itself,
instead `IrModelFields._prepare_update` is called *during*
uninstallation and its trailing `setup_models` causes the issue.

X-original-commit: 357b9f2c9fd44e14e5b7c9d3c17f1794691986f3
Part-of: odoo/odoo#121522
2023-05-16 15:55:37 +02:00
Xavier Morel 555386706f [FIX] mail: uninstallation
When modules get uninstalled, first the uninstall process will drop
all the fields (removing all the columns) then it drops all the
models (removing the tables).

When uninstalling mail, this means the various (res_)model(_id) fields
don't exist anymore by the time we're deleting models, so the queries
blow up.

Skip this step if we're unlinking the mail models, it means the tables
have already been dropped, so there's nothing to delete anymore. This
should not use `ondelete` because we *do* want to delete records from
those tables when deleting modules which depend on mail, and thus have
mail stuff associated with their own models which we're deleting.

X-original-commit: e43155f940c1f0ba30378d110fc371012d791e32
Part-of: odoo/odoo#121522
2023-05-16 15:55:36 +02:00
Xavier Morel 05dc244561 [FIX] core: flush after every uninstall hook
Confusion between uninstall hooks can apparently trigger errors during
uninstallation as two hooks can confuse one another?

In this here case, the issue triggered during the uninstall hook of
`account_accountant`, which apparently combines with the uninstall
hook of `industry_fsm_sale` to trigger an invalid in-memory state for
`project_project`. An implicit flush during the hook then blows up
with a check constraint error.

Flushing at the end of the `industry_fsm_sale` hook or at the start of
the `account_accountant` hook fixes the issue, so might as well flush
after each hook to ensure whatever they did using models is pushed to
the database and in good shape (hopefully).

X-original-commit: b28e9a7066d29d395421a11df2b9e170fb20d35a
Part-of: odoo/odoo#121522
2023-05-16 15:55:36 +02:00
Xavier Morel 9f2cfb2cdf [IMP] test_module_operations: make uninstall step more graceful
- process modules to uninstall individually in order to better handle
  their state at that point
- uninstall (and reinstall) modules in provided order, rather than
  whatever postgres feels like (or a sort which might not match what
  we want), mostly useful when uninstalling modules in bulk
- warn if a module is either missing or already uninstalled, rather
  than silently do nothing

closes odoo/odoo#120261

X-original-commit: 034b317a908d1aea6dc6d992c489b30ffff8f1ce
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2023-05-05 18:08:00 +02:00
Xavier Morel 44f7ccc100 [IMP] test_module_operations: logging configuration
- switch to `dictConfig` for easier bulk-manipulation of loggers
- set `unlink` and `ir_model` loggers to warning, to limit the
  humongous spam that is a normal uninstall session

X-original-commit: bee7898a5af68128b599d30ac7d4c0193e765afd
Part-of: odoo/odoo#120261
2023-05-05 18:07:59 +02:00
Xavier Morel d5a3e3e794 [IMP] module_operation_tester: use subcommands
Add subcommands to make running the script clearer (as exclusive
switches is a bit weird nowadays).

Keep the old `--uninstall` and `--standalone` switches, but make them
mutually exclusive (and optional) to retain current behaviour.

X-original-commit: 933841eda86292f180170b32da621f55fe6f0b84
Part-of: odoo/odoo#120261
2023-05-05 18:07:59 +02:00
Xavier Morel f2de38b84a [IMP] test_module_operations: make debugging easier
- ensure `test_module_operations` exits with a non-zero status on
  failure, as the current makes it a lot less convenient to notice
  uninstall / reinstall errors (especially with lots of warnings
  crowding the logs)
- allow uninstalling without reinstalling, so it's easier to inspect
  db state after uninstall

X-original-commit: 712977faf9bf98d9087368ab4fe95f089a072601
Part-of: odoo/odoo#120261
2023-05-05 18:07:59 +02:00
Xavier Morel c7826675a8 [FIX] base: restore deletion of dependencies on field removal
odoo/odoo#111651 improved and optimised triggers but dropped the
in-place cleanup of field dependencies. As a consequence, during
module uninstallation if a stored computed field is removed (because
it's part of a module being uninstalled), and one of its dependencies
is subsequently altered (e.g. it's itself removed, or written to) the
second update will break as the query trying to find out which
dependent records to update will error, either because of trying to
select / filter on a missing column, or because of trying to fetch
in a missing table.

The simplest examples of this issue are computed fields with a
dependency on `ir.model`:

- In `calendar`, `calendar.event.res_model` is a related on
  `res_model_id.model`, this prevents the removal of *any* `ir.model`
  record if it gets uninstalled.

  As a result the `calendar.attendee` and `calendar.event` tables
  don't get removed (just emptied of all their non-automatic fields),
  their records remain as well, and when the non-automatic fields get
  re-added during installation re-instating the NOT NULL constraints
  fails, breaking the uninstall/reinstall test.

- In `payment`, `payment.provider.module_state` is a related on
  `module_id.state`, this breaks *during* uninstallation, as after
  `module_uninstall` first calls `_module_data_uninstall` which
  removes the field, then it *updates the modules being uninstalled*
  (sets their state), which tries to find out which
  `payment.provider`'s `module_state` is should update, which breaks
  because the `module_id` column has been removed.

  This second one was worked around in odoo/odoo#118900, by marking
  modules as uninstalled before actually gutting them, but as it turns
  out the "actual" fix is needed anyway. So revert the workaround, and
  actually fix the issue.

closes odoo/odoo#119130

X-original-commit: 48a420efcf7c7b46416bab006003553cc9d23846
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-20 09:33:48 +02:00
Xavier Morel 98b5505305 [FIX] base: mark modules as uninstalled before gutting them
Before this, uninstalling the `payment` module (or any of its
dependencies) is broken: as payment.provider has a dependency on
ir.module.module.state, marking the modules causes a lookup of the
payment provides to update, but the table was removed by
`_module_data_uninstall`, so the lookup blows up.

This is a consequence of odoo/odoo#111651 which improved and optimised
triggers but dropped the in-place cleanup of the triggers tree.

Thus while the columns & tables get removed from the database the
in-memory structures (registry, models, fields, ..., as well as the
trigger and dependency caches) are not so the python side will happily
try to look up stuff which has been nuked if accessed at the wrong
moment (which is any moment between the start of
`_module_data_uninstall` and the creation of a new registry, really).

As `_module_data_uninstall` is nothing but a giant pile of dodgy state
anyway, making modules as uninstalled before it executes doesn't seem
like a huge deal. It may cause unnecessary extra recomputation for the
few models which depend on modules, but that doesn't seem like a major
issue, at worst it makes uninstallation a touch slower but they're not
a huge performance concern at the moment (they're more of a
correctness one).

closes odoo/odoo#118959

X-original-commit: 460efeb623ec62c980d187710bd4e7614af0e7bd
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-19 12:13:32 +02:00
Xavier Morel 215ccd1194 [FIX] website_slides_survey: fix uninstall_hook
When #114353 forward port of #112425 which adds an uninstall_hook was
merged, it was not updated with the API change of #108254, which
replaced the `(cr, registry)` parameters by a sole `(env)` as most if
not all uninstall hooks immediately created an environment anyway.

So this hook has been breaking uninstall on anything on which
`website_slides_survey` depends since it was merged.

Fix the hook to match the new API.

closes odoo/odoo#118296

X-original-commit: 201a4b369cf1652a2da763181f6837cfb7f44f3e
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-13 04:08:35 +02:00
Xavier Morel c2848b9bb4 [IMP] point_of_sale: modernize get_price
- Remove dead code around now-unused categories
- lift `price` processing outside of `find` which complicates it
- use arrow functions, which obviates the need for aliasing `this`
- use slightly more expression-oriented code-style where applicable

Results in code which is noticeably shorter and (I hope) easier to follow.

closes odoo/odoo#117874

Signed-off-by: Joseph Caburnay (jcb) <jcb@odoo.com>
2023-04-13 00:06:00 +02:00
Xavier Morel e702e3a776 [IMP] point_of_sale: simplify can_be_merged_with code
Existing version uses a visually complicated chain of if/else and
double negations, this is much more readable as a simple chain of
AND-ed conditions checking for positive merge-ability factors.

Part-of: odoo/odoo#117874
2023-04-13 00:06:00 +02:00
Xavier Morel cedc968f22 [FIX] base: better filter drop table warnings
Along with improving the table_kind API, #117439 added a warning when
dropping non-tables.

This warning turns out to trigger on many models, in two major cases:

- the hook is called on abstract models, which don't have a table in
  the first place, it probably should not be called for those but can
  easily be skipped
- the hook is also called on `_auto = False` which don't have a table
  anymore, this is because we use `DROP TABLE $table CASCADE`, which
  implicitly drops any view (or table) depending on the table, it
  might be possible to avoid this by ensuring we drop models in
  reverse topological order *and* drop all of a module's views then
  tables at once (by passing multiple names to `DROP`, but for now
  just ignore the case where we're trying to drop an object which
  can't be found

closes odoo/odoo#118194

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-11 15:17:32 +02:00
Xavier Morel 8932d447aa [FIX] core: table_kind semantics (incorrect classification of toast as temp)
17c4f47b0a updated table_kind to return
`pg_class.relkind`, however the semantics are *not* the same, and that
was ignored: `relkind = t` is for *toast* tables, not *temporary*
tables. In `pg_class` the temporary-ness is instead signaled by
`relpersistence` (which applies to both tables and sequences), temp
(and unlogged) tables have `relkind = r`. `existing_tables` does that
correctly, possibly unwittingly.

While at it, upgrade `table_kind` to return an `enum` (whose value is
the old discriminant).

closes odoo/odoo#117444

Related: odoo/enterprise#39185
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-07 11:37:25 +02:00
Xavier Morel b799ef34ae [FIX] web: disable scripting & autoprint in PDF viewer
Support is limited and more of a hindrance than a help, as some
clients' suppliers create PDFs with broken scripting which trigger the
autoprinting any time they're loaded and can have other odd
misbehaviors, which is a hassle.

OPW-3208409
OPW-3222228

closes odoo/odoo#115659

X-original-commit: a7331d9709c8e097356f89bf144986f441c8f2cc
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-03-17 20:21:09 +01:00
Xavier Morel e37109c8c3 [REM] core: support for werkzeug interactive debugger
With the special support for postmortem debugging removed, the
likelihood of needing / wanting the werkzeug remote debugger seems
even more remote (as it works in strictly less situations, only for
frontend non-json requests).

So remove that as well.

closes odoo/odoo#115176

X-original-commit: a2022783b652299155c460294c00dbced9b619ac
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-03-14 13:30:19 +01:00
Xavier Morel 09023b6f87 [REM] core: remnants of debugger support
It's done nothing since #78857 and it doesn't seem like anyone has
cared (found no issues or tickets).

Rather than restore the feature, just remove the leftover bits.

X-original-commit: 4a7cfc8844eb9b754f16f9d013452d5bde8770b5
Part-of: odoo/odoo#115176
2023-03-14 13:30:19 +01:00
Xavier Morel 18ebbce51b [FIX] core: ability to run tours in Chrome 111
Chrome 111 enabled checking of websocket origin: if the WS connection
sends an Origin head which is not whitelisted with the new
`--remote-allow-origins` switch it is rejected.

Turns out websocket-client (amongst others) *does* send an `Origin`,
which trips the check, and means tours immediately break when trying
to run them as Odoo's test harness is unable to connect to (and
control) the devtools.

Suppress sending `Origin` to fix the issue.

To make the watch mode work, set `--remote-allow-origins`: since we
specifically only bind the devtools to the loopback
address (127.0.0.1) whatever issues this plugs are unlikely to affect
us. We might eventually want to change the behaviour of the watch
feature for UX reasons and remove this in the future though, either by
working through the non-ws remote inspection (`chrome://inspect`) or
by having the `watch` mode run in a normal browser directly instead of
having a browser connect to a headless browser.

Chrome 111 changeset: https://chromiumdash.appspot.com/commit/0154caeefc74530d5cb57ce71608beb1b77bca39

Chrome tracker issue: https://crbug.com/1422444

closes odoo/odoo#115067

X-original-commit: 47a02b3924c3e4d1690e336fdd764383393ee378
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-03-14 11:08:07 +01:00
Xavier Morel eb5515bee9 [ADD] core: support for binding IPv6 in worker mode
Because Werkzeug supports ipv6 natively (and has since ~2010
pallets/werkzeug@13a76262a4), running
Odoo in threaded mode allows binding to an ipv6 address.

However the prefork server is part of Odoo, and only creates
AF_INET (ipv4) sockets. Thus switching from threaded to worker mode
breaks the bind.

Add a family switch on the `http-interface`, so binding to ipv6
addresses also works in ipv6. Gevent already does that switching
internally, so the evented worker works out of the box, it was only
the prefork / workers which did not.

Looking at both the Werkzeug and Gevent implementations:

- https://github.com/pallets/werkzeug/blob/f9906fa6d83dd668f9214acef458419756bbc062/src/werkzeug/serving.py#L607-L614
- https://github.com/gevent/gevent/blob/1e412d35526183b26c1abf2eb658cbef661f5f70/src/gevent/baseserver.py#L415-L433

Both primarily check whether the host contains `:`. Werkzeug also
checks if `socket.AF_INET6` exists, however gevent doesn't bother with
that. Looking at the
source (https://github.com/python/cpython/blob/7d801f245e2021d19daff105ce722f22aa844391/Modules/socketmodule.c#L7419-L7421),
it is indeed possible for `AF_INET6` to be missing, however that
requires specifically compiling Python in an environment which doesn't
support IPv6.

Meanwhile it's also possible to disable ipv6 at runtime, in which case
I'd assume the `socket.AF_INET6` constant is present, and creating the
socket fails, which nobody guards against. Therefore just ignore the
entire thing, it doesn't seem worth the hassle (or the questioning) to
check whether the constant is present, especially since this is only a
concern when the user specifically requires an IPv6 address.

Fixes #35782

closes odoo/odoo#114666

Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
2023-03-08 13:49:05 +01:00
Xavier Morel 0d5b9d2030 [IMP] core: increase FileStorage buffer size
Closes odoo/odoo#83176

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-01-23 09:32:06 +01:00
Xavier Morel d1a0ac989f [FW][FIX] core: misleading handle_error docstring
`handle_error`'s docstring states that it returns a `Response`, but in most case `HttpDispatcher.handle_error` returns an `HTTPException`.

After discussion, the implementation is correct, `handle_error` should be documented to return a WSGI Application (a callable taking an `environ` and a `start_response` callable) instead.

closes odoo/odoo#112889

Forward-port-of: odoo/odoo#112690
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-02-16 16:58:49 +01:00
Xavier Morel 298567f64c [FIX] core: disable werkzeug log color when logging to file
Werkzeug 1.0 colorised some outputs on POSIX IFF `click` was
installed.

Since 2.0 (pallets/werkzeug#2012) werkzeug unconditionally colorises
the log on POSIX. This is annoying when using output redirection (let
alone logging to a non-stream), as werkzeug will dump ANSI color codes
to the non-term stdout and thus the logfile.

Werkzeug provides no official knob to control this behaviour, but it
does have a secret flag which is normally used to check if colorama is
available on windows (so the ANSI codes are not output if colorama
won't be interpreting and stripping them on the way out). Since
`werkzeug.serving` is available in pretty much all versions, we can
just (un)set this flag if not logging to a tty, and versions 2+ should
pick it up and disable colorisation.

closes odoo/odoo#112829

X-original-commit: 7c9f883dc508a7a8a45bf7bf7e900da0be9b34be
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-02-16 09:01:10 +01:00
Xavier Morel a6d601dc4e [FIX] test_lint: deprecation warnings in pylint 3
Runbot currently doesn't use Pylint 3 in order to still have "style"
lints (since removed), however compatibility with pylint 3 is useful
to run tests / lints locally (and possibly eventually in future python
versions for which 2 might not be compatible).

Fix a few deprecation warnings:

- the `__implements__` magic thing has been deprecated
- `check_messages` has been renamed to
  `only_required_for_messages` (better explains the purpose)

Also remove second parameter of `is_message_enabled` call, if
specified it's supposed to be a `Confidence` value, not a
number. Recent pylints changed the way it's checked, so it now errors
even if not using the confidence system (or something like that).

closes odoo/odoo#107960

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-02-03 05:09:22 +01:00
Xavier Morel 6e700157d0 [REM] core: odoo.modules.module.loaded flag
Seems unnecessary, most of the information already lives in
`sys.modules`. We can just check that.

There are a few changes in behaviour, but they seem minor:

- if post_load fails, subsequent attempts to load the module will
  "succeed"
- since we didn't remove/reload the module, a failure because of an
  incorrect post_load wasn't fixable, however it was possible to
  update the manifest

Still seems like a wonky state to be in, and one we should ignore.

Also remove the logging of the error: since we're re-raising as-is,
the parent logs it with a traceback, so this is unnecessary and
redundant.

closes odoo/odoo#103933

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-02-03 05:08:40 +01:00
Xavier Morel 98dd7aba2f [FIX] *: used-before-assignment pylint warnings
Warnings show up when using a recent pylint. It's only a fraction of
what e.g. pycharm flags as "local variable might be referenced before
assignment" but seems a good idea to fix anyway in prevision of
possibly eventually updating the reference pylint.

closes odoo/odoo#107968

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-01-31 12:55:49 +01:00
Xavier Morel b951cb4422 [IMP] core: mark all responses as nosniff
Doesn't seem like it could hurt, and should only make CORB more
reliable by avoiding sniffing. Worst case scenario requires fixing a
few mimetypes, but aside from CORB it looks like modern (non-IE)
browsers only try to guess document mimetypes in very limited
contexts.

closes odoo/odoo#107957

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-01-31 12:55:46 +01:00
Xavier Morel d4f8f7ece6 [IMP] core: handling of the form tour checker
`DOM.querySelector` will generally return a `nodeId` of 0 when no node
is found, however there is a small window during which the document
can apparently get collected (?), which leads to an error

    Could not find node with given id

which can break otherwise successful builds (cf 20832402 / staging
60731).

Handle errors from the pipeline as if the node had not been found,
though the matter was not fully investigated so it's possible this
explanation is incomplete or incorrect.

The CDTP documentation does not document the failure modes for either
`DOM.getDocument` or `DOM.querySelector`.

closes odoo/odoo#105734

X-original-commit: 5ef975f75cf8044e7dcd67f403ca11058630c85a
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-11-15 11:38:43 +01:00
Xavier Morel 5d47883262 [REM] core: dead code for JSONRPC HTTP status
When `make_json_response` was added in
034d01b2f3 and `_response` was updated
to use it, the `http_status` extracted from the error object was
removed.

But it's still set by `handle_error` and a local is defined for
it. Drop that.

closes odoo/odoo#104507

X-original-commit: b0ea1cd6a2628a6acd1e2d79cc862976c63b1565
Signed-off-by: Julien Castiaux <juc@odoo.com>
2022-10-28 17:18:10 +02:00
xmo-odoo c9356b3582 [REM] core: SQL compiler helpers deprecated since 14.0
closes odoo/odoo#98138

Related: odoo/enterprise#33244
Related: odoo/documentation#2844
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-10-26 19:03:48 +02:00
xmo-odoo 6a343a32ef [REM] core: deprecated TestCase classes
Part-of: odoo/odoo#98138
2022-10-26 19:03:47 +02:00
xmo-odoo 84e6be4150 [REM] core: deprecated osv-memory-age-limit option
Part-of: odoo/odoo#98138
2022-10-26 19:03:47 +02:00
xmo-odoo 110770b7c1 [REM] core: openerp.addons alias and legacy ad_path
Part-of: odoo/odoo#98138
2022-10-26 19:03:47 +02:00
xmo-odoo e98b44998f [REM] base: legacy autovacuum API (power_on method)
Part-of: odoo/odoo#98138
2022-10-26 19:03:47 +02:00
xmo-odoo d200dcfb2c [REM] core: exceptions bits deprecated since 14.0
And formally deprecate the contents of `odoo.osv.osv` so we can
eventually finally remove it.

Part-of: odoo/odoo#98138
2022-10-26 19:03:46 +02:00
xmo-odoo e806427d23 [REM] web: deprecated dataset methods
Part-of: odoo/odoo#98138
2022-10-26 19:03:46 +02:00
Xavier Morel 18446f23f1 [REM] core: support for the deprecated <report> and <act_window> tags
Part-of: odoo/odoo#98138
2022-10-26 19:03:46 +02:00
Xavier Morel 6c6ca16626 [REM] core: deprecated Environment methods
Part-of: odoo/odoo#98138
2022-10-26 19:03:46 +02:00
xmo-odoo 6dbe95c4ce [REM] base_setup: last reference to gengo
All other references were removed in 481d1393e7
but this was left over "to be removed in master" (the commit landed in 14.1).

The actual module was removed in 14.0 (b38b72e456a) so this is quite overdue.

closes odoo/odoo#100008

Related: odoo/upgrade#3884
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-10-26 14:05:26 +02:00
Xavier Morel 59442f0988 [IMP] web: open external links of PDFs in new tab
Reimplementation of 74fa91e0b3 on top of
new version of pdfjs. See previous commit for explanation.

Ideally we would set that behaviour from the outside using
e.g. `PDFViewerApplicationOptions.set` but there are multiple
locations from which we embed the viewer, which makes that a difficult
proposition. There should probably be a clean component which handles
loading the library, configuring it to our specifications (possibly on
a per-embed basis), and exposing both manipulations methods and events
which pdfjs triggers on its eventBus.

Part-of: odoo/odoo#100067
2022-10-26 09:07:34 +02:00
Xavier Morel 542cb1dc21 [FIX] website_slides: outdated pdfjs API use
Apparently a PDF Document used to *be* a promise (a "thenable") but
that's not the case anymore. The loading promise has to be deref'd
explicitely. This is also the case for `page.render`, which is not a
thenable anymore.

Furthermore, the API seems to have changed to favor parameter
objects (getViewport) and setting callbacks (onPassword) rather than
having lots of positional parameters.

Finally, remove apparently long dead `disableWorker` feature in
`PDFSlidesViewer`, although really the entire thing should be
rewritten in modern javascript.

Part-of: odoo/odoo#100067
2022-10-26 09:07:34 +02:00
Xavier Morel 55d9f318fb [IMP] web: update pdfjs to 2.16.105
Our current version is getting rather outdated. Replace by
the *legacy* bundle: pdfjs now provides a non-polyfilled version of
the library, which is probably faster though it doesn't save overly
much (for the entire bundle anyway). Use polyfilled version for
safety, though it's unclear whether non-chromium Edge is still
supported by Odoo. If not, we could just use the non-polyfilled
version.

The difference is quite large for pdf.js (+32.5%), however it is much
less consequential for the sandbox (+1%) or for the much larger
worker (+5.6%). The biggest difference is likely performances
but... who knows?

Notes:

- The main reason for this change is that automated vulnerability
  scanner have apparently started scanning for `postMessage(..., '*')`
  and the previous bundles includes a version of corejs polyfills
  without zloirock/core-js#542), therefore triggering those scans. As
  the PR notes this is almost certainly not a concern because of the
  innocuous payload, but there is no reason to waste time on those
  reports if we don't *have* to.
- The bundle now includes "standard fonts", those were removed as
  they're heavy and may not be necessary for our usage (?).
- All the bitmap images were dropped and replaced by svgs, which is
  nice.
- The local changes since the previous update were *not* impacted in
  this, the entire thing was just reset to upstream. This means
  changes which were backported (922c7c72, 6943714f) are superseded
  but more odoo-specific changes will have to be reapplied in further
  commits.

Part-of: odoo/odoo#100067
2022-10-26 09:07:34 +02:00
Xavier Morel e2b3463b5a [IMP] core: formally deprecate __openerp__.py manifests.
`__manifest__.py` was introduced in Odoo 10 and quickly migrated
to (c758e9043a fixed the last holdouts).

Deprecate old manifests. People who need the compatibility can just
add a symlink (or duplicate the file if they're working on FAT or an
old os where core.symlinks can't be set to `true`).

closes odoo/odoo#103952

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-10-25 17:16:09 +02:00
Xavier Morel 75adabf11f [ADD] *: examples of Model.grouped
closes odoo/odoo#101522

Related: odoo/documentation#2783
Signed-off-by: Rémy Voet <ryv@odoo.com>
2022-10-21 13:06:09 +02:00
Xavier Morel d50db0587b [ADD] core: ORM-level internal grouping utility
When looking to fetch grouped data from the database, `read_group`
should work(-ish), especially with support for `array_agg`. This also
means this need should be more or less solved around RPC: client can
either `read_group` or group on the client side.

This leaves a glaring hole in the fabric: there's no convenient tools
to apply grouping to recordsets. `itertools.groupby` exists, but it
requires that the input be grouped by the key function, and it yields
an iterable of items which is not the most convenient for recordsets.

This here utility:

- is exclusive to recordsets
- returns a mapping of grouping keys to subsets of the input recordset
- keeps the prefetching of the source recordset
- allows grouping on a field, or an arbitrary (callable) key
- should work even on "new records" (aka should be suitable for
  onchanges / arbitrary compute functions)

While the method is RPC-compatible, it's not especially designed for
that, it is likely a much better idea for the client to `read` the
data they need then group that client side on whatever criteria they
are interested in, or `read_group` the aggregated information they
need directly if that's an option.

Part-of: odoo/odoo#101522
2022-10-21 13:06:09 +02:00
Xavier Morel 6d3c78fe30 [FIX] test_sale_product_configurators: re-enable test_01
closes odoo/odoo#103117

X-original-commit: c09b3c94ca6926669faf5df738a271e0e9789ce8
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-10-12 10:34:45 +02:00
202c377975 [FIX] web_tour: discard form helper
The helper's final trigger was hooked not on the list *view* but on
the list *widget/component*.

As a result, on a form containing an x2many field the trigger would
fire immediately, before the form view was actually discarded, leading
to a race condition in the Python-side check for unsaved forms, and
thus non-deterministic tour failures.

Update the check to look for the view specifically, this should be
exclusive with the form view.

X-original-commit: 28c569d7cb0ffeda4cd9abcb8332e96c551cf03a
Part-of: odoo/odoo#103117
Co-authored-by: Michaël Mattiello <mcm@odoo.com>
Co-authored-by: Aaron Bohy <aab@odoo.com>
2022-10-12 10:34:45 +02:00
Xavier Morel 9b891b6bb2 [IMP] core: error reporting on tour timeouts
When 2e8647bf16 converted the browser
runner to a more reactive / evented system, one bit was missed in
"wait_code_ok": concurrent.futures.Future raises exceptions on various
events, such as tour timeouts. Because those exceptions were not
caught (or just ignored) the code which takes screenshots was
bypassed, leading to a lack of screenshots on tour timeouts (and a few
other rarer errors), making debugging more complicated.

The error reporting was also not ideal as `wait_code_ok` would raise
an unexpected (by its caller) `TimeoutError` rather than
`ChromeBrowserException`.

Fix those two issues, should hopefully makes these occurrences clearer
and easier to diagnose.

closes odoo/odoo#102403

X-original-commit: 974217968ea970330946c5184bd3b3550ec3cde3
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-10-06 18:01:27 +02:00
Xavier Morel 482de91bfd [FIX] core: use proper break (hr) in PR template footer
A "thematic break"[1] is composed of 3 or more characters (-, _,
or *). The current separator using only 2 characters between the main
body of the PR and the footer / addendum makes it more difficult to
interpret the message.

Update to 3 so e.g. the mergebot can strip the footer.

[1]: https://spec.commonmark.org/0.30/#thematic-breaks

closes odoo/odoo#101148

X-original-commit: c7bf932c9eb2fc9bc34c653c51bce1552d6afb54
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-09-26 16:55:17 +02:00
Xavier Morel 9bcc30be8d [FIX] auth_password_policy: remove validation in meter
Not sure what I was thinking (there was no chance the validation would
be structural, probably didn't think about the validation itself).

Possibly thought of creating a proper `Recommendation` type then ended
up not doing it, but left the props types in.

closes odoo/odoo#100206

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-09-15 13:38:49 +02:00
Xavier Morel 598492ef4d [FIX] auth_password_policy_signup: port to modern js and fix
b3b85cae9b unwittingly broke
auth_password_policy_signup when it ported everything to owl.

- move the CSS file back to assets_common
- reintroduce the old password gauge, moved over to signup (as it's
  not used anymore by auth_password_policy), and converted to modern
  JS style
- convert signup_policy to more modern JS style (for consistency)

Manifest file doesn't need to be updated because it globs.

closes odoo/odoo#100169

Signed-off-by: Adrien Dieudonné (adr) <adr@odoo.com>
2022-09-15 13:38:46 +02:00
Xavier Morel b3b85cae9b [IMP] *: owlify password meter and convert change password to real wizard
The changes in `auth_password_policy` are largely the owlification of
the password meter widget:

- modernize the password policy module and convert it to an
  odoo-module (note: now exports a pseudo-abstract class which is
  really a policy, for the sake of somewhat sensibly typing
  `recommendations`)
- replace the implementation of the Meter and PasswordField widgets by
  owl versions

The changes to web and base stem from taking a look at converting the
ChangePassword wizard, and finding that it would be a pain in the ass
but also... unnecessary? It seems to have been done as a wizard
completely in javascript despite being backend-only for legacy
reasons: apparently one of the very old web clients (v5 or v6
probably) implemented it as a "native action" which was directly part
of the client's UI, and so it had to be implemented entirely in the
client.

Over time it was moved back into the regular UI (and moved around
quite a bit), hooked as a client action to maintain access to the
existing UI / dialog.

But since it's been an action opened via a button for years it can
just... be a normal wizard, with password fields, which
auth_password_policy can then set the widget of.

So did that:

- removed the old unnecessary JS, and its dedicated endpoint (which is
  *not* used by portal, portal has its own endpoint)
- used check_identity for the "old password check"
- split out `change_password` with an internal bit so we can have a
  safer (and logged) "set user password" without needing to provide
  the old password, which is now used for the bulk password change
  wizard as well
- added a small wizard which just takes a new password (and
  confirmation), for safety a given change password wizard is only
  accessible to their creator (also the wizard is restricted to
  employees though technically it would probably be fine for portal
  users as well)

Rather than extensive messy rewrite / monkeypatching (the original
wizard was 57 LOC, though also 22 LOC of template, the auth_policy
hooking / patching was 33, plus 8 lines of CSS),
`auth_password_policy` just sets the widget of the `new_password`
field in the new wizard, much as it did the bulk wizard.

Also improve the "hide meter if field is empty" feature by leveraging
`:placeholder-shown`. This requires setting a placeholder, and while
empty works fine in firefox, it doesn't work in chrome. So the
placeholder needs to be a single space. Still, seems better than
updating a fake attribute or manipulating a class for the sake of
trivial styling.

Notes on unlink + transient vacuum

Although the wizard object is only created when actually calling
`change_password`, and is deleted on success, it is possible for the
user to get an error and fail to continue (it should be unlikely
without overrides since the passwords are checked while creating /
saving but...).

While in that case the `new_password` in the database is not the
user's own, it could be their *future* password, or give evidence as
to their password-creation scheme, or some other signal useful to
attack that front of the user's life and behavior. As such, quickly
removing leftovers from the database (by setting a very low transient
lifetime) seems like a good idea.

This is compounded by the `check_identity` having a grace period of 10
minutes. 0.1 is 6 minutes, but because the cron runs every 10 the user
effectively has 6~10 minutes between the moment they create an
incorrect / incomplete version of the wizard and the moment where it
is destroyed if they just leave it.

closes odoo/odoo#99458

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-09-08 18:31:18 +02:00
Xavier Morel 5afcd06168 [REM] *: incorrect taggings which break tests when applied
Not entirely sure about TestAllocationRights. For TestEsEdiCommon
issue is quite obviously that it's inherited by tests which are
external, so when the `post_install_l10n` tag gets applied those tests
get run during "normal" l10n and they break.

closes odoo/odoo#98814

Related: odoo/enterprise#30825
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-09-05 08:33:13 +02:00