Explicit flushing of the cursor had been implemented in the "legacy"
`url_open` helper, but it was missing when going through `requests`
directly (via the `opener`), or when performing XML-RPC calls.
Fix that:
* extend `requests.Session` and `xmlrpc.client.Transport` so they take
a cursor
* move the setup of the XML-RPC clients to `setUp` so they can *get*
the cursor
* move the rest of `HttpCase.__init__` to `setUpClass` and drop the
override entirely
* remove the now-redundant flush in `url_open` (as it's done by the
`opener`)
closesodoo/odoo#77359
X-original-commit: c7bdae34690f31960d1fa95b5c3b4052540f5d07
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
This fixes a performance issue: ORDER BY clauses in subqueries can make
the query unexpectedly slow. We should avoid this situation, since
ORM-generated queries have an ORDER BY clause which is not relevant in
the context of a subquery.
The following example was found:
SELECT "pos_payment"."id" AS "id"
FROM "pos_payment"
WHERE ("pos_payment"."pos_order_id" in
(SELECT "pos_order".id
FROM "pos_order"
WHERE ("pos_order"."company_id" in (1))
ORDER BY "pos_order"."id"))
AND "pos_payment".id IN (1285508)
Here are the query plans made by PostgreSQL on this query with and
without the ORDER BY clause. The query time went from 1240ms to
0.402ms, which is 3000 times faster!
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order"."company_id" in (1)) ORDER BY "pos_order"."id" )) AND "pos_payment".id IN (1285508);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Merge Semi Join (cost=2.88..82726.85 rows=1 width=4) (actual time=1239.361..1239.364 rows=1 loops=1)
Merge Cond: (pos_payment.pos_order_id = pos_order.id)
-> Sort (cost=2.46..2.46 rows=1 width=8) (actual time=0.021..0.022 rows=1 loops=1)
Sort Key: pos_payment.pos_order_id
Sort Method: quicksort Memory: 25kB
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.014..0.015 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..66770.53 rows=1282120 width=4) (actual time=0.013..1148.194 rows=1182463 loops=1)
Filter: (company_id = 1)
Planning time: 0.272 ms
Execution time: 1239.396 ms
(11 rows)
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order
"."company_id" in (1)) )) AND "pos_payment".id IN (1285508);
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.85..4.89 rows=1 width=4) (actual time=0.047..0.049 rows=1 loops=1)
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.027..0.028 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..2.45 rows=1 width=4) (actual time=0.018..0.018 rows=1 loops=1)
Index Cond: (id = pos_payment.pos_order_id)
Filter: (company_id = 1)
Planning time: 0.322 ms
Execution time: 0.080 ms
(8 rows)
closesodoo/odoo#77357
X-original-commit: 947c3bd72ee84c3a298e0c9c365340ddf16b307d
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Stanislas Sobieski (sts@odoo.com)
The registry attributes 'registry_invalidated' and 'cache_invalidated'
are used to flag that the current request has modified the registry or
invalidated the ormcache, respectively. This provides a simple yet
efficient way to signal registry changes or cache invalidations to other
workers.
However, those flags were not meant to be used with multi-threaded
workers. For instance, a thread may signal registry changes that are
actually made by another thread. It can also happen that a thread
changes the registry, which makes another thread crash (like a thread
modifying a dict while another one iterates over it), and the latter
will reset the registry to its original state because it misinterprets
the registry changes as its own changes.
The situation can even get worse, making threads crash in cascade and
eventually leaving the registry in an inconsistent state. When this
happens, the worker is broken and has to be manually restarted.
The fix consists in making those flags thread-specific. This does not
prevent thread crashing because of concurrent changes, but at least it
avoids leaving the worker in a broken state.
closesodoo/odoo#77273
X-original-commit: 28adbfa5a9df9b7754529d45188c0eedeffdf783
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Magic return image/svg instead of image/svg+xml if file starts with <svg
+ refactor the import/else to if/else to avoid pylint error:
function already defined line 137 (E0102) at odoo/odoo/tools/mimetypes.py:181
That raises an error during CI/runbot
closesodoo/odoo#77193
X-original-commit: b76f13853a25a93307db2bcafb6a83ad75b92244
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
This pr proposes an auto retry mechanism for tests. This shouldn't
impact normal testing: tests are not supposed to fail, but the growing
number of tests and pull requests can lead to some bottleneck when a
staging fails because of a random error. This mechanism should help to
reduce splits/the need to retry a failed pr.
This branch have been tested with the nightly multi build, creating 40
identical build without test-tags to disable know random errors.
This multi build is used to detect test failing randomly, this is an
excellent candidate to detect the effect of the retry.
On average, with the current base of this pull request, there is between
10 en 15 failures over 40 build.
With the auto retry mechanism, only 1 build failed over 40 builds since
the same error was triggered twice.
This is simply because with the retry mechanism, an error that has a
probability of p to fail randomly will still have a probability of p² to
fail with the retry mechanism. A error that occurs 10% of the time
should only appear 1% of the time with one retry. In most of the case,
the retry is sucessfull: https://runbot.odoo.com/runbot/build/10053257
The current solution to allow to enable this mechanism only in some
cases (staging) is to check an environment variable
"ODOO_TEST_FAILURE_RETRIES" that defines a number of retry.
This will allow to retry more than once if an error still occurs to ofen
with the autoretry.
The mechanism will run multiple time the same test on the same
test_case, meaning that some modification on self may impact the second
execution. The following code is an example of how this could be
problematic, but also a good example to test the auto-retry mechanism.
```python
class TestRetry(HttpCase):
def test_fail(self):
self.t = getattr(self, 't', 0) + 1
if True or self.t == 1:
import logging
_logger = logging.getLogger('test_a')
with self.assertLogs(level="ERROR"):
_logger.error("This shouldn't be log at all")
with mute_logger('test_a'):
_logger.error("This shouldn't be logged (mute)")
_logger.error("This should be log")
```
As we can see here the error logs are also managed, and emit at a lower level the first time, butany log higher than 25 will make the test "failed" and the autoretry mechanism will be triggered. The second time, everything is logged normally. We also need to replace Traceback by _Traceback to avoid being catched by runbot Traceback detection regexes.
The inspiration here commes from the assertLogs, that replace all handlers. The mute_logger had to be adapted to use the same strategy, so that quite_logger won't detect logs catched by mute_logger or assertLogs.
closesodoo/odoo#76336
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When merging partners through the ``base.partner.merge.automatic.wizard``
wizard, messages and followers are merged. However activities are not
probably because its underlying model is "newer" then messages and followers.
This commits fixes that behavior.
Task-2641572
PR odoo#76159
Closes#71654
X-original-commit: 7e17678ec6ebbbe37b807a5d264cc5afd46bae1a
Part-of: odoo/odoo#77005
Co-authored-by: Thibault Delavallee <tde@odoo.com>
Since
https://github.com/odoo/odoo/commit/8a3e1a0ccdad25ba5b4b99639bdaeb9073a27f1f,
`_update_user_groups_view` is called only once per module installation. However,
module installation is not the only scenario when data files are loaded and
hence we need to add the method call.
---
opw-2602541
closesodoo/odoo#76869
X-original-commit: a46d20c89c24a4e60faef09cf8ad7721bd29d6cc
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Suppose the TZ is UTC+12 and a user creates a new rate in the morning:
the date will be the day before
The default date should consider the user's timezone.
OPW-2590972
closesodoo/odoo#76914
X-original-commit: 94eb684d83c3afb717f1a548ef02b4449e0db959
Signed-off-by: Quentin De Paoli (qdp) <qdp@openerp.com>
Signed-off-by: Adrien Widart <adwid@users.noreply.github.com>
Because the field is stored it doesn't make much sense to have a
context dependency and that can cause significant issues. However
`name_get` looks pretty innocent and it's not necessarily insane to
have a context dependency *there* (or even in the average
`display_name` as they're normally non-stored computed fields).
`_compute_display_name` previously blanked (blacklisted) just a few
context values, but doing the opposite is probably the better idea.
See also: odoo/odoo#68946
OPW-2453956
Task 2500978
closesodoo/odoo#76469
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
For searches, Odoo uses `unaccent` if it's available. On some
technical fields this is completely unnecessary and precludes the use
of indexes.
This PR provides:
* an opt-out (`unaccent = False`) on `String` and `Text` fields
* a warning if `unaccent` is enabled on *parent_path* fields as their
performance can be rather critical and not using the index is quite
an issue (note: the check that `parent_path` fields have been moved
outside of the check for their existence as we want to check that
the field is declared and correctly configured in all cases,
probably)
Task 2627454
Part-of: odoo/odoo#76436
`website` added its own sql-compatible version of
`get_unaccent_wrapper` in order to use `psycopg2.sql` for safety.
That is very sad.
Add a condition in the "standard" unaccent wrapper which checks
whether the input type is a `Composable`, and in that case return an
`SQL`. This increases the cost of the wrapper a tad but probably not
to a really sensible amount.
Task 2634851
Part-of: odoo/odoo#76436
PR #75856 introduced a default implementation of group_expand for
Selection fields that set the value of group_expand to True.
However the PR lacked the necessary bits that allow the same behavior
for fields created on the fly instead of through code.
This commit allows the propagation of the group_expand attribute for
fields of type Selection, this commit also exposes the checkbox in the
UI to enable/disable the setting.
closesodoo/odoo#76775
X-original-commit: 1cacc3e53cf722a31bedff7a76f8e5cc6ed45ce0
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When adding an image to the library from an URL, if the URL contains
parameters (e.g. ?width=200&height=200), the link will not be accepted.
In this commit, we remove these parameters from the URL before checking
the image, so that the URL is considered valid.
We also remove these parameters before computing the mimetype of the
image, since it is computed based on the end part of the URL.
task-2618494
Part-of: odoo/odoo#75205
* add t-out to set of text-generation attributes
* fix comment location
* don't handroll `iterancestors()`
* only check the `@string` on the toplevel button, doesn't really make
sense that a button would contain a button in the first place (in
fact it's not allowed per the HTML spec, a button can only contain
*non-interactive phrasing content* cf 4.10.6)
Part-of: odoo/odoo#76581
With the introduction of the guest feature, a lot of fields are now
accessed through a controller so that they can be available to guests,
with additional checking in the controllers.
Avatar is one such field, however the extra checks were too restrictive:
we only allowed people to see the avatar if the user whose avatar was
requested was also a member of the channel. This breaks the avatar on
previous messages if a user leaves a channel.
This commit relaxes this restriction for internal users (they can now
see all avatars through this route), and adds a fallback to the avatar
placeholder for people who do not have acces (ie: guests) so that the
UI doesn't look broken.
closesodoo/odoo#76341
X-original-commit: b849fd426806332c1ecc1bbba7b568384a021c65
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Since commit bbb1a8f151, module's upgrade script can also be added
into special 'upgrades' folder as an alternative to the old 'migrations'
folder.
As those upgrade scripts are not maintained from version to version, we
should not count them as maintenance.
OPW-2536046
closesodoo/odoo#76334
X-original-commit: 28a303a67935dc5566cc0d1fe2be157736691968
Signed-off-by: Xavier Alt (xal) <xal@odoo.com>
Commit cd12293386 introduced an
optimization in the way that models and their inheritances are loaded,
with this commit we defer the setting of each model class' bases
from the _build_model method to the _prepare_setup method.
This has the advantage of setting any given model class' `__bases__`
attribute only once, but it also means that when _add_manual_models is
called, the __bases__ for ir.model are not yet set (only the default
implementation exists), therefore any module overrides to ir.model do
not take effect when creating the custom models.
This meant that if one creates a custom model with chatter support (i.e.
custom ir.model behaviour implemented in mail) and one restarted the
server, the registry would not properly setup ir.model before creating
the custom model (yielding warnings about tracking and such not being
valid fields) and when creating a record of the custom model, the
registry would crash.
With this commit, ir.model's _prepare_setup is explicitly called before
_add_manual_models to ensure that all overrides to ir.model are taken
into account.
closesodoo/odoo#76325
X-original-commit: f3afb23cdf21f395855771a037c721e977dc93c8
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Clientside, the only key got from an `ir.actions.act_window_close` type
action is 'infos'. But this key isn't a part of the whitelisted
readable fields server side, which means you will get a warning if you
use it.
closesodoo/odoo#76109
Signed-off-by: Steve Van Essche <svs-odoo@users.noreply.github.com>
By activating the profiler debugger, the two qweb options is added. The
qweb templates that need to be rendered are compiled into a new function
to add instructions for saving data.
`Add qweb directive context`
It's a sub-option of "Record sql" or "Record traces", add some context on
thread at current call stack level. This context stored by collector
beside stack and is used by Speedscope to add a level to the stack with
this qweb directive information.
```
directive=t-call='website.layout', xpath=/t/t
t_call_content
directive=t-foreach='5' t-as="'a', xpath=/t/t/div/div
directive=t-esc='website.search([])', xpath=/t/t/div/div/t
execute
```
`Record qweb`
Add profiling data used by ProfilingQwebView widget. In the `ir.profile`
form view, the widget display the duration and number of sql of every qweb
directives with the xml of templates. Every xml is recorded to be
consulted even if the user change the xml templates.
```xml
<t t-call="website.layout">
<div>
<div t-foreach="5" t-as="a">
<t t-esc="website.search([])"/> <!-- will display 5 separate requests -->
</div>
</div>
</t>
```
closesodoo/odoo#74712
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Setting a default value on a readonly related field without an inverse
method is nonsense. We log some warning when it happens.
We also fixed other cases where a related field has a default value
that overrides the target field's value.
closesodoo/odoo#67762
Related: odoo/enterprise#17313
Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
This hook automatically generates custom tooltips (instead of
native ones) for all elements with the "data-tooltip" attribute
set, inside the component using it. The optional attribute
"data-tooltip-position" can be used to enforce a specific tooltip
position.
This commit also makes the WebClient use this hook.
Part-of: odoo/odoo#73311
Co-authored-by: Lucas Perais (lpe) <lpe@odoo.com>
When a subtest fails, the failure message maybe a bit cryptic like
`Fail: Subtest (login=admin)`
With this commit, the parent test case and test method are displayed
too.
closesodoo/odoo#76046
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Assume that model A has a computed field G that depends on both fields
F1 and F2. Assume that model B inherits from A (_inherits). When G is
computed during an onchange, it must use the form values for all its
dependencies F1 and F2. Before this commit, only the field triggering
the onchange was assigned on the parent record (see (*) below.)
record parent
--- ------------+-------------+------------
initial state | F1=0, F2=0 | F1=0, F2=0
assign F1=1 | F1=1, F2=0 | F1=1, F2=0
assign F2=2 | F1=1, F2=1 | F1=0, F2=1 (*)
The commit also includes another patch: when assigning the parent
record, the ORM determines the dependent records (in this case, the main
record in the onchange). If the delegate field (many2one from B to A)
has an inverse one2many field, the ORM uses that field to determine
dependent records. However, in the case of a new record, that field is
not in cache, and its value is determined to be... empty! The patch
consists in "fixing" the value of that field when we determine the
parent record (when the delegate field is accessed.)
Part-of: odoo/odoo#76042
Co-authored-by: William André <wan@odoo.com>
Here is the idea: when creating a view V that inherits another view, and
V only adds elements in the parent view, only the elements added by V
must be validated. The other elements are not modified, and thus do not
need to be validated again. However partial validation cannot be done
when modifying an existing view, or when the view replaces elements in
the parent view. In such cases, the method _check_xml() validates the
whole combined architecture.
The time spent in method _check_xml() to install a database from scratch
with module sale_management went from 2.730s to 2.243s (-18%), and the
number of queries made in the method went from 2283 to 1614 (-29%).
closesodoo/odoo#75796
Related: odoo/enterprise#20633
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Also speed up view validation by not calling fields_get when possible.
This avoids a call that is usually not necessary for validating a view.
Most of the time, fields can be found on the model itself. Only
overridden fields_get() with fake fields are actually useful, like the
one of model 'res.users'.
The time spent in method _check_xml() to install a database from scratch with
module sale_management went from 3.510s to 2.730s (-22%), and the number
of queries made in the method went from 2995 to 2283 (-24%).
Part-of: odoo/odoo#75796
Besides simple code improvements, remove node_info['attr_model'], and
check domain on specific elements. Also refactor _validate_attrs. The
changes have no impact on performance.
Part-of: odoo/odoo#75796
This makes it easier to optimize the algorithms for both cases:
postprocessing without validation, and validation without
postprocessing. Also factor out some common bits like the editability
of nodes.
The net result is a speedup in performance. The time spent in method
_check_xml() to install a database from scratch with module
sale_management went from 4.955s to 3.578s (-28%), and the number of
queries made in the method went from 3779 to 2995 (-21%).
Part-of: odoo/odoo#75796
This generalizes an existing optimization and saves a few time to
validate a view. The time spent in method _check_xml() to install a
database from scratch with module sale_management went from 5.145s to
5.044s (-2%).
Part-of: odoo/odoo#75796
The project overview is a significant technical debt
as it is a custom qweb view. It is quite limited:
- It is not possible to group the data, to filter on dates,
SOs or Field Service projects.
- Improving this is very difficult and would require weeks
of development that are not worth it.
In any case, all the information provided by the project overview
can be found elsewhere. The stat buttons of the report are basically
duplicates of the ones from the project form view.
Therefore, we are removing this report and its twin,
the Project Costs and Revenues.
In addition, analytic items lack context for the user to understand
what generated a certain cost or revenue:
- there is no link to the source document;
- the entries are not categorized (e.g. it is not easy to understand
if a cost comes from a timesheet cost, a purchase order or an expense);
- the billable type group by works fine for timesheets but then all
the other entries are flagged as Undefined, which is not very helpful;
- there is no option to easily isolate costs from revenues.
task-2545084
See odoo/enterprise#19250
See odoo/upgrade#2706closesodoo/odoo#72736
Signed-off-by: LTU-Odoo <IT-Ideas@users.noreply.github.com>
+ Added the 'Trusted Devices' feature
+ Added 'Remember this Device' checkbox on /web/login/totp
+ Added trusted device's OS / browser on Profile > Account Security
Added '2FA Trusted Devices' feature to allow users to remember their
device to bypass the 2FA for the next connections. The trusted devices
are displayed in a 'Trusted Devices' One2Many under the 'Developer API
Keys'. It is possible to revoke all the trusted devices at once with a
special button. It is also possible to revoke one at a time on the
desired one.
Task-id 2523092
closesodoo/odoo#75535
Related: odoo/upgrade#2800
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Co-authored-by: Martin Trigaux <mat@odoo.com>
Right after some module operation (install, uninstall), the framework
looks for some ir.actions.todo to execute. Currently those actions
cannot be found, because the cursor seems to see the database in the
state it was just before the operation. Simply add the commit() which
was removed by revision 1595c0ee27.
closesodoo/odoo#75980
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The XML-RPC interface has a compatibility shim for binaries as
historically Odoo has returned "binary" data as base64 strings. To
avoid breakages during the Python 3 transition, the shim was
introduced to decode the output binary data (under the assumption that
it'd be ASCII-compatible).
In the case where the data is *not* ascii-compatible, however, it can
generate invalid XML documents: "C0" control codes (with the exception
of tab, LF, and CR) are not valid in XML 1.0 (which XML-RPC is an
application of), however they're perfectly valid string characters and
the standard library's marshaller does not check for them, embedding
them directly in the output document and breaking the client's
decoding.
Work around the issue by replacing such binary data with an empty
string.
While at it, move the bytes shim to the customized marshaller, this
way everything's at the same place and it's not necessary to waste
time trying to understand why the marshaller is just not calling what
it's supposed to call.
Fixes#61919closesodoo/odoo#75973
Forward-port-of: #75952
Forward-port-of: #74699
X-original-commit: 1a0b3f7
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Xavier Morel