Before when we do `reserved(records)`, it iterates in a reverse
order of `records`. Unfortunately, the `__reserved__` method don't
exist explicitly in BaseModel then Python fallback on
its own implementation using `__getitem__` and `__len__` (coming from Sequence): https://github.com/python/cpython/blob/3.10/Lib/_collections_abc.py#L1047-L1049
Because it uses __getitem__, it breaks the prefetch of the recordset.
Example:
-------------
```
partners = self.env['res.partner'].browse(1, 2, 3, 4, 5)
for partner in reversed(partners):
partner.name
```
will generate 5 SQL requests to fetch data (one by record)
Then create our own `__reversed__` and handle the prefetch correctly
(like `__iter__`). Now in the example it will correctly generate only
1 SQL request because of the prefetch.
task-2687953
closesodoo/odoo#79622
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
The processing of the LINK command in a one2many should fail when the
line being linked does not exist. However, there is one case where it
should not fail: when that line existed and was linked before applying
the commands. That use-case may seem strange, but it actually exists:
deleting a line in a sales order automatically deletes the corresponding
reward lines.
closesodoo/odoo#78318
X-original-commit: 4bfe1b5cab10d3171d386d76799a7d7729f49154
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The license is missing in most enterprise manifest so
the decision was taken to make it explicit in all cases.
When not defined, a warning will be triggered starting from
14.0 when falling back on the default LGPL-3.
closesodoo/odoo#74245
Related: odoo/design-themes#48
Related: odoo/enterprise#19862
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Writing on x2many field should be done last, because deleting x2many
lines causes pending computations and updates to be flushed. Writing on
column fields after that inevitably adds extra update queries.
We introduce an attribute `write_sequence` on fields to order fields for
write. The prescribed order is: all fields except monetary and x2many,
monetary fields, x2many fields.
closesodoo/odoo#65959
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The goal of this change is to simplify the code managing `create_date`
and `write_date` in methods `create()` and `write()`, and also to remove
weird behaviors caused by the way those fields were updated.
Assume we update a simple field on a record. This adds pending updates
for the field and `write_date`. However, the value of `write_date` is
not known yet: it will be updated as `NOW() AT TIME ZONE 'UTC'` in SQL.
So `write_date` is actually given a dummy value in pending updates, and
it is invalidated from cache, until its value is flushed to the database
and fetched again.
Now assume we access another field on the record, and that field is not
in cache. The prefetching mechanism will read all column fields,
including `write_date`, and flush them first.
# this adds pending updates foo: 42, write_uid: 1, write_date: False
record.foo = 42
# assume 'bar' is not in cache; this prefetches all column fields,
# which flushes the pending updates above before reading them back
result = record.bar
We can avoid flushing pending updates if the values read from database
do not overwrite existing values in cache. If you assume that the value
of a pending update is in cache (in the example, `foo: 42`), you don't
need to flush the corresponding field. Indeed, the value of `foo` will
remain 42 in cache, whatever its value in the database. This assumption
(pending updates are in cache) is true for all fields *except* for
`write_date`: it is invalidated from cache, and given a dummy value in
pending updates. This branch actually makes this assumption true for
all fields. The avoidance of flushing pending updates will be done in
another commit.
In order to directly assign `write_date` its value, we use a cache for
the value `NOW() AT TIME ZONE 'UTC'` from the database. This costs at
most one query per transaction, and potentially saves a few queries.
Co-authored-by: Victor Feyens <vfe@odoo.com>
Adapt some query counts to their optimal value, in order to measure the
effect of the following commits on queries.
In module test_performance, some query counts were actually not correct:
the initial flush() done by the context manager assertQueryCount() may
prefetch some data to the cache, and that prefetching is not accounted
for in the query count. This is very true when assertQueryCount() is
preceded by a cache invalidation. We have to move the invalidation
inside the context manager, so that the prefetching is now counted.
Make the method `_search` return a `Query` object, and make that object
generate a subquery when used on the right-hand side of a condition.
closesodoo/odoo#52403
Related: odoo/enterprise#10945
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, calling `mapped()` on a relational field of a single
record inside a for loop does not correctly propagate the prefetch ids
from the bigger recordset down to the record inside the loop:
[rec.line_ids.mapped('name') for rec in recs]
To be more precise, `recs` correctly propagates prefetching information
down to `rec.line_ids`, but method `mapped()` does not pass it along to
the record that triggers the prefetching.
This resulted in one query per record in `recs`, so if `recs` were a
1000 records recordset, at least 1000 queries would be necessary.
With this commit, the `mapped()` function correctly propagates the
`_prefetch_ids` of the larger recordset (`rec.line_ids`) so that the
prefetching works properly and the 1000 queries are brought down to 1.
This commit is a followup on #42611.
closesodoo/odoo#54565
X-original-commit: 0e97053fee36c0c76b70cb94be5340c3144b178b
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Consider a context-dependent field, and successively access it on a
recordset with different contexts. On the first context, the field is
correctly computed in batch. After that, the field is always computed
one by one.
The bug is in the method that determines which records in a given set
have no value in cache. On the first context, the cache is empty for
the field, so all records are returned. After that, the method
considers that all records have a value in cache: they do, but for
another context key! Simply using the context key when looking up the
cache fixes the issue.
closesodoo/odoo#52360
X-original-commit: 35d69589d9b43afe6c6fc9779458323f7180153e
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Co-authored-by: Xavier-Do <xdo@odoo.com>
Also non-browser jsonrpc (as it goes through a similar process): for
internal performance reasons, name_search and read_group have been
converted to a *lazy* name_get, so the "display name" is not
unnecessarily computed.
However this is an issue for the RPC endpoints (/xmlrpc and /jsonrpc)
as they have no support for `lazy` and thus tend to blow up and / or
do the wrong thing when trying to output a lazy:
* xmlrpc has no way to handle lazy at all and straight blows up
* jsonrpc falls back to `json_default` so they try to stringify the
lazy, which might have worked except
*Problematically* both endpoints delegate the actual work to
`dispatch_rpc` which handles dispatching between various services and
ultimately creates a *new* cursor before calling model
methods (`object` service and `execute`/`execute_kw`).
This means by the time the result is serialized to be output, the
lazy's cursor has long been closed, and thus any access to an
unevaluated `lazy` errors out when trying to fetch the underlying
item.
This also means we can't just add a hook to serialize the lazy
in the xmlrpc marshaller, though we do have to do that. We *also* (for
both xmlrpc and jsonrpc) have to force evluation of lazy values before
our cursor is closed, meaning it has to be done right after the method
is invoked, iterating the entire response.
Related to task 2170343
closesodoo/odoo#49286
X-original-commit: e2b5a359c1d5eccbe725c1c3169b4130d7bca49b
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When the `select` returns nothing more than ids that are already known, there is
no need to make it at all.
This removes one query from `_read` every time it has to fetch only fields that
are stored in a different table (o2m, ...), which happens all the time when
reading a stored field first (triggering prefetch) and then reading a o2m.
The query that is now removed was used to check access rules, but the trick is
to use `check_access_rule` to verify the rules in python instead.
Part of task-2061122
closesodoo/odoo#36263
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Query counts weren't adapted to recent performance changes.
This commit updates the different query counts to make sure
any commit changing the query counts knows it and does it on purpose.
Some query counts may vary between community and enterprise
and therefore have a higher value than needed in community version.
closesodoo/odoo#43202
Related: odoo/enterprise#7682
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Before this commit, creating a record of a model with at least one
property/company_dependent field would trigger the field's inverse
method regardless of whether an actual value was being passed in for the
field or not.
This means that if no value was given or if the value was the same
as the default value, we would waste precious time in the property
field's inverse method.
With this commit, we do not call the inverse method if:
1) there is no value for it in the vals dict (use default)
2) the value in the vals dict is the same as the default value
As an example, when creating a res.partner record while having account
installed (which introduces a property field in res.partner), the
creation time goes from 24ms to 14ms.
closesodoo/odoo#36267
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This branch is the combination of several optimizations in the ORM:
* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;
* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;
* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);
* make method `modified` take advantage of inverse fields to inverse
dependencies;
* filter records by evaluating a domain on records in Python;
* a computed field with `readonly=False` behaves like a normal field
with an onchange method;
* computed fields are computed in superuser mode by default.
Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.
closesodoo/odoo#35659
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
After 12744bc81ebdaa1671a953c14ba54c637d6a9255, x2m operations are
fully batched, and deletions are processed last when the operations are
flushed, regardless of the order in which they were specified by the
write() or create() call.
This carries a risk of violating (non-deferred) unique SQL constraints,
when the operations for deleting previous lines and re-creating new
ones are processed in the same batch.
This patch executes the deletions before other operations during a
flush, which should be safer with regard to SQL constraints.
An extra constraint is added in test_performance.line to simulate this
corner case, then covered by an extra unit test.
Another unrelated test had to be altered to avoid violating the
new constraint.
This can be reproduced easily by upgrading the `project` module in master,
due to the unique constraint[1] on `ir.actions.act_window.view`, that
gets violated when processing the batch write on this o2m[2].
[1] https://github.com/odoo/odoo/blob/ccc42f16/odoo/addons/base/models/ir_actions.py#L288-L289
[2] https://github.com/odoo/odoo/blob/ccc42f16/addons/project/views/project_views.xml#L394-L396closesodoo/odoo#28314
Purpose of this commit is to give description more "business oriented"
because those descriptions appears in Odoo Studio which is supposed to be used by end users, not only by developers.
Related Task ID : 37311
When a record fields are prefetched, the currently accessed record is
prefetched as well as PREFETCH_MAX (=1000) minus currenly accessed
records.
Since 439fa826 the id field was thought as "prefetched" but was not,
which caused that in an onchange, we would have a prefetching as follow
for 1200 records:
- get records 1-1000
- get record 1001 (1001-1200 INTERSECTION 1-999 (id) + 1001 (current))
- get record 1002 (1002-1200 INTERSECTION 1-999 (id) + 1002 (current))
- get record 1003 (1003-1200 INTERSECTION 1-999 (id) + 1003 (current))
- ...
- get record 1200 (1200 INTERSECTION 1-999 (id) + 1200 (current))
So we would do 201 queries instead of 2 when prefetching the records.
The added test without this change failed the query count with:
"AssertionError: 284 not less than or equal to 5 : admin"
opw-1837548
opw-1837552
closes#24326
Add the context manager method `assertQueryCount` to count queries,
the decorator `@users(...)` to execute a test for several users, and
the decorator `@warmup` to execute a test twice (warm-up, then real).
Containing
* fix self._round not always existing;
* fix counter for activity creation;
* use self.str() in order to have some variation in test_mail;
* separate base performance to mail feature specific performance tests;
Test performance will now hold the base class for performance tests as
well as tests related to the ORM, depending only on base. A new module
test_mail is introduced at this commit that contains performance tests
related to mail module. This commit contains only code move and should
not impact anything.
Future commits will move mail tests into test_mail so that all mail
related tests are located in the same optional module. This allows
notably to avoid creating a lot of unnecessary tables when installing
mail module on production databases.
The command does not need to remove existing relations, as there is no relation
yet on new records. This saves one query per many2many field with that command
in a `create`.
