Overly attached cache's overhead turns out problematic. Clearing the
cache after batches of records keeps the cache overhead low and does
not change performances.
Explanations:
env.cache is 3 levels of maps {field: record_id: {env: value}}, where
env can be either an Environment or a pair (cr, uid) depending on the
field's dependency on the context or not.
This can be an issue when the current request loads many fields in an
enormous number of records in a single environment. The (PaaS) case
here was the export of a res.partner field from 36358 records in a single
environment[0]:
* prefetch expanded the single field to 68, leading the base `cache`
to have 68 entries. getsizeof(d<len=68>) == 3360 (3kB, which we will
soon see we can ignore entirely).
* *each* of these entries would hold a map of 36358
records. getsizeof(d<len=36358>) = 3146016 (3MB), 68 times = 213MB.
* finally each record entry is also a {(cr, uid): value}, here the
dicts have a single entry which makes them 280B, and their key is a
2-tuple "worth" 72B, or 352B/record/field, or 352 * 36358 * 68 ~
870MB[1].
For a total of ~1GB, which is roughly the issue we can observe.
Future possibilities: extract the cache-clearing iterator to be more generic
and available on BaseModel directly? Or even make the default iterator
batched & cache-clearing?
[0] note that sys.getsizeof only provides the size of the object it's
called on, it is not recursive
[1] slightly more in actuality as there's some variation between the
leaves depending on the field type e.g. M2O values are a 1-tuple
adding 60B, ...
Fixes#22475
If a record is deleted *and* has property fields, the deletion of the
property fields will trigger the various recomputes before the record
itself has actually been deleted, and thus stuff which depend(ed) on
that record will be recomputed under the assumption that the record
still exists, which probably will not work.
Under the assumption that the same issue would occur when deleting
more than one batch of records (unlinking the data, values or
attachment would also trigger a recompute before the proper end of the
function and could lead to incoherence, e.g. some of the records being
deleted being visible to recomputes), lift the entire effective body
of the unlink into a norecompute context, to ensure that only the very
final recompute (from the top-level unlink) is actually triggered.
opw-807036
Before this commit, if the first exported field of an M2M is `id` (the
xid) the entire M2M is folded into a single cell with comma-separated
xids and any following field is ignored. If `id` is any but the first
field, the export behaves normally (with the m2m exported as a "table"
inside the parent record).
This behaviour makes sense for import-compatible exports where the id
is the only thing which can be exported anyway, but it is troublesome
outside of that mode as the behaviour of m2m under export becomes
incoherent/unpredictable (ish) as it depends on the position of the
m2m's `id` in the exports list.
Change it so we only perform folding in import-compatible mode (which
is the default for backwards compatibility with e.g. API calling
export_data directly & the like).
opw-813361
Fixes#22600
We fell on a strange situation in method `onchange`, where some data was
missing from cache when serializing the result. In that case, the
prefetching
overwrites so many stuff in cache that the results of the onchanges are
lost.
Don't let that happen.
The function `stripped_sys_argv` was broken because of `itertools.groupby`
being hidden by the local `groupby`. Aligning APIs makes the local one a valid
substitution for `itertools.groupby`.
Put the code to update the MPTT in specific methods, and reduce the number of
queries being made (from 5-6 queries to 2-3 queries). Add test on MPTT to
validate the refactoring.
Current behaviour when duplicating a record:
apply copy override in the user language (e.g. "%s (copy)")
copy the previous term in all the other languages (including)
So if product was named:
cheeses (en)
kaas (nl)
fromage (fr)
Duplicating the product, while in French, will result into:
cheeses (en)
kaas (nl)
fromage (copie) (fr)
This is a problem if there is a unicity constrain on the field, it will be
raised (there is already a product named 'cheese')
After this PR:
Duplicating the product, while in French, will result into:
fromage (copie) (en)
kaas (nl)
fromage (copie) (fr)
This applies **only** if there is an override of copy
This is detected by comparing the old and new value for the user language
Fixes#7010Closes#22052
backport of afef71d6b9
Original commit message:
Let `create` and `write` round monetary field values before sending them to the
database. Pass the values to be written to `field.convert_to_column`, so that
the currency can be retrieved from the values, and the value be rounded.
When doing a group_by in a tree view, and then ordering by an
aggregated field which contains at least one capital letter in its name,
you will get an error and won't be able to ordery by this field.
Why? read_group is called when sorting a group by, which calls
_read_group_raw which then goes to call _read_group_prepare, which will
return order by terms in this manner ['id asc', 'x_Test desc']
the problem with this is that postgres automatically converts
non-quoted/non-qualified column name to lowercase, of course this means
that if we have a field "x_Test" but no field "x_test", it will try to
look up "x_test" instead of "x_Test" and since the column doesn't exist,
it will throw a ProgrammingError.
This commit changes this behavior by wrapping the order_by terms generated in
_read_group_prepare with double quotes, which postgres interprets
correctly.
Fixes#21348
Cherry-Pick of c338e24ee7 as previous
forward-port has been badly done...
When you import a record in Odoo, you can specify an id without
prefixing it by the name of the module. e.g. 'project_project_x'
The result of this is an empty string as module name.
(This value is valid for a required field...)
Plus, when you check metadata, you see 'false.project_project_x'.
Now, the default module name is '__import__' instead of an empty string.
We already have similar behavior with '__export__' triggered by
'export_data' method.
Remove weird special case: when the first field of the first line of a one2many
is empty, replace this first field by the comma-separated names of the lines,
and discard the other lines.
Remove weird special case: when the first field of the first line of a one2many
is empty, replace this first field by the comma-separated names of the lines,
and discard the other lines.
Christophe Simonis