time.clock is deprecated since python 3.3 and no longer exists in python 3.8
time.process_time was introduced in python 3.3
Replace "is" by "==" as this produces a SyntaxWarning in python 3.8
Fixesodoo/odoo#41313closesodoo/odoo#44502
X-original-commit: f03b4cb5576f264b6845242ba112f02422af4f5f
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Before this commit executing m2m commands in batch would lead to undesired
results, such as duplicating the related records on every subsequent record of
the batch.
Indeed since [1] the `create` and `unlink` are called in a loop, but their
content was not reset at each iteration.
[1] 9920f20e4cclosesodoo/odoo#43950
X-original-commit: 0800f64b011c8f020c0d7dd6e1c149aebd1a4627
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Previously, mapped was following a very naive approach, which was simply
calling the field name passed as input for every record in a recordset,
sequentially.
The problem with this approach is that we will potentially recompute the
same fields multiple times for differents records, when this could be
done once per field for ALL records, and store this value in cache for
further access.
Another potential problem is that we don't take advantage of the ORM's
prefetching to fetch all the records that are not in cache at once,
instead of doing the same query for every record in the recordset.
Yet another problem is the conversion of each cache value to a record
format and then combining all of the individual records into a single
recordset, which, depending on the size of the recordset, can take an
unbelievable amount of CPU time.
With this new implementation of `mapped()` we take care of all of these
problems:
This is done by first delegating `mapped()` from the model to the field,
this mapped takes a recordset as input and it will try to batch compute
and prefetch as much as possible for the entire recordset, but it will
not keep these values for the actual output, it just stores everything
in cache and then at the end, retrieves everything from the cache to
guarantee the same order.
After the mapped, the conversion from cache format to record format is
delegated to the new `convert_to_record_multi` which will fetch all the
ids and then perform a single browse to encapsulate all of the records
into a single recordset with the least amount of overhead possible.
Part of Task 2170344
closesodoo/odoo#42611
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Computing the `cache_key` turned out to be a big factor during the
lifespan of a `BaseModel.mapped` call and a lot of this time is spent
computing the same `cache_key` over and over.
These unnecessary computations can be easily reduced to a couple by
moving the `cache_key` method on the environment (instead of the field)
and by implementing a memo for that method. The rationale is that the
`cache_key` of a field does not change for a given environment.
The result of this patch is up to 50% faster `Field.__get__` which in
turn means a GLOBAL gain in performance, especially for methods /
functions that rely heavily on `__get__` such as `BaseModel.mapped`.
closesodoo/odoo#42674
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Consider an x2many field with `context={'active_test': False}`. The
value of that field should always contain inactive records, as the
field's own context overrides the context of the current record.
closesodoo/odoo#42824
X-original-commit: a2fc37adc179fb8bfc11251137334f0a43c58135
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Assume F and G are computed by the same method on a missing record R.
During recomputation of F on R, the compute method is called but fails
because R is missing. Both fields are re-marked to compute (because
computation failed), then F is discarded (because R is missing). Then
comes G's turn: G is accessed on R and the computation fails. Both
fields are re-marked to compute (because computation failed), then G is
discarded (because R is missing). Now F is marked again to compute: the
process never ends.
To avoid this situation, discard all fields to recompute on missing
records.
closesodoo/odoo#42234
X-original-commit: 78bf4dbaa1c4adfa1dac68a4d79b87800003ac05
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
While x_name has long been automatically supported as an equivalent
of name, x_active was not.
This commit adds this behaviour OoB, both in the ORM and the web client.
On the ORM-side, a new `_active_name` attribute is supported on models.
This attribute specifies the field that should behave as an active marker
for records of the model. It is supported the same way `active` has been
until now (filtering by the `active_test` context key and toggled by the
`action_archive`, `action_unarchive` and `toggle_active` methods).
Although no check has been added on the field's type, it is assumed to
be a boolean field (the same way no check is present on the `(x_)name`
field).
On the client-side, the list view and form view now both support
detecting the presence of either `active` or `x_active` on records,
automatically adding an '(Un)Archive' button in the Action menu if such
a field is detected.
Note that the ORM implementation does actually need the field to be named
in any specific way, but since the web client has no mechanism to load
information regarding a model (the lifecycle of an action loading
includes loading the action, view and record(s) but no generic
information about the model itself besides what is included in the
views), we restrict the field's name to `(x_)active` to avoid confusion
as any other name would work at the ORM-level but not in the client.
In the future, the client might be able to more elegantly get
information about models, but this was not the scope of this change and
this solution should cover most cases.
Note that the `active` field will always takes precedence over the
`x_active` field to avoid confusing the polarity, even if both fields
are present on the model.
In the case of a custom field, it might be slightly annoying that the
default value of a Boolean field is `False`, which means that upon
adding the column, all existing records are automatically archived. This
can easily be worked around using an `ir.default` record for that
particular field and an update of existing records (e.g. through the
list view). The goal of this change was not to make it easy to add
support for custom active fields, but to make it possible - we have
therefore kept this implementation which introduces few changes while
adding enough flexibility for developers.
See https://twitter.com/zubair_shafiq/status/1202587553871880192?s=20
for more info regarding supporting any `_active_name` in the web client.
Co-authored-by: Raphael Collet <rco@odoo.com>
Makes it way easier to realise that a cache access blew up because of
a @depends_context('active_ids'), which returns a list, which is not
hashable.
closesodoo/odoo#41863
X-original-commit: 5d2ce1fb3f2f14c9274924d3f4052279671a0c3b
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Those cases are already checked in models._check_company() but not considered
when defining the default domain on `check_company=True` fields.
* Consider res_users relational field
Companies allowed for user = user.company_ids
* Consider company-dependent fields
Companies allowed = self.env.company
closesodoo/odoo#40906
X-original-commit: a8de9509823c213dad2f5e8ff44ceb670f81ea70
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
This is a performance fix. It avoids the cost of XML/HTML translations
for fields that may not be necessary when prefetching fields on records.
This patch marks such fields as non-prefetchable by default.
The code that changes the attribute `translate` on HTML fields (from
`translate=True` to `translate=html_translate`) has been adapted to
allow the setup of textual fields to mark the field as non-prefetchable.
Jairo Llopis made a comparative benchmark: evaluating `name_get` on
`event.event` records. The method needs the field `name` and without
the patch, the prefetching mechanism reads the translated HTML field
`description` as well. This patch speeds up the benchmark from 5800ms
to 800ms (see https://github.com/odoo/odoo/pull/37967#issuecomment-538364011).
closesodoo/odoo#40771
X-original-commit: e5ee5e5b65f85d66c8d59594ead9eece172c4282
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Pedro Baeza <pedro.baeza@tecnativa.com>
Co-authored-by: Jairo Llopis <jairo.llopis@tecnativa.com>
From now on, if one wants to force following operations to happen in a given company,
use with_company(company) or with_company(cid) to update the environment.
Before this commit, when modifying a Many2one in a form view to add a
new record, it will create the new record, and calculate the inverse
and write it in all the records of the list. This will call the write
method in all records even if they weren't changed.
Now, the inverse is set to be modified only if it's different from the
current value.
opw-2091842
closesodoo/odoo#40258
X-original-commit: 417cee7f0fdb7ed7eb424178efb656b967fa0a5e
Signed-off-by: Jorge Pinna Puissant (jpp) <jpp@odoo.com>
Co-authored-by: Raphaël Collet <rco@odoo.com>
View creation/edition represent a important part of an install and a lot
of possible view errors are not detected, like fields used in domain
filters. Some part of the code a difficult to maintain, and view checks
are splitted in multiple places.
This commit aims at refactoring view validation by regrouping most part
of the logic in ir_ui_view and trying to optimize the overall process.
Since most of the lines were touched, this task was also an opportunity
to modernize the API.
Main changes on method `check_xml`:
- extract node processign and validation to individual postprocessor
- add validation for filter node, buttons, ...
- fix accessibility checks (and improve their performance)
- move xpath check to specific Python node validator
- clarify error messages (wip to continue)
Main changes on method `read_combined`:
- optimize the search for inheriting views in a single query doing the
whole recursive search
Indeed, after removing xpath validations, `get_inheriting_views_arch`
was the most expensive method in `check_xml`, spending most of the time
in `search` because of recursive calls to retrieve children views.
The view Backend Assets is a good example of the latter point, since a
line is added in the view for almost every module. 70 views (community)
are added at first level, `get_inheriting_views_arch` is efficient and
returns all 70 views. Then at the second level, the method
`get_inheriting_views_arch` is called 70 times for nothing. 70 calls to
`search` (squared/2 since each view is checked independently) are almost
useless. The same case applies to the settings view.
As a result, the average module installation time is 25% faster, and the
average time spent in `read_combined` is almost divided by 2.
When assigning a default value to a computed monetary field, if self
contains records with different currencies, the assignation will fail
when trying to round the value (because self[currency_field] is a
recordset of multiple currencies).
Globally, assigning a monetary value to records in different currencies
should never happen. The only case we want to support is defaulting a
computed monetary to 0.0.
Add clear error when trying to set monetaries in multi-currency.
closesodoo/odoo#40155
X-original-commit: 3a58dfa1e51702a659e9113271f19d087e3ec349
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
With this commit, the functions float_compare, float_round and
float_is_zero are available from the fields.Float class as static
methods.
This allows the usage of these helper methods without the need to import
them from odoo.tools.
closesodoo/odoo#39695
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Before this revision, any m2o with an ir.* model as inverse could have
the ondelete policy as 'restrict', either by default if it was required
or by setting it explicitly in the field declaration.
This is problematic because ir.* models are reflection models and they
*MUST* be deleted during an uninstall of the module that introduces
them, otherwise tables, fields, constraints, etc. are left in the
database, this is why 'restrict' doesn't make sense UNLESS the unlink is
being performed by an user and not by the system (uninstall).
As a solution, this revision sets the default to cascade if the field is
required, otherwise null.
If the programmer explicitly sets a required m2o field with an ir.*
model as an inverse to ondelete='restrict', it is considered an
unsupported use-case and the registry will crash during loading with a
clear error explaining that it is not supported.
closesodoo/odoo#39739
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Before, en_US was almost always the default in case of 'no language'.
Now, that should be superseded by lang=None;
yet still, en_US plays the role of source language.
In a multilanguage database where en_US was not installed,
writing a translation with lang=None would crash.
This could prevent module installation.
A test is added to cover that case.
In the case where we write a record with lang=None, the source is the
same as the record; in that case the cache contains the value for the
field.name under the context keys en_US and None, so we need to
invalidate the cache to avoid getting back the old value.
co-authored with mart-e
opw 2088487
opw 2083710
closesodoo/odoo#39130
X-original-commit: 1a5a999c2dd69c9a6a2dacb59e8bf45b09c6bc13
Signed-off-by: Nans Lefebvre (len) <len@odoo.com>
The default `compute_sudo=True` makes sense for recomputing stored
fields that are indirectly related to a business operation. This
ensures that the recomputation of the field does not break an operation
that is not aware of the fields to recompute.
However, computing non-stored fields in superuser mode is usually not
necessary. It even leads to unexpected values: counting a partner's
sales orders does not give the same result in superuser mode as in
normal mode. That is why non-stored fields are not computed in
superuser mode by default.
[FIX] account, delivery, event, hr_recruitment, point_of_sale, stock:
adapt the model definition to make all fields with the same compute
method have the same value for `compute_sudo`.
[FIX] sale: split the computation of `invoice_ids`, `invoice_count`
(non-stored) and `invoice_status` (stored), as no code is actually
shared.
closesodoo/odoo#39195
X-original-commit: 843fd38a97f02b49dc09d7f55919072d272fd80e
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
This commits fixes user duplication.
Suppose there is a monetary field, with a currency_field that is a related.
When going through model's _create, we do:
`if field.type in ('one2many', 'many2many'):
self.env.cache.set(record, field, ())`
knowing that thse values are false, with the intent to clean them later:
`for record, field in cachetoclear`
However, when setting the scalar values, we go through:
`accessing cache_value = field.convert_to_cache(value, record)`
In the case of the monetary field, this depends on another field value
(the currency_field). If it is a related, we can access its value.
However, at this point, if we check any access rights, we might use the value
of a relational in cache for which the value is incorrectly set to False.
In the case of the user, this is what happens: it inherits its currency_id
from partner, as well as its debit_limit which depends on it.
When the access if checked, company_ids is set to False in cache.
So when going through the rule 'user rule', which checks that the company_ids
intersects with the env.companies.ids, the result is always False.
In some way this is essentially hiding the problem, but the true fix is
probably not feasible in stable.
opw 2086661closesodoo/odoo#39017
X-original-commit: aa05a9359b42317261c8e7ffcadbeb9fe74dcd58
Signed-off-by: Nans Lefebvre (len) <len@odoo.com>
This fixes a regression introduced by 291a0e99d137142bd24addd1054c1357ddd7125a.
closesodoo/odoo#39178
X-original-commit: 852ee32e3419c210a3844b0f943af8dd2bbdae88
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, if `max_width` and `max_height` are 0, the image would not
be opened by Pillow at all (to gain a bit of CPU) however this was actually a
bad idea in this case because then we don't ensure the given value is actually a
valid image when we save it, only later at display we notice it and it crashes.
This optimization only made sense when `image_process` is called many times,
from the route displaying images every time a visitor is requesting an image,
for example. It doesn't really have an impact for one time operations such as
creating/writing.
So when size parameters are 0, which is the default, another parameter must be
passed to `image_process` to ensure image validity, and `verify_resolution` is
actually built for this, as it does the minimal amount of processing:
- loading the image
- making sure it is valid
- and ensuring the resolution is not completely crazy (`IMAGE_MAX_RESOLUTION`).
It doesn't alter the original image if no other operation was requested.
Pr: #38292
X-original-commit: e16c36bb4bfa3ae1739712c69c72849665a057dd
Binary fields are supposed to be saved as binary in cache, and not as string
which is also supported as an input value.
Pr: #38292
X-original-commit: 64e0d106adccd8132cbfeb6b60da74b3feaf05e1
The compute methods are always expecting the real value and not the bin_size.
The solution is to always compute with `bin_size=False`, and then manually
compute the `bin_size` and set it on the cache of `bin_size=True`.
PR: #38292
Co-authored-by Sébastien Theys <seb@odoo.com>
X-original-commit: 7744886d6141ca7971d91807d0444c707e10fdf8
Users sometimes define custom models on SQL views
e.g. @nseinlet
In such a case, Odoo should not attempt to create foreign keys
as it just cannot work on views.
This could prevent the migration of a database
with such a custom model using a view
when it attempted to fix the missing foreign keys
when updating the modules.
closesodoo/odoo#38988
X-original-commit: dfaea03de57394a9a188f499da664c57dd9adc29
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Remove individual m2m indexes and replace with a single mirrored
composite
This will help for logical replication strategies (postgresql BDR)
closesodoo/odoo#37963
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Issue: after many2one updates, the cache of the corresponding one2many
fields was inconsistent when the latter depends on `active_test`. One
of the values in cache was updated, while the other was left intact.
We simplify the cache by not making the field depend on context: the
cache value contains all the records in the relation (corresponding to
`active_test=False`). The value of the field is automatically filtered
by the `active` field when the value is accessed.
This makes it easier to maintain the cache value, guarantees its
consistency, and avoids queries to read the one2many field with
`active_test=False`, after having set it with `active_test=True`.
This makes sense because in `create` the "previous" value of a field is
obviously non-existent.
This avoids having to fetch pre-existing attachments when they are known to be
inexistent, which optimizes stored related binary that are computed post-insert.
Before this commit, flushing a model that uses the `image.mixin` would generate
twice as many queries as it should.
This is because `_compute_related` is not meant to be overridden. In the
override of `Image` we call `super()` then post process the records by
reassigning each record's relevant image field with `_image_process`.
The `super()` will assign the field of the records once which
will in turn trigger a `write` (and thus, will generate queries) then
after the `super()` call we reassign them which will re-trigger the same
`write` and the same queries.
With this commit, instead of overriding `_compute_related` we extract the
processing to a method that can be overridden, so that the `write` is called
simply once.
The result, obviously, is that queries related to the `write` are cut in half.
Related PR: #36683 & #36288
Co-authored-by: Adrian Torres <adt@odoo.com>
Co-authored-by: Sébastien Theys <seb@odoo.com>
This fixes an issue when a Model A with a `many2one_reference` is inherited in
Model B with `_inherits`, and another Model C defines a `one2many` having as
inverse the inherited `many2one_reference` in Model B.
Without this commit, the value of `model_field` is `False` by default for the
field in B, which would lead to incorrect queries when building
`get_domain_list` for the `one2many`.
TransientModel (wizards) can be annoying because if they have required
many2one fields, those fields will default to `ondelete='restrict'`,
preventing user to delete the comodel records without deleting the
transient model records first.
To improve such case, we default to `ondelete='cascade'` for required
many2one field on a TransientModel (unless specified otherwise)
closesodoo/odoo#36738
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This "bugprovement" resolve the old issue of writing on a translatable
field a falsy value.
Before this commit, only the translation value was removed. When no
translation is present (or when the value is not set), the value is
retrieved directly from the source field.
Before 18d9c2cab2, it was still possible to go around this issue by
removing manually all ir.translation entries and writing with a user
in en_US.
This is no longer possible as writing on a translatable field always
uses translations when in multi-language environment, without
exception of en_US.
When writing a falsy value on a field, the expected behaviour is to
reset the value and see that falsy value after saving, not the old
value stored on the source model.
Remove all translations and force to update the column.
Removing one individual translation is still possible in the
translation popup.
Task id: 2062415
closesodoo/odoo#37077
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
The value of that field can be given as a dict containing the fields of
the corresponding record.
Co-authored with Michael Mattiello <mcm@odoo.com>
closesodoo/odoo#36779
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This patch avoids to create new browse records for nothing,
(as creating new browse (__new__) is costly),
and to avoid to loop multiple times on the records
(with multiple different filtered, etc.)
This brings performance gain to __set__,
which is used when setting the value of a compute field
e.g. This improves the performances of
`env['product.product'].search_read([])`
closesodoo/odoo#36006
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Instead of using a filtered with `cache.get`,
use a dedicated method in the Cache class to get
the records having a different value in cache then asked.
This is mainly to avoid the creation of intermediate
`browse` of 1 record, when doing `for rec in self`
in `filtered`.
Creating browses is costly, and avoiding it leads
to performance gains.
The dedicated method `get_records_different_from`
loops on the record ids, instead of on browse records
Consider a model M that defines `display_name` as a computed stored
field, then an extension of M that introduces a mixin model A before
that definition in the MRO of M's class. The model A is expected to
have an automatic, non stored field `display_name`, while M must have
its non-automatic, stored field `display_name`.
closesodoo/odoo#36427
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Determine `model._rec_name` before the dependencies of the fields. The
compute method of the automatic field `display_name` uses a callable
depends that retrieves `model._rec_name`.
closesodoo/odoo#36484
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This prevents a bunch of queries that are useless when creating a
record. Indeed, right after a record has been created, no other record
has a many2one reference to it. In other words, inversing a many2one
field from the record just created always gives an empty recordset.
Those useless inversions generate about a dozen queries when creating a
`res.partner`, for instance.
This saves queries, but not much time.
closesodoo/odoo#36566
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The ORM overrides the default defined programmatically on the field, and
set a default function that retrieves the value in `ir.property`.
closesodoo/odoo#36545
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Assigning `check_company=True` to a field will
- set a default domain filtering the companies
- allow to call `_check_company` on the records to ensure the domain is
respected
Setting `_check_company_auto = True` on a model will ensure
`_check_company` is called at create and write, enforcing the multi
company domain.
Joint work with Raphael Collet <rco@odoo.com>
task-1985992
When a translated field is set on new records, one should not update
translations in 'ir.translation'.
closesodoo/odoo#36138
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Martin Trigaux