Previously, mapped was following a very naive approach, which was simply
calling the field name passed as input for every record in a recordset,
sequentially.
The problem with this approach is that we will potentially recompute the
same fields multiple times for differents records, when this could be
done once per field for ALL records, and store this value in cache for
further access.
Another potential problem is that we don't take advantage of the ORM's
prefetching to fetch all the records that are not in cache at once,
instead of doing the same query for every record in the recordset.
Yet another problem is the conversion of each cache value to a record
format and then combining all of the individual records into a single
recordset, which, depending on the size of the recordset, can take an
unbelievable amount of CPU time.
With this new implementation of `mapped()` we take care of all of these
problems:
This is done by first delegating `mapped()` from the model to the field,
this mapped takes a recordset as input and it will try to batch compute
and prefetch as much as possible for the entire recordset, but it will
not keep these values for the actual output, it just stores everything
in cache and then at the end, retrieves everything from the cache to
guarantee the same order.
After the mapped, the conversion from cache format to record format is
delegated to the new `convert_to_record_multi` which will fetch all the
ids and then perform a single browse to encapsulate all of the records
into a single recordset with the least amount of overhead possible.
Part of Task 2170344
closesodoo/odoo#42611
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
1/ When the new database is created without demo data, the admin has a
'silhouette' as a default picture. When a new user is created without
picture given by the current user, the new user will have a 'silhouette'
as a default profile picture.
2/ web: image for fa-user-slash. This image will be used when a record is
unassigned.
3/ web, *: Change placeholder by default when record is unassigned.
We want to have a fa-user-slash icon when a record is unassigned instead
of 'placeholder.png'. A method is created in the BaseModel to have a
generic method to change easily the placeholder for other models.
4/ Adapt kanban test to keep the same behaviour. Attention the behaviour is
a bit different. Because, now the default image is given by the server to
change easily the default image when a record doesn't have an image.
Thus, we don't say if it's the default placeholder, but we can say it's
not the same image of the record (in this test, the record, it's the
partner).
5/ misc: display 'Unassigned' in the hover on kanban cards if record is unassigned
closesodoo/odoo#41356
Taskid: 2060206
Related: odoo/enterprise#7758
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
Co-authored-by: jdoutreloux <jud@odoo.com>
Co-authored-by: Yannick Tivisse <yti@odoo.com>
Fix the prefetching mechanism to never consider a recordset with
duplicates.
closesodoo/odoo#43426
X-original-commit: c312e1e5e5e23d1d8dc172aa6e5f8f07f5638792
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Apparently some solutions (e.g. bitnami) deploy odoo using async
workers, and not all pg/psycopg2 features are supported in that mode,
notably COPY FROM (bulk-copying data from a stream to postgres). Work
around this issue by disabling "async mode" as we do the copy (by
resetting the wait callback) then re-enabling it.
While this is not an officially supported run mode, it should Do No
Harm™ for normal operations and could help users and clients.
Of important note: while this fixes an error running in async mode, it
will also prevent the worker from yielding while copy_from is
executing. Hopefully that doesn't take too long (as the entire point
of the copy_from is to be fast) but there you are.
Fixesodoo/odoo#24145closesodoo/odoo#43481
X-original-commit: c4583cf89c2b24085a2eeafea5ef76f474c5ff84
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Purpose
=======
If check company is set on a field and if the user has no access
right to the field value (example: address_id on an expense sheet
could be set to a private res.partner, on an onchange method when
setting the employee), then the check_company mechanism will
raise an AccessError when trying the validate the companies on
the different records.
Specification
=============
As we only wish to validate the new record values and not the
access rights, the validation could be done as a superuser to
avoid unecessary errors.
closesodoo/odoo#43240
Taskid: 2170006
X-original-commit: 37a9b6c63dcbc268013fbe1a890cf9390eb8e223
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
* add a runtime warning when an onchange method contains the string
'domain'
* add a lint test to forbid the string 'domain' in onchange methods,
this is easy to defeat (e.g. `dict(domain=...)`) and can have false
positives (literal `'domain'` strings for reasons other than an
onchange result) but overall it seems to work nicely
* implemented as a non-pylint test as pylint takes ages to run. While
at it, remove the long-disabled and long-useless
PEP3110TokenChecker (checks for `except Exception, a` which is a
syntax error in P3)
Task 2115472
closesodoo/odoo#41918
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The orderby specification is a string, not a list
Using a list as a orderby clause produces an error in the construction
of the group by query
closesodoo/odoo#42451
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Assume F and G are computed by the same method on a missing record R.
During recomputation of F on R, the compute method is called but fails
because R is missing. Both fields are re-marked to compute (because
computation failed), then F is discarded (because R is missing). Then
comes G's turn: G is accessed on R and the computation fails. Both
fields are re-marked to compute (because computation failed), then G is
discarded (because R is missing). Now F is marked again to compute: the
process never ends.
To avoid this situation, discard all fields to recompute on missing
records.
closesodoo/odoo#42234
X-original-commit: 78bf4dbaa1c4adfa1dac68a4d79b87800003ac05
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
While x_name has long been automatically supported as an equivalent
of name, x_active was not.
This commit adds this behaviour OoB, both in the ORM and the web client.
On the ORM-side, a new `_active_name` attribute is supported on models.
This attribute specifies the field that should behave as an active marker
for records of the model. It is supported the same way `active` has been
until now (filtering by the `active_test` context key and toggled by the
`action_archive`, `action_unarchive` and `toggle_active` methods).
Although no check has been added on the field's type, it is assumed to
be a boolean field (the same way no check is present on the `(x_)name`
field).
On the client-side, the list view and form view now both support
detecting the presence of either `active` or `x_active` on records,
automatically adding an '(Un)Archive' button in the Action menu if such
a field is detected.
Note that the ORM implementation does actually need the field to be named
in any specific way, but since the web client has no mechanism to load
information regarding a model (the lifecycle of an action loading
includes loading the action, view and record(s) but no generic
information about the model itself besides what is included in the
views), we restrict the field's name to `(x_)active` to avoid confusion
as any other name would work at the ORM-level but not in the client.
In the future, the client might be able to more elegantly get
information about models, but this was not the scope of this change and
this solution should cover most cases.
Note that the `active` field will always takes precedence over the
`x_active` field to avoid confusing the polarity, even if both fields
are present on the model.
In the case of a custom field, it might be slightly annoying that the
default value of a Boolean field is `False`, which means that upon
adding the column, all existing records are automatically archived. This
can easily be worked around using an `ir.default` record for that
particular field and an update of existing records (e.g. through the
list view). The goal of this change was not to make it easy to add
support for custom active fields, but to make it possible - we have
therefore kept this implementation which introduces few changes while
adding enough flexibility for developers.
See https://twitter.com/zubair_shafiq/status/1202587553871880192?s=20
for more info regarding supporting any `_active_name` in the web client.
Co-authored-by: Raphael Collet <rco@odoo.com>
The method `unlink` checks whether any of the records to delete is used
as the default value of a company-dependent field. The query simply
blows up when deleting several millions of records. The fix consists in
performing the check in batches.
closesodoo/odoo#41331
X-original-commit: 30e8b8f2554e5d16b19f16ea90e3503098dc98a0
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Forward-port of ffee5bdbd12b0080349da515d01981943355d0cd;
because of 1a315e87b05fcbdefcb3db6d8d4837f6cc3e835c by the same author,
only the following are left:
- we need to translate the untranslated message
- give the temporary cursor a name different from cursor or cr
so that _'s _get_cr method does not get the closed cursor
closesodoo/odoo#41140
X-original-commit: 926873031d479ec96ca122fe8c3a481b31f2a32b
Signed-off-by: Nans Lefebvre (len) <len@odoo.com>
From now on, if one wants to force following operations to happen in a given company,
use with_company(company) or with_company(cid) to update the environment.
View creation/edition represent a important part of an install and a lot
of possible view errors are not detected, like fields used in domain
filters. Some part of the code a difficult to maintain, and view checks
are splitted in multiple places.
This commit aims at refactoring view validation by regrouping most part
of the logic in ir_ui_view and trying to optimize the overall process.
Since most of the lines were touched, this task was also an opportunity
to modernize the API.
Main changes on method `check_xml`:
- extract node processign and validation to individual postprocessor
- add validation for filter node, buttons, ...
- fix accessibility checks (and improve their performance)
- move xpath check to specific Python node validator
- clarify error messages (wip to continue)
Main changes on method `read_combined`:
- optimize the search for inheriting views in a single query doing the
whole recursive search
Indeed, after removing xpath validations, `get_inheriting_views_arch`
was the most expensive method in `check_xml`, spending most of the time
in `search` because of recursive calls to retrieve children views.
The view Backend Assets is a good example of the latter point, since a
line is added in the view for almost every module. 70 views (community)
are added at first level, `get_inheriting_views_arch` is efficient and
returns all 70 views. Then at the second level, the method
`get_inheriting_views_arch` is called 70 times for nothing. 70 calls to
`search` (squared/2 since each view is checked independently) are almost
useless. The same case applies to the settings view.
As a result, the average module installation time is 25% faster, and the
average time spent in `read_combined` is almost divided by 2.
When testing the import of records, a flush should be done to check
all databases constraints and fields computations.
Moreover, in case an exception is raised within a savepoint,
the environment should be cleaned up, to avoid tricky bugs (see below).
We replace the manual handling of savepoints with the savepoint
context manager, which already handles all of this.
This code predates the context manager, so in a way it was archaic.
When testing the import of records, the following could happen:
"An unknown issue occurred during import (possibly lost connection,
data limit exceeded or memory limits exceeded)."
This would happen on project tasks; the fields 'working_hours_open',
'working_hours_close', 'working_days_open', 'working_days_close',
all depend on the computation of _compute_elapsed.
Because this is a test import, the records would raise a MissingError,
and fields_to_compute would always contain 3 of the fields.
The resulting is an infinite loop giving this error.
opw 2092134
closesodoo/odoo#39682
X-original-commit: 1a315e87b05fcbdefcb3db6d8d4837f6cc3e835c
Signed-off-by: Nans Lefebvre (len) <len@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The purpose of this patch is to avoid `filtered_domain` to crash with
domains on date/datetime fields when the fields are `False` on some
records:
records.filtered_domain([('date', '<', '2019-10-28')])
closesodoo/odoo#39449
X-original-commit: dae379adebedb3ed9bd38a2d6925ba6aefef1e7f
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The queries made by the method `_is_an_ordinary_table` represent about
8% of the time to do a full Odoo installation. Use a cached query for
all tables on the registry to reduce that time to some negligible
amount.
closesodoo/odoo#39262
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The basic ORM methods should not silently discard unknown fields, as
they may be a sign of broken code.
X-original-commit: 9595dd06a1d349b2fb84969374d57c32a5faab74
- Let a model M define a Many2oneReference field F1.
- Let a model N define a One2many field F2 whose inverse is M->F1
- Define a computed field N->F3 that depends on N->F2
If, for whatever reason, a recordset of M contains records that have
been unlinked already and we try to unlink them again, the system will
crash with a MissingException error.
This happened because, while most _modified_trigger cases cover the case
of a MissingException (i.e. record not in cache), the case for a
Many2oneReference didn't.
This is fixed by simply ignoring these "stale" records in the
_modified_triggers section for Many2oneReference fields.
closesodoo/odoo#38816
X-original-commit: f3b05032f0576befda4dca870718afd429c21b0f
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Co-authored-by: mreficent <miquel.raich@eficent.com>
Because of 4b1cb41cf7, we might try to read
the field 'active' on a record on which we can't read fields besides the name.
This thus triggers an access error where there should not have been.
In particular, this is the case for portal users: most often the records they
can access point to records that they can't read
(e.g. the partner of the internal user assigned to the ticket).
As a result, clicking on any link creates an ACL, and thus redirects to the
home page.
It turns out that filtered_domain was primarily used on already loaded records,
typically for the write, so it was assumed that the records could be read
in the first place.
However in the use-case of the portal, there is an explicit check on the read
rights with the portal user, explaining the discrepancy.
Since in the general case filtered_domain should be able to read all fields
to evaluate the domain, we put it in sudo.
fix co-authored with @rco
closesodoo/odoo#38540
X-original-commit: cff5cc8cc78a32eb204368e84ec4980324046405
Signed-off-by: Nans Lefebvre (len) <len@odoo.com>
Example of workflow to reproduce the issue:
- Add a selection field 'Test' to `project.task` with Studio
- Go to Projects > Planning > By Project
- Switch to list view
- Add the newly created field to the view 'Task > Test'
The interface loops indefinitely because the server is stuck in an
infinite loop.
The loop is the following:
https://github.com/odoo/odoo/blob/89270ee39c0daa46ae8e2e3e57d7e512a1a65ddf/odoo/models.py#L5624-L5625
The root cause of the issue comes from an incorrect object ID added in
the list of fields to recompute. Since the method is set in a
`post_init`, the object `field` is different from `recs[field_name]`.
To prevent the error, we retrieve the object directly in the
`mark_fields_to_compute` method.
opw-2083852
closesodoo/odoo#38550
X-original-commit: 446cd3f3d610c9a872969f5d6843ccb6ebdb0d40
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Co-authored-by: Denis Ledoux <dle@odoo.com>
One2many fields can use it actually
closesodoo/odoo#38193
X-original-commit: 765566032e60252ec8fb7800200ed05addbfce65
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
en_US may not be activated as it is possible to create a database in
another language using the database manager.
When trying to install a chart of account, the tax return entry tried
to format a date at the installation of the module, with no lang in
the context. The fallback was made on en_US but an error is raised if
that language is not activated.
As it is a very common scenario to retrieve a language from the
context, add a generic tool method to do it.
Replace and closesodoo/odoo#37629closesodoo/odoo#37568
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Before this commit, when a record of a model who has a `res.users` field
it could fail the `check_company` because the check looks for
`company_id` who is the default company of the user.
In the case of `res.users`, `check_company` must look after the
`company_ids` of the user.
When a model inherit from another, it still must have a _description
specified if the _name is different
Before this commit, mail.test.cc inherited from mail.thread.cc without
specifying a _description but this was not raising any issue
This makes sense because in `create` the "previous" value of a field is
obviously non-existent.
This avoids having to fetch pre-existing attachments when they are known to be
inexistent, which optimizes stored related binary that are computed post-insert.
Some model can have really complex _order, which can induce unnecessary slow
queries; here as we don't need the right business order we force ordering by
'id' instead
OPW-2007478
closesodoo/odoo#36660
X-original-commit: a1fe8976156a7b6e14aa2d68650176a0e64ba73f
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
6a366b21 was aiming to uniquify translations, this is not usefull
anymore because of unique constraints added on ir_translation table.
Removing the subselect slightly improve performances.
This fixes 957c78b98f to make it
compatible with 55e1b44492.
Upon record creation, we do not set a company-dependent field when its
value is the same as its default in `ir.property`. Here we take into
account that `field.default` does not necessarily return that value.
closesodoo/odoo#36905
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
various UI changes
------------------
* renamed "test import" button to "test"
* move relation fields thing to debug mode
* remove "Defer parent/child computation" option as it was deprecated
/ removed from the backend in
80f1ac3599, turns out this checkbox
existed inactive for longer than it's been of any use (added in
68cb2ade09 on 2017-11-29, made
non-operating on 2018-01-24, that's so sad)
batching
--------
* add support for batching imports (skip & limit parameters)
* modify client to use batched imports & properly adapt responses so
it still looks like a single import for the client (more or less)
e.g. update row numbers in error messages, etc...
* properly handle partial imports though
* disable usual loading throbber to have a single progress
notification displayed continuously throughout all the batches: the
normal throbber only shows after 3s of waiting for an RPC response,
so it would keep flashing in and out (appear 3s into a batch's
import then disappear at the end only to reappear 3s into the next
batch's loading)
NOTE: the limit is row-wise. If a record straddles the limit (because
of nested O2M records), the record is imported in full and the "next
row" is whatever row follows the record. This means a limit of 10 can
lead to an import of 17 lines, and as the progress indicator is in
records# the increments can jump around.
Task 2059448
Purpose
=======
On a record creation, a valid and user friendly error message is raised
when a sql constraint is violated. (Eg: 'The start date must be anterior
to the end date').
On a record modification, this is not always the case anymore, since the
latest ORM modifications.
Eg: Write on a leave with a date_from > date_to, you got the postgreSQL
error, which is not understable by a classic human.
This is due to the fact that the error is correctly catched and logged,
but the result is set into the cache, triggering the execution of computation
methods, and calling _validate_fields. It is highly probable that a flush
is done in the constraint method (search, read, ...). In that case the
postgreSQL error is converted into a ValidationError, with a stringified
version of the IntegrityError, and thus is not catched by the check wrapper.
Adrian Torres