The purpose of onchange2() is to adress two shortcomings of onchange():
- reduce the payload of the RPC call by minimizing the diff
- use the "unity" format for returning the data
Because of the dependency of onchange2() on web_read(), the new method
has been introduced in module web.
closesodoo/odoo#119510
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Julien Castiaux <juc@odoo.com>
Co-authored-by: VincentSchippefilt <VincentSchippefilt@users.noreply.github.com>
Rationale
---------
Given the following domain:
['|',
('company_id.name', 'like', 'BE'),
('company_id.city', 'like', 'Brussels')]
The ORM would generate the following SQL query:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."name" like 'BE'
) OR "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."email" like 'help@odoo.com'
);
Which is sub-optiomal as the WHERE clause of the two subqueries could be
regrouped inside of a single query like so:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company" WHERE (
"res_company"."name" like 'BE'
OR "res_company"."email" like 'help@odoo.com'
)
);
Postgres-wise, it is faster to execute the latter query than the former
one. This commit is about optimizing the ORM so that it generates
queries where the WHERE clause of compatible subqueries are grouped
together.
Technical solution
------------------
The solution explored by this work is to replace the regular relational
field accesses (over a dotted path) by a new `any` operator that look as
follow:
(relational_field, 'any', domain)
For instance, the above `('company_id.name', 'like', 'BE')` leaf becomes
`('company_id', 'any', [('name', 'like', 'BE')]` using `any`.
Having a domain as right-hand-side allow for the combination of the
domains of same compatible relations:
[('company_id', 'any', ['|',
('name', 'like', 'BE'),
('city', 'like', 'Brussels')])
Having combined domains allows for generating better SQL queries with
minimal changes to the expression parsing algorithm, which can only
translate a single leaf at a time to SQL. With the new `any` operator,
it is still a single leaf but the right-hand-side contains the merged
domain, thus the combined SQL WHERE clause is immediately generated.
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
This commit hads a few tests that report the existing behavior (before
merging subqueries WHERE clauses).
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
A mistake introduced in 234db70d86, the
`groupby` of `_read_group` should be list/tuple of `str`, not a `str`.
closesodoo/odoo#119400
Signed-off-by: Raphael Collet <rco@odoo.com>
Introduce an optimised way of reading a graph of data from the webclient.
Before this commit:
When reading data from the webclient it could at most read multiple ids of the same model in one RPC.
This mean that when reading x2many or specific information on many2one, that could only be done after the initial read (when the client knows the ids of the comodels) and model by model.
After this commit:
Introduce methods web_read and web_search_read_unity. Both method receive a specificiation for the fields instead of a list of fields. The specification can request fields from the model, as well as follow relations and request fields for each relations, recursively.
Example of web_read specification for account_move
```python
{'name': {}},
{'date': {}},
{'journal_id': {'fields': {'display_name':{}}}
},
...
{'invoice_line_ids' :
{
'fields': {
'journal_id' : {'fields': {'display_name:{}}},
'move_name' : {},
...
'tax_ids' : {
fields: {
'display_name':{},
...
}
}
}
}
}
```
Result for this example with 2 invoice lines
```python
{
'id': 1234,
'name' : 'invoice name ABC',
'journal_id: {
'id': 999,
'display_name': 'Customer Invoices'
},
...
'invoice_line_ids': [
{
'id': 666,
'journal_id': {
'id': 999,
'display_name': 'Customer Invoices'
},
'move_name': 'a move name',
'tax_ids': [
{
'id': 333,
'display_name: "15% tax",
...
}
]
},
{
'id': 667,
'journal_id': {
'id': 999,
'display_name': 'Customer Invoices'
},
'move_name': 'another move name',
'tax_ids': [
{
'id': 334,
'display_name: "21% customer tax",
...
}
]
}
]
}
```
closesodoo/odoo#119034
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
This commit fixes a use case where a user would not be able to access the
property field values of a record because he did not have access to that
record's parent (from the property field definition configuration).
In a concrete example:
- In the Knowledge App
- Marc Demo has access to the "Article B".
- The parent article of "Article B" is "Article A".
- Marc Demo does not have access to "Article A".
- Article B has some property fields values configured.
- If Marc Demo tries to open "Article B" -> crash
This is caused by the framework having to read the property definition from the
parent record, which we don't have access to.
A "sudo" was added to allow that use case (which is supported).
In addition, this commit also adds further checks on the web client side to
gracefully handle a special use case where the end-user could not write on the
parent record because of record rules.
(It only checked *access rules*).
To avoid making an extra RPC every time we load the record (to check if we can
write on the parent), we only make that check when the user tries to modify
(add/edit/remove) a property and show a proper error message.
(This also applies when creating new property field tags).
Task-3062064
closesodoo/odoo#118929
X-original-commit: bc538f6944461a642bac0f757ec96d7f8cc14c9a
Related: odoo/enterprise#39946
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Currently, the property fields of type integer and decimal does not
behave like the standard fields. There are the following issues:
1. The property fields of type integer and decimal do not fallback to
the value 0 or 0.0 when the field is emptied.
2. The user can not write the value 0 in the property field of type
integer and decimal. The value gets discarded whenever the user
unfocuses the input field.
This commit will fix those two issues and ensure that the property
fields have the same behavior as the standard fields.
task-3226202
closesodoo/odoo#117253
X-original-commit: 08c317743ba90802f471058a8f12d7211f9eb150
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When we create a new record X and its values contains an one2many field
with either a recordset Y or a Command.set(ids), the ORM generates extra
SQL queries to search and remove the existing lines of the new record.
The call chain is as follows:
Model._create
-> _RelationalMulti.create
-> _RelationalMulti.write_batch
-> One2many.write_real
But because record X is brand new, it has no lines to begin with. The
fix consists in skipping the search and removal in that case.
Also remove any nondeterministic behavior of method write_real() using
an OrderedSet instead of a builtin set.
closesodoo/odoo#114726
X-original-commit: 428827e5213ee002a5b50d864f8ce1f24d1842b3
Signed-off-by: Raphael Collet <rco@odoo.com>
Consider a form view with a one2many field, which has no form subview.
Also the form view of the comodel (the one2many field's lines) contains
the inverse many2one field of the one2many field. When adding a new
line on some existing record, the form view shows the many2one field as
empty, instead of being the main record.
Explanation: the form view of the line invokes onchange() with the main
record's values (dict) as the value of the many2one field. Inside
onchange(), the field is actually set to a new record corresponding to
the main record. Alas, when that value is sent back to the form, the
new record is serialized as False.
Solution: let onchange() serialize the new record as its origin record
instead.
closesodoo/odoo#114648
X-original-commit: d137ea4915da8d46909fcb082b5a88fd347874a4
Signed-off-by: Raphael Collet <rco@odoo.com>
Steps to reproduce:
- Go to Contact-Configuration-Countries
- Remove the code of a country
- Create a new contact
- Select the country from which you deleted the code
- Put any vat number starting with country code you deleted
Issue:
Traceback
Cause:
in `_run_vat_test` we want to `country.code.lower()` -> country code does not exist
Solution:
Prevent the user to delete a country code by making the field required.
sentry-3923412146
closesodoo/odoo#113207
Signed-off-by: William André (wan) <wan@odoo.com>
Steps to reproduce:
- Install `Contacts` module
- Create 3 contacts with the same name
- Archive one of the contacts
- Go to list view
- Edit search filter to display archived and non archived contacts
- Select the 3 created contacts
- Open `Action` menu and click on `Merge`
Issues:
- Archived partner is set by default as destination partner.
- Archived partner is not displayed in the list of partners to merge.
Cause:
- The default destination partner set is the last record of the record
set returned by `_get_ordered_partner` (where `Archived` partners
are the last ones in the record set).
- The field `partner_ids` does not allow archived records.
Solution:
- Sort the partners to have the archived ones on top (since we took
the last one).
- Set `active_test` to False in the context of the partner_ids field.
opw-3205577
closesodoo/odoo#114508
X-original-commit: 8d0cec6844b248d3787bd7d6b8868d930d25e38c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Nasreddin Boulif (bon) <bon@odoo.com>
Purpose
=======
Allow to search properties fields.
For now, the properties fields are meant to be used with the frontend
top bar filter on views in a later commit, and so everything as been
kept as simple as possible for that particular use case.
Technical
=========
Relational properties
---------------------
We can not search on relational properties using the rec name, or
using one of their fields.
The following domains do not work
```
[('properties.partner_id', 'ilike', 'alice')]
[('properties.partner_ids', 'ilike', 'alice')]
[('properties.user_id.name', 'ilike', 'bob')]
[('properties.user_ids.name', 'ilike', 'bob')]
```
Instead, we should give the ids
```
[('properties.partner_id', '=', 55)]
[('properties.partner_ids', 'in', 55)]
```
IN operator
-----------
The "in" operator is used to either search specific tags / many2many
values, either doing a "OR like" condition on "non-array" value.
It can be used if the left or the right side is not an array
```
[('properties.my_char', 'in', ['a', 'b'])]
[('properties.my_tags', 'in', 'a')]
```
But it can't be used if both side of the condition are array, because
it will require to check type in SQL (which is technically possible,
but add extra complexity, so for now we keep the feature as simple as
possible).
So the following domain is not supported.
```
[('properties.my_tags', 'in', ['a', 'b'])]
```
Instead we should use the "OR" operator
```
['|', ('properties.my_tags', 'in', 'a'), ('properties.my_tags', 'in', 'b')]
```
Tags / many2many - "=" operator
-------------------------------
Tags and many2many have to be searched using the "in" operator.
So the following domain will return unexpected results and raise a warning.
```
[('attributes.mytags', '=', ['a', 'b'])]
```
The reason is that we don't want to dynamically check the type in SQL,
and so we need the "in" domain operator to know what to do in SQL.
Task-2980121
Part-of: odoo/odoo#101901
Purpose
=======
Do not modify the cache in order to improve the performance when we
read in batch. Instead we create a new batched method "convert_to_read"
that check existence in batch, and generate a dict with the result.
Now, all the properties field checks (many2one existence, selection option
still exists, tag value stiff exist, etc) and done in "convert_to_read".
It means that doing `record.properties` won't do all those checks.
Having the batched field fetch at the convert_to_record required too
many changes for the scope of this task, so "record.properties" having
the cache values unchecked is an acceptable tradeoff currently, hence
we batch convert_to_read.
Task-2980121
Part-of: odoo/odoo#101901
This fulfills the goal of searching and fetching fields in a single SQL
query. We introduce the new method search_fetch() for that purpose.
Also introduce method fetch() to fetch some fields for a recordset if
they are not in cache yet.
The call graph is as follows:
search() calls search_fetch()
search_read() calls search_fetch() and _read_format()
read() calls fetch() and _read_format()
search_count() calls _search()
search_fetch() calls _search() and _fetch_query()
fetch() calls _search() and _fetch_query()
The methods _search() and _fetch_query() are usually the ones to
override to implement business-specific logic. The method _search()
returns a Query object to retrieve the records that satisfy the given
domain and are accessible for reading. The method _fetch_query() uses a
Query object to retrieve fields from the database and store them in
cache.
Also use search_fetch() to save one query in search_read() and the
reading of one2many fields.
Part-of: odoo/odoo#112126
The field is updated in SQL but not in cache. Also remove hacks in the
business code to prevent potential issues from this behavior.
closesodoo/odoo#113119
X-original-commit: 8bb660c985c7a1349cae1bf58dc5bf7981bd9058
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
The class is a simple extension of dict, and adds an explicit attribute
for the content of the root node of a tree. It makes the code much more
readable with very small performance overhead.
closesodoo/odoo#111946
X-original-commit: 8a01d74e5f7d035cfd05bc02596f075270e0fcf8
Signed-off-by: Raphael Collet <rco@odoo.com>
Those APIs are aimed at hiding the implementation of trigger trees,
dependent fields and fields modifying relations. Explicit APIs simplify
the profiling of executions and comparison of implementations for
building trigger trees.
X-original-commit: d12b9270375634e738f5288d0c523ac0eec2fa18
Part-of: odoo/odoo#111946
The issue occurs when a computed field depends on a many2many field with
a corresponding inverse field on its comodel. Consider two models like
class User(models.Model):
_name = _description = 'test_new_api.user'
group_ids = fields.Many2many('test_new_api.group')
group_count = fields.Integer(compute='_compute_group_count', store=True)
@api.depends('group_ids')
def _compute_group_count(self):
for user in self:
user.group_count = len(user.group_ids)
class Group(models.Model):
_name = _description = 'test_new_api.group'
user_ids = fields.Many2many('test_new_api.user')
When a user is added to a group with
group.write({'user_ids': [Command.link(user.id)]})
we expect the field `group_count` to be recomputed on `user` only, but
it is actually triggered on *all* the records in `group.user_ids`. This
is a real performance issue when there are many records in the relation.
The explanation comes from the fact that
- the framework considers the field `user_ids` is modified on `group`;
- the field `group_count` implicitly depends on `group_ids.user_ids`,
which makes it triggered on the users `u` such that `u.group_ids`
intersects `group`.
The solution consists in handling the dependencies on inverse many2many
field in the field itself. The field no longer adds the implicit
dependency on its inverse field in the trigger tree, but instead
determines which records in the comodel are actually impacted by the
relation change in the method field.write().
closesodoo/odoo#111943
X-original-commit: bb3a6e378b5f6b2e14b74ce4efb6d749ad54cb1e
Signed-off-by: Raphael Collet <rco@odoo.com>
Update counters according to latest runbot counters. It allows to better spot
side effects of upcoming changes.
Task-2710804 (Mail: Clean MailThread Posting API)
Part-of: odoo/odoo#99482
The revision 65a012c2bba0ed81d9b525a7779c726043b17707 changed the
behavior of method _read() when invoked on a record with id None.
Before the change, the method would consider that no records have been
fetched because the id is falsy. After the change, it keeps the id None
in the fetched records, and attempts to make subsequent SQL queries like
"column IN ()", which is syntactically incorrect.
OPW 3104957
closesodoo/odoo#108785
X-original-commit: 712d03cb2ee5f2fde0e80619be5db38aa09ed1ac
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This is a rare case where method _read() raises a MissingError instead
of just ignoring it.
The issue is triggered by several conditions on a model M:
- at least one ir.rule on M with a domain using a column field on M;
- one deleted record Y which is in the prefetch set of a record X;
- one reads a non-column field on record X.
Fix method _read() to manage that case. It adds an extra call to
exists() in that case, but adds no overhead in the general case.
closesodoo/odoo#108050
X-original-commit: 1876dc87e5c88c711c9b3882c39be704ffc46532
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Rémy Voet <ryv@odoo.com>
this commit improves the performance of importing translations by
1. groupup udpating data for different languages and different modules
2. read model_terms translations data directly with xmlid to get rid of fetching
data from ir_model_data
for modules:sale_management, point_of_sale, industry_fsm, helpdesk, mrp, mrp_plm
(62 installed modules)
time for test_language_install is 45.14% less
for all modules:
time for test_language_install is 68.03% less
closesodoo/odoo#107332
X-original-commit: 727ef9e0a2ca93089b472eb1313117fee8dc876d
Signed-off-by: Raphael Collet <rco@odoo.com>
When creating a record, if a value is related to a monetary field and if
the currency field is a non-stored related one, the value will not be
rounding: when writing the monetary value in the database, we first call
`convert_to_column`. In the parameter, `record` is empty (it is not yet
created) and `values` only contains the stored values (so the currency
value is not present). As a result, `currency` will not be defined and
the value will not be rounded.
OPW-2955202
closesodoo/odoo#106283
X-original-commit: 28196693b50b9d424887391ad13db69c5f5dfe4f
Related: odoo/enterprise#34265
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Adrien Widart <awt@odoo.com>
For a trigram indexed translated field field_x,
domain leaf ('field_x', '=', value) and ('field_x', 'ilike', pattern)
will miss records if the pattern contains accent characters
This commit set ensure_ascii=False for json.dumps to avoid escaping special
characters
opw-3048182
closesodoo/odoo#105240
X-original-commit: 26b5c8c2baeaee9fcd5384aafa5f9b39543d6903
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
There are two issues with the index naming convention used by the ORM:
Problem 1: it is possible to have naming conflict for indexes. For
instance, the name 'slide_channel_tag_group_sequence_index' is used for
both fields slide.channel.tag.group.sequence and
slide.channel.tag.group_sequence. Only the first index will be created.
Solution 1: we separate the model and field names with a double
underscore instead of a single one, which is the same strategy as with
LEFT JOIN aliases. This is correct because model names don't contain
such double underscores or underscores as prefix or suffix (it is not
forbiden but model name should follow the 'dot notation'.)
Problem 2: index names can be longer than 63 chars, but PostgreSQL
silently truncates it. This doesn't actually break anything (PostgreSQL
also truncates values when we check the existence of indexes) but it can
lead to using the same name twice. There is hopefully not any example
in our code.
Solution 2: if the name is too large, we truncate it and pad it with a
hash of the complete name to match 63 characters, which is also the
strategy used for LEFT JOIN aliases.
task-2984730
closesodoo/odoo#100736
Related: odoo/upgrade#3957
Signed-off-by: Raphael Collet <rco@odoo.com>
To be able to create easily a jsonb column in database,
we create a Json type Field. Currently, it is quite limited
field:
- We cannot modified the value in-place, we need to always set
the entire jsonify value.
- No domain operator is done to work with jsonb. Now, it works as a
text field.
closesodoo/odoo#103097
X-original-commit: 7eeba9d205d2dace571b5d0895ddba6290a512db
Related: odoo/enterprise#32729
Signed-off-by: Quentin De Paoli <qdp@odoo.com>
For non-translated field, when search(['name', 'in', list_right])
The behavior of False and the behavior of None in list_right should be the same.
Translated fields need customized process for 'in' operator
X-original-commit: a9a3c666f08c5612432dba40a69c7cfdc54ba96e
Part-of: odoo/odoo#103031
The trigram index function jsonb_path_query_array("column_name", '$.*')::text
uses all translations' representations to build the indexed text. So the
original text needs to be JSON-escaped correctly to match it.
X-original-commit: 7547df664945dddcb839e4903068f7f25ecfc08c
Part-of: odoo/odoo#103031
Purpose
=======
Check the existence of the relational properties (many2one / many2many)
in batch and prefetch the values in batch as well to reduce the number
of SQL queries.
Technical
=========
The existence is checked in the read method of the properties field,
because we have the entire recordset. Then, the non-existing ids are
remove from the properties values and the cache is updated.
Task-2965523
X-original-commit: dba9b684d29c0041a32a5508284573851b8dd097
Part-of: odoo/odoo#102243
Bug
===
If we re-write the same definition record, the default values were
applied again (even if the definition record didn't change).
This is because the compute on the properties is called even if the
definition record didn't change.
Task-2965523
X-original-commit: 70d80771f2430f83c91e7ef46eeebde8de70c3fe
Part-of: odoo/odoo#101487
Bug
===
- create a new helpdesk ticket without properties
- create a new property
- change the customer
=> The added property is removed
The fix in the onchange of models.py needs to be more specific, and
should be used only when the definition record is changed.
Task-2965523
X-original-commit: 909ee004183068d9ffa94abe0d1472a8f0534f34
Part-of: odoo/odoo#101487
Purpose
=======
Log a message each time a user change a properties definition
by writing on the record properties values.
Task-2965523
X-original-commit: 6fa6fcd13e248d4bdf1febb0d8399cfe634037f5
Part-of: odoo/odoo#101487
Purpose
=======
Improve the behavior / visual of the properties field when the user does
not have access to some relational properties (many2many, many2one).
When the user does not have access to one many2many value,
show the others record in the list.
Fix a traceback when the model is not set on the definition.
Task-2965523
X-original-commit: cb8b751a335b01192aab185cd0230fa0f036b974
Part-of: odoo/odoo#101487
Purpose
=======
Shorten the property name to take less space in database (while keeping
a very low probability of collision).
E.G.: "aa34746a6851ee4ea1f8d95746e45788" -> "aa34746a6851ee4e"
Automatically generate property name in python if they are missing.
Task-2965523
X-original-commit: d597083ff5e0a4abde67a450a61e04e6bf119c26
Part-of: odoo/odoo#101487
Purpose
=======
Most of the time we will need to read some field on the definition
record when we read the record itself (e.g. fetching some project
information on a task).
Because of this, it makes sense to prefetch by default the properties
definition, and it reduces the number of queries as well.
Task-2965523
X-original-commit: 1295e76f0dc3f3162a1b81a58ed1dc73d0fce3e7
Part-of: odoo/odoo#101487
The cache currectly fails to correctly invalidate relational fields that depend
on a non-relational field. Two passes of invalidation are done, to reflect
dependencies on both the old and the new written values. In the first pass
only relational fields are considered, as explained in the comments:
> It is best explained with a simple example: consider two sales orders SO1 and
SO2. The computed total amount on sales orders indirectly depends on the
many2one field 'order_id' linking lines to their sales order. Now consider the
following code:
>
> line = so1.line_ids[0] # pick a line from SO1
> line.order_id = so2 # move the line to SO2
>
> In this situation, the total amount must be recomputed on *both* sales order:
the line's order before the modification, and the line's order after the
modification.
The written values can be seen as the roots of a dependency forest (a
collection of dependency trees). Before this commit all non-relational roots
and their corresponding trees were filtered out during the first pass. However,
this approach is wrong, as relational fields can also depend on non-relational
fields. Instead, the complete dependency forest has to be traversed, skipping
invalidation for non-relational fields during the first pass.
The test that was previously included accidentally succeeded because of a
separate and unrelated bug in the orm domain parser: in certain one2many or
many2many leafs the domain parser would not take into consideration the domain
included in the definition of the field. As a result, the test still passed
by accident, because the records that no longer matched the domain after the
write were still invalidated during the second pass.
The problem can clearly be demonstrated, however, when the dependency is
generated by a compute function.
closesodoo/odoo#101038
X-original-commit: d4a5827b42d80f0f830455dcd2056701eb09aed1
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The tour test_01_admin_widget_x2many was failing because of a missing
discussion. This discussion was in fact coming from a patch
during at install tests
This wasn't detected on runbot during merge since at_install and
post_install are executed in different builds. This was detected
during the nightly "all no auto tag" build.
Simply use a proper patch to avoid keeping the default value at the
end of the test.
closesodoo/odoo#100491
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
For non-stored computed fields, `_modified_triggers` will traverse the
tree (at the cost of extra queries) only to know which record to
invalidate in cache. But in most cases, these fields have no data in
cache, so they can be ignored from the start, which allows us to prune
entire subtrees from the merged tree.
By example:
With simple write on `show_operations` of one `stock.picking.type`, the
`_modified_triggers` will fetch every ids (from database) of `stock.picking`
and `stock.move` related to this `stock.picking.type`
(For `stock.move`, it is because of the
`show_operations = fields.Boolean(related='picking_id.picking_type_id.show_operations'`))
In that case, there isn't any data of `show_operations` (`stock.move`)
in cache, then there are nothing to invalidate and
the `_modified_triggers` cost
is high (extra queries/processing) for nothing.
Then we cut parts of the tree when we know that
they won't invalidate anything (=> if the cache is empty for the field).
Also refactor the way to merge trees to be more efficient.
Performance improvements:
- For all tests at-install done by the runbot, we gain -+ 3.5% of
queries.
- In the example above, we reduce the number of queries (potentially
bottleneck queries in large DB) from 7 to 4.
- In term of CPU (without counting time in SQL), the new version is -+
20 % faster (on install of stock,purchase,mrp and with
--test-tags=/stock,/purchase).
closesodoo/odoo#76322
task-2780812
closesodoo/odoo#99274
Related: odoo/enterprise#30951
Signed-off-by: Raphael Collet <rco@odoo.com>
Purpose
=======
Add a new field "Properties" to be able to light customization of workflows
based on a parent model. Those properties acts in some ways like Odoo fields
without requiring specific columns e.g. add new properties on tasks of a
specific project.
Usage
=====
Define properties on a parent model (e.g. project) with
```
attributes_definition = fields.PropertiesDefinition('Message Properties')
```
It defines properties available on children: types, default, value, model for
relational properties,...
Use it on children records (e.g. task) with
```
attributes = fields.Properties(
string='Properties',
definition='parent_id.attributes_definition',
)
```
Technical
=========
Parent | Properties definition
------------------------------
The properties definition is stored on the parent, on a JSON field.
This definition contains the type of the properties, the default value,
the model of the many2one,...
```
[
{
'name': 'name',
'string': 'Name',
'type': 'char',
'default': 'Default Name',
}, {
'name': 'partner_id',
'string': 'Partner',
'type': 'many2one',
'comodel': 'res.partner',
},
]
```
Child | Properties values
-------------------------
The value is stored on the child, using a Properties field.
```
{
'name': 'Mitchel',
'partner_id': 1337,
}
```
When we read this field, we will automatically read the definition on
the parent, and merge both JSON into one, so the web client has the
value of each property, and their definition.
```
[
{
'name': 'name',
'string': 'Name',
'type': 'char',
'default': 'Default Name',
'value': 'Mitchel',
}, {
'name': 'partner_id',
'string': 'Partner',
'type': 'many2one',
'comodel': 'res.partner',
'value': 1337,
},
]
```
Integrity
---------
If we remove a property on the parent, we won't update the child value.
Instead, when we read the child properties, we will filter them based
on the parent. So the removed properties will be removed the next time
we write on the field.
In the same logic, the many2one existence is checked when we read the
field. There's no foreign key between the integer stored in the JSON
in the SQL row corresponding to the record in database.
Write
-----
We can write on the Properties field with a list of field definition
+ value.
Some types are not JSONifiable (like the date, datetime), they are
stored as string in database and parsed when we read the value.
In order to update the parent definition by writing on the child,
you need to add the dict key `definition_changed` or
`definition_deleted`. This is because we need to be able to know
if the definition has been changed without doing extra SQL queries.
Access rights
-------------
A user can add a many2one / many2many property to a model only if he
has the access rights to it.
Many2one / Many2many
--------------------
The model choice of a many2one / many2many properties was subject to
changes.
First implementation stored models in both parent and children to easily
spot changes and avoid complex queries when fetching records, trying to
synchronize them, ...
As this leads to storing a lot of duplicated content we choose to instead
reset the value on the child if the model has been change. We generate a
new name for the property. So it behaves like if we removed the property
and created a new one.
To be able to restore the old value (e.g. if by mistake we changed the
model, and go back to the old model), we store the initial states.
Task-2852259
Part-of: odoo/odoo#95184
Add the possibility to flag a HTML field as `sanitize_overridable`.
The sanitizer will then be bypassed if the user doing the operation is
part of the new group `base.group_sanitize_override`.
The "Settings" users are part of that new group.
If such a user wrote some HTML that would have normally been removed by
the sanitizer, then users without the right to bypass the sanitizer
won't be able to write on that field anymore.
Otherwise, it would sanitize the previously written data.
Such cases are detected, and the modification prevented by the system,
which will warn the user about it.
For instance, with a `field.html(sanitize_overridable=True)`: one being
part of `group_sanitize_override` could write `<script></script>`.
Then, someone not part of the group trying to add an element inside that
field like appending a `<p/>` -> `<script></script><p>New Content</p>`
would not be able to because it would go through the sanitizer and
ultimately, removing part of the original value:
`<p>New Content</p>` (`<script></script>` would be removed).
== Real use case ==
In the website builder, there is 2 editor rights:
- group_website_publisher: restricted editor
- group_website_designer: editor & designer
The designer editor can edit pages and views, while the restricted
editor can't do anything unless he is part of other groups.
In edit mode, the restricted editor will only be able to edit fields of
record he has access to.
For instance, being a sales manager allows you to edit a product
description on the website.
Being an event manager -> edit event. Slide manager -> Slides etc.
As those restricted editor are able to edit those fields, they are
(almost) always sanitized to prevent them to introduce malicious code.
Since those fields are sanitized, even admins / designer editor are not
able to fully use the website builder in such fields.
Some clients don't really care about that sanitation, they'd prefer to
avoid it as they trust their manager and would prefer to have the full
builder capability instead.
This is typically the case in small project (butcher, hairdresser,
reseller etc) and in SMEs.
With the new `sanitize_overridable` feature, they will be able to do
that, as the "Designer & Editor" group now also receive the group
`base.group_sanitize_override` (done in next commit).
Part-of: odoo/odoo#97398
Consider two models A and B, where
- model A has a many2one_reference field 'res_id' with model field 'res_model';
- model B has an auto-join one2many field 'stuff_ids' to A using field 'res_id';
- the field 'res_model' is not flushed on some record.
model | A | B
-----------+-------------------+-------------------
memory | res_model = B |
-----------+-------------------+-------------------
database | res_model = NULL | id = 42
| res_id = 42 |
| foo = 'bar' |
Now, perform a search on model B that should return record with id=42 by
matching some condition on the unflushed record in model A, like:
B.search([('stuff_ids.foo', '=', 'bar')])
Before this patch, the search method would not flush the field
'res_model', which causes the method to return incorrect results. This
patch fixes the issue by ensuring that searches on one2many fields flush
all the fields on which the one2many field depends.
The issue was discovered while working on task 2735672.
closesodoo/odoo#96115
Signed-off-by: Raphael Collet <rco@odoo.com>
This allows to create a repair.order record without
the need to call the onchanges to set the uom and locations
or to set them manually during the `create` call.
For instance, this makes easier to create repair orders
using XMLRPC when you do not use multiple UOMs or multiple locations.
closesodoo/odoo#95321
Signed-off-by: Raphael Collet <rco@odoo.com>
