For non-stored computed fields, `_modified_triggers` will traverse the
tree (at the cost of extra queries) only to know which record to
invalidate in cache. But in most cases, these fields have no data in
cache, so they can be ignored from the start, which allows us to prune
entire subtrees from the merged tree.
By example:
With simple write on `show_operations` of one `stock.picking.type`, the
`_modified_triggers` will fetch every ids (from database) of `stock.picking`
and `stock.move` related to this `stock.picking.type`
(For `stock.move`, it is because of the
`show_operations = fields.Boolean(related='picking_id.picking_type_id.show_operations'`))
In that case, there isn't any data of `show_operations` (`stock.move`)
in cache, then there are nothing to invalidate and
the `_modified_triggers` cost
is high (extra queries/processing) for nothing.
Then we cut parts of the tree when we know that
they won't invalidate anything (=> if the cache is empty for the field).
Also refactor the way to merge trees to be more efficient.
Performance improvements:
- For all tests at-install done by the runbot, we gain -+ 3.5% of
queries.
- In the example above, we reduce the number of queries (potentially
bottleneck queries in large DB) from 7 to 4.
- In term of CPU (without counting time in SQL), the new version is -+
20 % faster (on install of stock,purchase,mrp and with
--test-tags=/stock,/purchase).
closesodoo/odoo#76322
task-2780812
closesodoo/odoo#99274
Related: odoo/enterprise#30951
Signed-off-by: Raphael Collet <rco@odoo.com>
Purpose
=======
Add a new field "Properties" to be able to light customization of workflows
based on a parent model. Those properties acts in some ways like Odoo fields
without requiring specific columns e.g. add new properties on tasks of a
specific project.
Usage
=====
Define properties on a parent model (e.g. project) with
```
attributes_definition = fields.PropertiesDefinition('Message Properties')
```
It defines properties available on children: types, default, value, model for
relational properties,...
Use it on children records (e.g. task) with
```
attributes = fields.Properties(
string='Properties',
definition='parent_id.attributes_definition',
)
```
Technical
=========
Parent | Properties definition
------------------------------
The properties definition is stored on the parent, on a JSON field.
This definition contains the type of the properties, the default value,
the model of the many2one,...
```
[
{
'name': 'name',
'string': 'Name',
'type': 'char',
'default': 'Default Name',
}, {
'name': 'partner_id',
'string': 'Partner',
'type': 'many2one',
'comodel': 'res.partner',
},
]
```
Child | Properties values
-------------------------
The value is stored on the child, using a Properties field.
```
{
'name': 'Mitchel',
'partner_id': 1337,
}
```
When we read this field, we will automatically read the definition on
the parent, and merge both JSON into one, so the web client has the
value of each property, and their definition.
```
[
{
'name': 'name',
'string': 'Name',
'type': 'char',
'default': 'Default Name',
'value': 'Mitchel',
}, {
'name': 'partner_id',
'string': 'Partner',
'type': 'many2one',
'comodel': 'res.partner',
'value': 1337,
},
]
```
Integrity
---------
If we remove a property on the parent, we won't update the child value.
Instead, when we read the child properties, we will filter them based
on the parent. So the removed properties will be removed the next time
we write on the field.
In the same logic, the many2one existence is checked when we read the
field. There's no foreign key between the integer stored in the JSON
in the SQL row corresponding to the record in database.
Write
-----
We can write on the Properties field with a list of field definition
+ value.
Some types are not JSONifiable (like the date, datetime), they are
stored as string in database and parsed when we read the value.
In order to update the parent definition by writing on the child,
you need to add the dict key `definition_changed` or
`definition_deleted`. This is because we need to be able to know
if the definition has been changed without doing extra SQL queries.
Access rights
-------------
A user can add a many2one / many2many property to a model only if he
has the access rights to it.
Many2one / Many2many
--------------------
The model choice of a many2one / many2many properties was subject to
changes.
First implementation stored models in both parent and children to easily
spot changes and avoid complex queries when fetching records, trying to
synchronize them, ...
As this leads to storing a lot of duplicated content we choose to instead
reset the value on the child if the model has been change. We generate a
new name for the property. So it behaves like if we removed the property
and created a new one.
To be able to restore the old value (e.g. if by mistake we changed the
model, and go back to the old model), we store the initial states.
Task-2852259
Part-of: odoo/odoo#95184
Add the possibility to flag a HTML field as `sanitize_overridable`.
The sanitizer will then be bypassed if the user doing the operation is
part of the new group `base.group_sanitize_override`.
The "Settings" users are part of that new group.
If such a user wrote some HTML that would have normally been removed by
the sanitizer, then users without the right to bypass the sanitizer
won't be able to write on that field anymore.
Otherwise, it would sanitize the previously written data.
Such cases are detected, and the modification prevented by the system,
which will warn the user about it.
For instance, with a `field.html(sanitize_overridable=True)`: one being
part of `group_sanitize_override` could write `<script></script>`.
Then, someone not part of the group trying to add an element inside that
field like appending a `<p/>` -> `<script></script><p>New Content</p>`
would not be able to because it would go through the sanitizer and
ultimately, removing part of the original value:
`<p>New Content</p>` (`<script></script>` would be removed).
== Real use case ==
In the website builder, there is 2 editor rights:
- group_website_publisher: restricted editor
- group_website_designer: editor & designer
The designer editor can edit pages and views, while the restricted
editor can't do anything unless he is part of other groups.
In edit mode, the restricted editor will only be able to edit fields of
record he has access to.
For instance, being a sales manager allows you to edit a product
description on the website.
Being an event manager -> edit event. Slide manager -> Slides etc.
As those restricted editor are able to edit those fields, they are
(almost) always sanitized to prevent them to introduce malicious code.
Since those fields are sanitized, even admins / designer editor are not
able to fully use the website builder in such fields.
Some clients don't really care about that sanitation, they'd prefer to
avoid it as they trust their manager and would prefer to have the full
builder capability instead.
This is typically the case in small project (butcher, hairdresser,
reseller etc) and in SMEs.
With the new `sanitize_overridable` feature, they will be able to do
that, as the "Designer & Editor" group now also receive the group
`base.group_sanitize_override` (done in next commit).
Part-of: odoo/odoo#97398
- the x2many tour would finish on a click canceling the creation but
not wait for that to resolve, leading to the driver catching it in
the act (checking before the client had had the time to switch),
just need to wait for the list view to be displayed
- the constraints tour would just drop in the middle of a creation
with an error dialog open, so need to finish the entire thing and go
back to the list view
Part-of: odoo/odoo#96517
This commit aims at removing unuseful help message to:
1/ reduce translators work, to focus on more useful translations
2/ not sending unuseful information in load_views
3/ reduce help message to useful messages, so that we can mark
fields having a tooltip in the future UI.
4/ some cleanup of existing messages too
The main use cases:
- REMOVED: help redundant with the field name, providing no extra info
- MOVED TO COMMENT: technical help messages, that should not be in UX
closesodoo/odoo#97279
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
The new list and form views were merged recently [1], but they
weren't activated because they weren't 100% ready yet. This is now
the case. This commit adds those views to the view registry. As a
consequence, a lot of qunit tests and tours needed to be adapted,
mostly for selector changes.
We also add legacy list and form views to the view registry, with
keys 'legacy_list' and 'legacy_form'. This allows to force those
legacy views when necessary. For instance, we did it in views
using complex custom legacy x2many field widgets that haven't been
converted yet (we have a compatibility layer but it isn't complete
and doesn't support every advanced usecases).
[1] odoo/odoo#92475
Part-of: odoo/odoo#78221
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Géry Debongnie <ged@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Simon Genin (ges) <ges@odoo.com>
Co-authored-by: Francois (fge) <fge@odoo.com>
Co-authored-by: Michael Mattiello (mcm) <mcm@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais (lpe) <lpe@odoo.com>
Co-authored-by: Jorge Pinna Puissant <jpp@odoo.com>
Co-authored-by: luvi <luvi@odoo.com>
Consider two models A and B, where
- model A has a many2one_reference field 'res_id' with model field 'res_model';
- model B has an auto-join one2many field 'stuff_ids' to A using field 'res_id';
- the field 'res_model' is not flushed on some record.
model | A | B
-----------+-------------------+-------------------
memory | res_model = B |
-----------+-------------------+-------------------
database | res_model = NULL | id = 42
| res_id = 42 |
| foo = 'bar' |
Now, perform a search on model B that should return record with id=42 by
matching some condition on the unflushed record in model A, like:
B.search([('stuff_ids.foo', '=', 'bar')])
Before this patch, the search method would not flush the field
'res_model', which causes the method to return incorrect results. This
patch fixes the issue by ensuring that searches on one2many fields flush
all the fields on which the one2many field depends.
The issue was discovered while working on task 2735672.
closesodoo/odoo#96115
Signed-off-by: Raphael Collet <rco@odoo.com>
This allows to create a repair.order record without
the need to call the onchanges to set the uom and locations
or to set them manually during the `create` call.
For instance, this makes easier to create repair orders
using XMLRPC when you do not use multiple UOMs or multiple locations.
closesodoo/odoo#95321
Signed-off-by: Raphael Collet <rco@odoo.com>
In module mail, invalidating 'message_ids' on a mail thread also
invalidates its inverse field 'res_id' on messages. If you haven't
flushed it before, your cache will be inconsistent, as shown by the test
/mail:TestMailgateway.test_message_process_bounce_records_channel.
In module purchase_stock, add depends on report.stock.quantity. This
ensures that when the model is queried after changes in other models,
the data on which the SQL view depends is flushed to the database before
querying that model's table.
closesodoo/odoo#66938
Related: odoo/enterprise#16722
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Vincent Schippefilt <vsc@odoo.com>
Consider two models A and B, a many2many field F from A to B and its
inverse relation G from B to A. When F is modified, G is updated
accordingly in cache. The statement
records.write({F: [Command.link(b.id)]})
is expected to add b.id to records.F, and add records.ids to b.G. In
order to avoid nondeterminism, the additions should be done at the end
of the recordset, and in order. In other words, if none of "records"
are in b.G, then after the statement above, b.G should be updated as
b.G = b.G + records
This patch ensures that "records" are added in order.
closesodoo/odoo#94909
X-original-commit: b8fbc460673827497f6817e1751b5d365d8ae29a
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
No method was readily available to know if a user is `internal` (has
group `base.group_user`), which was inconsistent with other base groups.
_is_internal is now used in the codebase where it is clear that
`.has_group('base.group_user')` is called on a single record.
Part-of: odoo/odoo#85703
Remove most values uselessly specified because giving the same value as
the default one (see _DEFAULT_MANIFEST in odoo/modules/module.py)
* auto_install is Falsy by default
* author is Odoo SA by default
* summary & description are empty strings by default
* application is False by default
* test, demo, depends and data are empty lists by default
This will reduce noise/inconsistencies between manifests specifications,
simplify analysis of manifests content, ...
closesodoo/odoo#90209
Related: odoo/enterprise#26807
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.
e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`
The goal of this revision is to change that so it sends the list of all fields
only once.
In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
- `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
- `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`
The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
As it no longer contains the fields,
the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
which is a dict with as key the model name and as values
the model fields description. It contains the fields description
for all models implied in the view:
the model of the main view and the model of all one2many and many2many fields.
With this change, the fields description will only be sent once by model
implied in the view.
In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.
- one2many and many2many fields views are passed directly in the main view
architecture rather than being put in the `views` key
of the field description.
This is actually easier to treat by the web client,
and this will allow in a future work to cache an entire view in one block
of text rather than having to combine multiple cached blocks of text
to return one view.
- one2many and many2many fields which do not have directly embedded views
have their views directly injected in the architecture,
so the web client doesn't have to do RPC calls to `load_views`
for each one2many and many2many fields not having embedded views.
For instance, this allow to reduce the number of RPC calls to `load_views`
from 8 to 1 when loading the form of `product.product`.
Currently, this behavior is limited to 1 level deep but we consider making it
go all the way down in future works. We did not do it for the moment because
in certain cases it rises the processing time and the size (bytes) too much.
e.g. the sale.order view can be 5 levels deep,
meaning you can reach 4 dialogs on top the main view.
```
sale.order form > order_line > sale.order.line form > invoice_lines >
account.move.line form > asset_ids > account.asset form >
depreciation_move_ids > account.move form.
```
This will also benefit in future works to cache an entire view in one block
of text rather to having to combine multiple cached block of text
to get one view.
- `fields_view_get` becomes `get_view`.
As it no longer returns the fields description,
keeping the `fields` in the name `fields_view_get` no longer makes sense.
Hence removing `fields` from the method name, it becomes `view_get`.
As it gets renamed anyway, we take the opportunity to rename it `get_view`,
which is more in line with the general getter/setter guidelines
in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
This is not mandatory, there is no technical reason to rename `load_views` as
it practically sends the same info as before,
the view architectures and their fields description. Just in another way.
We just take the opportunity of this pull request to suggest a cleaner API:
`_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
`_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
in `_get_view` and `get_view`.
The rationale is that submenu was already no longer used (deprecated)
and the mobile options is introduced.
The mobile options is necessary to tell the server to send the mobile views
for x2many fields (kanban instead of tree).
Instead of adding a new argument each time we add a new option to
`fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
view information. Now, `get_view` returns a tuple with the view architecture
as an `etree` node, and the view as a browse record. The rationale is that all
overrides of `_fields_view_get` were about modifying the arch only
(e.g. changing the address format/re-organizing the address related field
nodes of the partner according to the company country).
To do so, all these overrides were doing `etree.fromstring` to parse the arch
which was sent in text to convert it to an `etree`,
then operations were done on the `etree`,
and then `etree.tostring` was called to convert back the arch to string.
With this change of signature to send the arch as an `etree`,
all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
has been performed in `get_view`:
- `fields` is removed, as explained above,
- `view_id` is renamed `id`,
- `name` is removed, it was unused by the web client,
- `type` is removed, it was unused by the web client,
- `field_parent` is removed, it was unused by the web client,
- `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
(now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
`fields_view_get`, `_fields_view_get` and `load_views` are provided,
with deprecation warnings in them.
- The web client could cache the model fields description
(as it already caches the views),
so it doesn't need to fetch them again if it asks for another view of a model
for which he already has the fields description.
If we do so, `get_views` could return only the list of models used by
the views, without the fields description as of now,
and the web client would then call `fields_get` independently only for
the models for which it doesn't have yet the fields description.
This would avoid the server to return the fields description
and to call `fields_get`, which is costly, for each `get_views`,
therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
This is already done for qweb views, it's not done for back-end views.
Therefore the postprocessing of the views is performed for each `get_views`,
which is costly, while the view architecture doesn't change for users
belonging to the same groups, according to the groups implied by the view.
This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.
Part-of: odoo/odoo#87522
This is a followup of #86567, where we change the computation of
display_name to match the values of name_get(). Forcing its value to
False instead of the empty string causes many regressions in frontend
applications like website_blog.
closesodoo/odoo#88438
X-original-commit: d467881f66b11021005005584854f30059971b6c
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
This fixes a problem that occurs with conditions on one2many fields,
where most conditions are translated with a subquery like
"model".id IN (
SELECT "comodel"."model_id" FROM "comodel" WHERE ...
)
The issue occurs when the subquery return NULL values, i.e., when the
field "model_id" in the subquery above can be NULL. The fix simply
consists in adding the condition "model_id IS NOT NULL" in the WHERE
clause of the subquery.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
closesodoo/odoo#87285
X-original-commit: 6d680bac0333c6c7ce9f056cba0e1f7e9fa22f9c
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This fixes two problems that occur with conditions on many2many fields:
the subtle bug caused by NULL values that are returned by subqueries,
and the performance of those subqueries.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
Regarding performance, consider a domain like [('tag_ids', '=', False)].
It is currently translated in SQL as
"model".id NOT IN (
SELECT "model_id"
FROM "model_tag_rel"
WHERE "model_id" IS NOT NULL
)
PostgreSQL does not handle well this "NOT IN" expression when the
relation table is large, and uses some very inefficient query plan in
that case. The inefficient query plan is possibly related to the
special handling of NULL values, by the way. This commit changes the
above expression to a "NOT EXISTS" expression (as below), which is
executed with a much more efficient query plan, even with large tables.
NOT EXISTS (
SELECT 1
FROM "model_tag_rel"
WHERE "model_tag_rel"."model_id" = "model".id
)
This change was motivated by a use-case with 2.7M account moves, 11M
account move lines and a many2many relation table with 2.3M lines
(`account_move_line_account_tax_rel`). The query was timing out (15
minutes), and it now takes 15 seconds.
We have replaced all the conditions on many2many fields to use the
relational operators EXISTS and NOT EXISTS.
A test has been added to reflect a case where the many2many table
contains NULL values, like the one for channel_partner in regards to
partner_id and guest_id.
task-2753782
X-original-commit: 334b2aa2aa437d2ed86130f2ed426025f8fe4b86
Part-of: odoo/odoo#87285
Co-authored-by: Didier Debondt <did@odoo.com>
Because of 4347491670, the method
name_get() can now return records with False as label, and this makes
code crash when it expects a string. Make sure that name_get() always
return strings as labels, while keeping False as empty value for the
field 'display_name'.
closesodoo/odoo#86688
X-original-commit: ff37a2fef89df292b52b1501c28fcf841e1628f5
Signed-off-by: Raphael Collet <rco@odoo.com>
The issue is simply creating a related stored many2many field, which is
pretty easy to do with Studio. When creating the field, Odoo crashes
with a traceback caused by ondelete being None on the field.
The source of the bug is the fact that the attribute field.ondelete is
set to a sensible default ('cascade') on non-related fields only. The
fix consists in setting the default value on the attribute itself, so
that the setup of the field never falls on a case where field.ondelete
is unset.
closesodoo/odoo#86303
X-original-commit: d0e0d80806a1b63ac986e9bcbaf88f9fc5243319
Signed-off-by: Raphael Collet <rco@odoo.com>
The field prefetching mechanism was poorly customizable. Before this,
we could only tell if a field was prefetched with other fields or not at
all. We have no way to inform the framework, like: "When I need data of
that field, prefetch these other fields, which are likely be used in the
same transaction".
From now on, the `prefetch` attribute is used as a grouping key for
prefetching fields. When a field is fetched, all the fields with the
same value for `prefetch` are taken for prefetching.
For example, consider a small set of fields that are rarely used, except
for one flow A using them. You want to prefetch those fields only in
the flow A, and you want to fetch them in a single query. With the new
feature, simply set `prefetch=A` for some string `A` on those fields,
and they will be grouped for prefetching.
closesodoo/odoo#85220
Signed-off-by: Rémy Voet <ryv@odoo.com>
Just before deleting records, we mark fields on dependent records as "to
compute", and their actual computation should happen after the records
have been deleted. The problem is that checking security rules may
trigger the computations that have been prepared. So we should check
security rules before marking dependent records.
closesodoo/odoo#85574
X-original-commit: 79908bc757a239c97bd2b4cda505b7b8df769325
Signed-off-by: Raphael Collet <rco@odoo.com>
The field prefetching mechanism was poorly customizable. Before this,
we could only tell if a field was prefetched with other fields or not at
all. We have no way to inform the framework, like: "When I need data of
that field, prefetch these other fields, which are likely be used in the
same transaction".
From now on, the `prefetch` attribute is used as a grouping key for
prefetching fields. When a field is fetched, all the fields with the
same value for `prefetch` are taken for prefetching.
For example, consider a small set of fields that are rarely used, except
for one flow A using them. You want to prefetch those fields only in
the flow A, and you want to fetch them in a single query. With the new
feature, simply set `prefetch=A` for some string `A` on those fields,
and they will be grouped for prefetching.
Part-of: odoo/odoo#83818
Purpose
=======
The parent_path is invalidated from cache when computing it from scratch
(with method `_parent_store_compute`) but not when it is recomputed
(method `_parent_store_update`).
The issue is that a computed field depending on 'parent_path'
won't receive the updated value using the cache, and could lead
to inconsistencies if the developer is not aware of that.
Specification
=============
On `_parent_store_update`, set the new value of 'parent_path' in the
cache before marking the records as modified.
task-2766452
closesodoo/odoo#84795
X-original-commit: 861854a7598f31c0b971560484e3eb576af006b4
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Rémy Voet <ryv@odoo.com>
Co-authored-by: Rémy Voet <ryv@odoo.com>
When we are only trying to know accepted raw values, there is no
need to fetch the translations for the description part of the
selection tuples.
This performance problem was first observed on res.config.settings
record creation, where the values are pruned to avoid writes and
invalidation for unmodified related values (cf create override
in base/models/res_config.py & 5a4e7b711e).
After more analysis, the 'problem' also impacts record updates on
any related field (readonly or not).
* Res.config.settings performance gains:
In a database with test_main_flows (& its diverse dependencies),
this reduces by ~50% the number of queries done when triggering
the creation of res.config.settings record through the interface
(aka with all the related updatable fields receiving their current
value as create value).
For the creation of 150 res.config.settings records, with full cache
invalidation between each record creation, we notice a gain of:
* 11853 to 6303 total queries
* 11.2s to 7.5s total creation time
Part-of: odoo/odoo#83104
The `_create` don't batch the SQL request to insert of data which it
isn't efficient we there is a several record to create.
Batch the SQL insertion and measure performance
depending of batch size to avoid any regression (see result in the
PR: https://github.com/odoo/odoo/pull/80961).
task-2299314
closesodoo/odoo#80961
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
The `column_format` of the `Field` class was unused and create useless
noise in the ORM. Then remove it and simplify some flows.
task-2735546
Part-of: odoo/odoo#82727
The `_sequence` was only used for the insertion of data in the DB for
the `id` value. But in Odoo, the `id` is always a `SERIAL`, then the
PostgreSQL fill already the value by the right SQL sequence. Then,
avoid doing the job of the DB by ourselves.
Sadly, we need to manage the case when we create an empty record to
always get a valid query.
task-2735546
Part-of: odoo/odoo#82727
Issue
-----
Via the field prefetch mechanism, when we need a value of one field
(not in cache of course), the ORM will prefetch all fields
(which has the attribute to `prefetch=True`, the default value of this
attribute is `True`) for all record ids in `_prefetch_ids`.
Then, for each translate fields (where translate is not a callable)
the ORM need to make a `LEFT JOIN` on the `ir_translation` to fetch the
translated value. For big model, it leads to a simple `SELECT` with
several `LEFT JOIN` on ir_translation but each LEFT JOIN have a cost
in the planner time (a small cost in the execution time) of PostgreSQL.
By example, for `product.template` (stock/sale/purchase installed),
there are 6 LEFT JOIN to get all translated fields (5 of this
fields are rarely used).
Proposed solution
-----------------
Deactivate the prefetch by default for all translate fields expect if
this field is the `_rec_name` of the model (which is more likely to
be used).
In the example on the `product.template`:
Without prefetching the translated fields, there is only one LEFT JOIN
(the name, which is translated but is the `_rec_name` of the model).
With the 6 translated fields to fetch, the
query takes 5 ms to plan and 2 ms to execute VS with 1 translate field,
it 1 ms to plan and 1.5 ms to execute.
Side change note
----------------
- All translate of fields of `website.seo.metadata` should be prefetch
to avoid lot of website errors (it is because, website put in cache data
in sudo before reading it without sudo)
- `description` (`mail.message.subtype`), `subject` (`mail.template`),
`body_html` (`mail.template`) should be prefetch to avoid lot of extra
query from mail module.
- `vat_label` (`res.country`) should be prefetch to avoid a extra query
for each website page.
- Increase some queryCount (when it is legit, due to `subtitle` of
`blog_post` or `description` of `event.type.ticket`, etc)
task-2738029
closesodoo/odoo#82896
Signed-off-by: Raphael Collet <rco@odoo.com>
If a form view contains the field 'display_name', when creating a new
record, the initial value of 'display_name' is the string "False", and
that weird value also appears in the breadcrumb instead of "New". In
order to avoid this unexpected behavior, the conversion of a value to a
display name should be False, like any other field would.
closesodoo/odoo#81788
Signed-off-by: Raphael Collet <rco@odoo.com>
This fixes inconsistencies when dealing with fields that are computed
and inversed by the same methods.
Consider two fields F1, F2 with the same compute and inverse methods.
Consider a record where we wrote on a dependency of the common compute
method. At this point, both fields F1 and F2 are marked to be computed.
Now let us write on F1 only. Here is what happens:
- the write discards the computation of F1, but not F2
- the inverse method of F1 is called:
- the method accesses F2
-> this calls the compute method, which assigns both F1 and F2
- the method accesses F1
-> the value of F1 has been replaced by the computation above
The issue comes from a combination of factors:
- the value of F2 must be determined by the computation;
- the computation assigns both F1 and F2;
- the computation is done while inversing F1 (and F2).
The solution is to force the computation before actually writing on the
fields and calling their inverse methods. Note that this is necessary
only when part of the fields computed by a common method are updated.
When all fields computed by a common method are updated, the computation
will automatically be cancelled.
closesodoo/odoo#81172
X-original-commit: fb4d6ee4ba8bcb5cd8030105ac57e9e02850bfc4
Signed-off-by: Raphael Collet <rco@odoo.com>
This is a bunch of fixes (65 corner cases) for the search on
company-dependent fields:
Without a default value:
- `char` fields:
- operators `not like`/`not ilike` don't return records with unset value
- `(..., '=', False)` doesn't return records with unset value
- `(..., '!=', '<string>')` doesn't return records with unset value
- `(..., 'in', [..., False])` doesn't return records with unset value
- `(..., 'not in', value)` without `False` inside `value` doesn't return records with unset value
- `date` and `datetime` fields:
- `(..., '!=', <Date/datetime>)` doesn't return records with unset value
- `(..., '=', False)` doesn't return records with unset value
- `many2one` fields:
- operators `not like`/`not ilike` don't return records with unset value
- `(..., 'in', [..., False])` doesn't return records with unset value
- `(..., 'not in', value)` without `False` inside `value` doesn't return records with unset value
- `boolean` fields:
- `(..., '=', False)` and `(..., '!=', True)` don't return records with unset and `False` values
- `integer`/`float` fields:
- `(..., '!=', <number>)` doesn't return records with unset value
With a truthy default value:
- `many2one` fields:
- operators `not like`/`not ilike` don't return records with unset value
- `(..., '=', False)` returns the record with the default value (which isn't `False`)
- `boolean` fields:
- `(..., '=', False)` doesn't return records with unset and `False` value
- `(..., '!=', False)` returns records with unset and `False` value
- `integer`/`float` fields:
- all `(..., operator, value)` which include value 0, return all records with unset value, even if the default value does not satisfy the domain
closesodoo/odoo#80994
X-original-commit: 3e3be652ece83420782070bdb13da2c3a4930046
Signed-off-by: Raphael Collet <rco@odoo.com>
Until now, stored compute fields were computed after database insertion.
This meant that required fields should not be computed, for instance,
unless some hackish code was added to make it work. Another trick was
to provide some default value, but this actually prevents the field for
being computed after insertion.
This commit provides a field parameter to specify that the field should
be precomputed: adding precompute=True on the field definition force the
method create() to compute its value before inserting the new record in
the database. For the reason explained below, precomputing fields is
not always correct, and therefore the default remains to not precompute
a field.
Some stored fields must be computed after insertion, for instance:
* statistics fields computed with search/read_group/...
* fields referencing the current record (res.partner.commercial_partner_id)
* fields referencing another record that does not exist yet (think about
records created by one2many fields)
* fields depending on the create_date/write_date/create_uid/write_uid
Those fields shouldn't be defined with precompute=True, which triggers
their computation post record creation. This is why, by safety, we
consider the default behavior to be precompute=False.
closesodoo/odoo#80449
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Yannick Tivisse <yti@odoo.com>
A related field copies the attributes from its target field, except for
the attributes defined on the related field itself. The implementation
of this feature was not working properly for attributes with a truthy
default value.
closesodoo/odoo#79025
X-original-commit: dec4a7ec478fa02f19dee8c8426c88d17dc3c7f3
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
After some analyze on a lot of customer databases, seems like most of the time
their are performance probleme, and big store, it is due to a lot of big file
uploaded without reason. E.g. barcode, photo, ... a small one will be enough.
Now, we decided (in stable) to auto resize these pictures to 1920x1920px
by default and compress it with a quality of 80 when the source is bigger.
You can bypass this behaviour in your specific use case,
using a context key: 'image_no_postprocess' set to True.
You can disable the resize (and quality implicitely)
using an icp: 'base.image_autoresize_max_px' set to '0'.
You can change the default resize (1920x1920) format using an icp:
'base.image_autoresize_max_px' set to '<width>x<height>' (e.g. '1024x768')
You can change the default quality (80) using an icp:
'base.image_autoresize_quality' with a value between 0 and 100 where 0 skip it.
You can change the type of file that will be post process using icp:
'base.image_autoresize_extensions' (subtype of the mimetype comma separated).
Api of image has not be changed in this commit, only refactored to allow to
work with image directly without the need to encode/Decode in base64 the raw.
We decide to keep 1920x1920 by default instead of 1080p to avoid to resize
portrait picture in 1080px and stay consistent with field image_1920 that
return a 1920px image for width or height whatever the orientation.
+ fix some lint diff for ci style in master
closesodoo/odoo#78556
X-original-commit: d9ce0507960f247e1187baf7bd8399f90be237aa
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
The processing of the LINK command in a one2many should fail when the
line being linked does not exist. However, there is one case where it
should not fail: when that line existed and was linked before applying
the commands. That use-case may seem strange, but it actually exists:
deleting a line in a sales order automatically deletes the corresponding
reward lines.
closesodoo/odoo#78318
X-original-commit: 4bfe1b5cab10d3171d386d76799a7d7729f49154
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
In some cases we migh want to change to a value that isn't the default
one when uninstalling.
For instance: when we uninstall the module `event_sale`, a product has
the field `detailed_type` set to `event`, which is a subtype of `service`.
So we want to update the value to `service` when uninstalling instead of
the default value, which would be `consu` and wouldn't make any sense.
Part-of: odoo/odoo#77876
This commit removes all the 'extend' initially introduced to avoid code
repetition and ensure visual consistency across Bootstrap and Owl dropdowns.
Despite achieving the desired results, using 'extend' in this context
was seriously impacting the bundle generation time, probably due to an
underestimated amount of Apps' legacy-code applied on these elements.
In order to achieve the same results, the chosen strategy is to add
Bootstrap default classes directly into Owl dropdowns.
Also, it moves code related to bootstrap dropdown in 'webclient.scss',
leaving 'core/dropdown/dropdown.scss' for Owl code only.
Due to the discrepancies between Bootstrap and Owl html
structure, the '.dropdown-item' class could not have been added
directly to Owl's '.o_dropdown_item' itself, without refactoring
the Dropdown component structure.
// ==== Bootstrap 4.6 default Structure ================================
<div class="dropdown-menu">
<button class="dropdown-item" type="button">Action</button>
<a class="dropdown-item" href="#">Another action</a>
</div>
// ==== OWL default Structure before this commit =======================
<ul class="o_dropdown_menu">
<li class="o_dropdown_item">
<span>Action</span>
</li>
<li class="o_dropdown_item">
<a href="#">Another action</a>
</li>
</ul>
// ==== OWL Structure after this commit ================================
<div class="o-dropdown--menu dropdown-menu">
<span class="dropdown-item">Action</span>
<a class="dropdown-item" href="#">Another action</a>
</div>
// ==== web.assets_backend.css Bundle Generation Comparison ============
With all modules installed (enterprise edition over runbot):
Before this commit, bundle took ~2.5s and ~4s to generate and weighted ~322kB (~2.5MB uncompressed)
After this commit, it takes between ~1.2s and ~1.6s and weights ~257kB (~1.6MB uncompressed)
closesodoo/odoo#77649
X-original-commit: 84715436d87bb05b421bc9ccaacda67d07571690
Related: odoo/enterprise#21370
Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
Co-authored-by: Stefano Rigano <sri@odoo.com>
Co-authored-by: François Georis <fge@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Assume that model A has a computed field G that depends on both fields
F1 and F2. Assume that model B inherits from A (_inherits). When G is
computed during an onchange, it must use the form values for all its
dependencies F1 and F2. Before this commit, only the field triggering
the onchange was assigned on the parent record (see (*) below.)
record parent
--- ------------+-------------+------------
initial state | F1=0, F2=0 | F1=0, F2=0
assign F1=1 | F1=1, F2=0 | F1=1, F2=0
assign F2=2 | F1=1, F2=1 | F1=0, F2=1 (*)
The commit also includes another patch: when assigning the parent
record, the ORM determines the dependent records (in this case, the main
record in the onchange). If the delegate field (many2one from B to A)
has an inverse one2many field, the ORM uses that field to determine
dependent records. However, in the case of a new record, that field is
not in cache, and its value is determined to be... empty! The patch
consists in "fixing" the value of that field when we determine the
parent record (when the delegate field is accessed.)
Part-of: odoo/odoo#76042
Co-authored-by: William André <wan@odoo.com>
The license is missing in most enterprise manifest so
the decision was taken to make it explicit in all cases.
When not defined, a warning will be triggered starting from
14.0 when falling back on the default LGPL-3.
closesodoo/odoo#74245
Related: odoo/design-themes#48
Related: odoo/enterprise#19862
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Before this commit, if a related field's path definition contained a
non-existing field (either because of a typo or field removal) the ORM
would simply send a generic Python KeyError that didn't really help in
debugging.
With this commit, we raise our own KeyError stating which related field
has the wrong path definition and exactly which field within the path is
incorrect.
See test for an example.
closesodoo/odoo#31889
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
The use-case is when a "parent" field is put on the form view, as a
readonly field, and we create a new record. Before the fix, the fields
that depend on the parent record are always False, until the record is
actually created.
closesodoo/odoo#73446
X-original-commit: ff2af932c3b73dc59620a4ce17da3a13cab12cbe
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Rémy Voet (ryv)