This fulfills the goal of searching and fetching fields in a single SQL
query. We introduce the new method search_fetch() for that purpose.
Also introduce method fetch() to fetch some fields for a recordset if
they are not in cache yet.
The call graph is as follows:
search() calls search_fetch()
search_read() calls search_fetch() and _read_format()
read() calls fetch() and _read_format()
search_count() calls _search()
search_fetch() calls _search() and _fetch_query()
fetch() calls _search() and _fetch_query()
The methods _search() and _fetch_query() are usually the ones to
override to implement business-specific logic. The method _search()
returns a Query object to retrieve the records that satisfy the given
domain and are accessible for reading. The method _fetch_query() uses a
Query object to retrieve fields from the database and store them in
cache.
Also use search_fetch() to save one query in search_read() and the
reading of one2many fields.
Part-of: odoo/odoo#112126
Goal: make _search() always return a Query object, in order to make
search_read() in a single query when possible
Adapt the overrides of _search() towards the given goal.
Part-of: odoo/odoo#112126
This API is much more sensible for making subqueries. Specifically, one
can generate a subquery without the clauses LIMIT and ORDER BY.
Part-of: odoo/odoo#112126
This simplifies the use of subqueries by avoiding some costly default
order on the model or the idiotic order='id'. Method _flush_search()
has been adapted accordingly.
Part-of: odoo/odoo#112126
The parameter in search() is redundant with method search_count(), and
was making the calls less readable.
The method _search() is aimed at always returning a Query object. The
method can therefore never return an integer, hence the removal of the
parameter. This does not actually remove any functionality from the
method; counting result is simply given by using it differently.
Part-of: odoo/odoo#112126
A set iteration was creating a random number of queries on some tests.
This was noticed in TestEventPerformance were a random additionnal query
could appear in default_get depending on _get_description and
_get_default_stage_id order.
This commit uses a list to get a deterministic order. It looks like
the set was not useful anyway.
closesodoo/odoo#113563
X-original-commit: 13572df256ccc6351588851d385c381ace155e0d
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Steps to reproduce:
- Make sure language preference is 'en_US'
- In accounting, in the dashboard click on bills
- Filter 'due_date' by week
Issue: The start day is Monday and should be, for 'en_US', Sunday as it
is the case in the dashboard view in accounting (see appendix).
Cause: The query uses the `date_trunc('week', date)` which in Postgres
retrieves the first day of the week as Monday (ISO week).
Solution: Create an offset in the query depending on the first day of
the locale variable.
Note: the `web/tests/test_read_progress_bar.py` has been modified: since
the default language is 'en_US' there will be an offset of one day. To
make it less confusing, I used only two anglo-saxons countries so the
day offset is not the variable tested. (for this matter, pleaser refer
to `test_read_group/tests/test_read_group_process_groupby.py`)
Appendix:
Language (english-US)
VIEW (per week) | DASHBOARD
___________________________________________________________________
W23 -> 06/05 | 05/29 -> 06/04
W24 06/06 -> 06/12 | 06/05 -> 06/11
W25 06/13 -> | 06/12 -> 06/18
(Monday - Sunday) (Sunday - Saturday)
Language (french-BE)
VIEW (per week) | DASHBOARD
___________________________________________________________________
W22 -> 06/05 | 05/30 -> 06/05
W23 06/06 -> 06/12 | 06/06 -> 06/12
W24 06/13 -> | 06/13 -> 06/19
(Monday - Sunday) (Monday - Sunday)
opw-2747066
closesodoo/odoo#93053
Related: odoo/enterprise#29539
Signed-off-by: Raphael Collet <rco@odoo.com>
The field is updated in SQL but not in cache. Also remove hacks in the
business code to prevent potential issues from this behavior.
closesodoo/odoo#113119
X-original-commit: 8bb660c985c7a1349cae1bf58dc5bf7981bd9058
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
Consider field F in module X with parameter translate=False, and F is
overridden in module Y with parameter translate=True. During the
upgrade of module X, module Y hasn't been loaded yet, and the ORM
considers the field to be `translate=False`. Therefore it converts its
database column from type jsonb to varchar, and accidentally drops
non-en_US values:
{"en_US": "English value", "fr_FR": "French value"} (jsonb)
-> 'English value' (varchar)
As a result, translations are lost after upgrade.
This commit fixes the bug by checking whether the field is translated in
database and patches the field accordingly when loading the registry.
This avoids the ORM considering the field as non-translated while
upgrading modules.
In order to "force" translated fields to become non-translated ones, at
the end of the loading process the patch above is discarded, and fields
are checked again. We then adapt the schema of models that have such
fields. This extra step handles the uninstallation of modules like
module Y in the example above.
The patching of the fields has one potential issue. While upgrading
module X, field F is patched with translate=True. If module Y actually
overrides F with translate=xml_translate or so, this may cause the
behavior of the upgrade to be slightly incorrect. Because of the
complexity, we have chosen to not support this case.
closesodoo/odoo#112223
X-original-commit: 1e6e482a9763baa5fae70d9b6676d16a46458f4a
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The class is a simple extension of dict, and adds an explicit attribute
for the content of the root node of a tree. It makes the code much more
readable with very small performance overhead.
closesodoo/odoo#111946
X-original-commit: 8a01d74e5f7d035cfd05bc02596f075270e0fcf8
Signed-off-by: Raphael Collet <rco@odoo.com>
Those APIs are aimed at hiding the implementation of trigger trees,
dependent fields and fields modifying relations. Explicit APIs simplify
the profiling of executions and comparison of implementations for
building trigger trees.
X-original-commit: d12b9270375634e738f5288d0c523ac0eec2fa18
Part-of: odoo/odoo#111946
Before this commit, there is no way to "void" model translations, i.e.,
discard a translation and let the value fall back on the 'en_US' value.
The API of methods write() and update_field_translations() can only
overwrite the translations for the specified languages.
After this commit, calling update_field_translations() with a falsy
value except the empty string discards the corresponding translation
value, and let the value of the field in the given language fall back on
the 'en_US' value of the field.
X-original-commit: 5434fb845c393327db377abf872c448f4860a7d3
Part-of: odoo/odoo#111869
model/model_terms translations:
A transifex link if available is displayed after the language in the translation
dialog to help translators to contribute their translations. The tooltip message
for mouse hovering is 'Contribute'
code translations:
Setting -> Translations -> Application Terms -> Transifex Code Translations
In order to reuse the list view and search bar, model transifex.code.translation
is created to store all code translations.
This model is
1. readonly
2. reloaded on demand/by cron(7 days) to avoid increasing the Odoo restart time
3. updated for new installed modules/languages while opening the list view
4. shared to all users without duplicated translations
closesodoo/odoo#111685
X-original-commit: 7b30badb52e2a948c41f4340a01f6118818c6711
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
There is only one legitimate usage of `_patch_method` (base_automation)
and none of `_revert_method`. The other uses are in tests and they are
all wrong: if the test crashes between the `_patch_method` and the
`_revert_method`, the method will not be reverted.
For the only proper usage of `_patch_method`, move the code to this
place. Correct tests using `_patch_method`/`_revert_method` by calling
the `patch` method of `BaseCase`.
Also, remove the `api.returns` from the `create` method of `BaseModel`
as it is useless and confusing. In fact, we never use it because we
have a special treatment at the RPC level for the `create` method
(see `_call_kw_model_create`).
closesodoo/odoo#110370
Signed-off-by: Raphael Collet <rco@odoo.com>
Warn when constrains name combine with table name will be more than
63 characters.
This is to avoid case like the one fixed in #103148
Unlike index, since constrains name are defined in the code, we prefer
to avoid automatic truncate and add a warning since devs can chose
an appropriate short-enough name.
The linked fixes will just truncate the name to the max length
to match the name in existing databases. Renaming could be done in other
pull requests with upgrade scripts to avoid constrains re-computation.
Part-of: odoo/odoo#109065
The revision 65a012c2bba0ed81d9b525a7779c726043b17707 changed the
behavior of method _read() when invoked on a record with id None.
Before the change, the method would consider that no records have been
fetched because the id is falsy. After the change, it keeps the id None
in the fetched records, and attempts to make subsequent SQL queries like
"column IN ()", which is syntactically incorrect.
OPW 3104957
closesodoo/odoo#108785
X-original-commit: 712d03cb2ee5f2fde0e80619be5db38aa09ed1ac
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This is a rare case where method _read() raises a MissingError instead
of just ignoring it.
The issue is triggered by several conditions on a model M:
- at least one ir.rule on M with a domain using a column field on M;
- one deleted record Y which is in the prefetch set of a record X;
- one reads a non-column field on record X.
Fix method _read() to manage that case. It adds an extra call to
exists() in that case, but adds no overhead in the general case.
closesodoo/odoo#108050
X-original-commit: 1876dc87e5c88c711c9b3882c39be704ffc46532
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Rémy Voet <ryv@odoo.com>
The model terms translations assume the number of terms in each language of a
model_term translated field should be the same.
However, sometimes the assumption cannot be promised for sake of bad
translations
This commit
1. drops illegal model term translations while translating
2. drops mismatched terms at run-time in case the database has been contaminated
closesodoo/odoo#107373
X-original-commit: f9ab5ca3e99b2883ada18a8f6deb12d45608b43f
Signed-off-by: Raphael Collet <rco@odoo.com>
The main goal of this commit is to reduce the size of the registry by
removing the (almost) useless __last_update field.
Statistics # of fields with all modules installed:
before 30184 fields, 1299x last_update (4.30%)
Before this commit, the computed field __last_update was added on every model.
The idea behind this field was to have a computed field that had either
the write_date or the create_date if the write_date was empty. However,
the write_date is always written, even on creation, making it useless
to have the computed field __last_update
After this update, we completely remove from BaseModel:
* __last_update
* CONCURRENCY_CHECK_FIELD that was always defined as "__last_update"
* _compute_concurrency_field that was the compute function for __last_update
closesodoo/odoo#105739
Task-id: 3062140 (part of 3062137 improve registry load time)
Related: odoo/upgrade#4038
Related: odoo/enterprise#33939
Signed-off-by: Raphael Collet <rco@odoo.com>
Sometimes a constraint is too complex to be expressed in the general
`_sql_contraint`.
This allows to hook on a constraint if it matches the name in order to
display a user friendly error message.
X-original-commit: e1f06479a526c703ccabc441b1e194646206b966
Part-of: odoo/odoo#106325
The mapped call is supposed to populate the cache, but that's already
taking care of by prefetching. It ends up adding an overhead instead.
The perf improvement is more noticeable on large recordsets.
For example, on a database populated with 100k res.partner records.
Before:
.. code-block:: python
partners = env["res.partner"].search([])
partners.filtered("name") # warm up
timeit.timeit(lambda: partners.filtered("name"), number=10)
# result: 7.15
After:
.. code-block:: python
partners = env["res.partner"].search([])
partners.filtered("name") # warm up
timeit.timeit(lambda: partners.filtered("name"), number=10)
# result: 4.67
closesodoo/odoo#105350
Signed-off-by: Rémy Voet <ryv@odoo.com>
O2MIdMapper Class inherit of 'base' Model to change the behavior of
`create`.
Because it is in base and the `_import_current_module` is set in
models.py, move it at the end of create in models.py.
closesodoo/odoo#99550
Related: odoo/enterprise#33063
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
In the PR #97692, writing empty str `''` to a model term field no longer removes
all translations. Instead, only the translation for the current language is set
to empty str `''`.
Before this fix, if a web user sets a translation to emtpy str, and reopens
the translation dialog, the translation will be reverted to the 'en_US' value
which is wrong.
closesodoo/odoo#104181
X-original-commit: f97ff6308d472121e65a34d30b4dc5ab35065220
Signed-off-by: Samuel Degueldre <sad@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
When looking to fetch grouped data from the database, `read_group`
should work(-ish), especially with support for `array_agg`. This also
means this need should be more or less solved around RPC: client can
either `read_group` or group on the client side.
This leaves a glaring hole in the fabric: there's no convenient tools
to apply grouping to recordsets. `itertools.groupby` exists, but it
requires that the input be grouped by the key function, and it yields
an iterable of items which is not the most convenient for recordsets.
This here utility:
- is exclusive to recordsets
- returns a mapping of grouping keys to subsets of the input recordset
- keeps the prefetching of the source recordset
- allows grouping on a field, or an arbitrary (callable) key
- should work even on "new records" (aka should be suitable for
onchanges / arbitrary compute functions)
While the method is RPC-compatible, it's not especially designed for
that, it is likely a much better idea for the client to `read` the
data they need then group that client side on whatever criteria they
are interested in, or `read_group` the aggregated information they
need directly if that's an option.
Part-of: odoo/odoo#101522
* clean and improve docstrings in orm
* fix typos found with codespell
* rely on the Environment class docstring instead of doc content (and
therefore move part of the doc inside the class docstring)
closesodoo/odoo#102969
X-original-commit: 8250cd4b210005d223a4cdb8afa4014425ca6fa3
Related: odoo/documentation#2803
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
Before this commit the neutralize system introduced in v16 was using ORM
methods in order to change appropriate records. Although flexible, this approach
could lead to call some methods with side effects while neutralizing
(eg: overloads of write).
This patch converts the neutralize system to a safer "inert" SQL based approach
by migrating the generic method _neutralize to SQL files exposed in the
data folder.
Task id: 2961687closesodoo/odoo#102792
X-original-commit: e5dbded9bb363351feff7ca8a56c7f8a6860f492
Related: odoo/enterprise#32580
Signed-off-by: Fabien Meghazi <fme@odoo.com>
Purpose
=======
Check the existence of the relational properties (many2one / many2many)
in batch and prefetch the values in batch as well to reduce the number
of SQL queries.
Technical
=========
The existence is checked in the read method of the properties field,
because we have the entire recordset. Then, the non-existing ids are
remove from the properties values and the cache is updated.
Task-2965523
X-original-commit: dba9b684d29c0041a32a5508284573851b8dd097
Part-of: odoo/odoo#102243
By definition, views are useless without the web client.
The web client is in the module `web`.
Therefore, in a perfect world, the `get_views` model method
and any related method should be within the `web` module,
as without it they are useless.
Let's imagine you would like to use fully in command-line,
without web client, all those view related methods are useless.
Maybe excepted for reports, as you might still like to be
able to print reports while using odoo fully in command line.
Views related method therefore shouldn't be in `odoo/models.py`.
Also, when you think about it, `get_views` related methods do not
make sense without the model `ir.ui.view`, which is loaded after
the `get_views` related methods, which also doesn't make sense.
However, moving these methods fully in the `web` module is an harder
work. For instance, there are base models, such as res.partner,
already overriding `get_view` in the `base` module,
and therefore relying on these view related methods.
As a first step, we move view related methods direcly in the
`odoo/addons/base/models/ir_ui_view.py` file, where
the `ir.ui.view` model is loaded.
This is not only a design / cleaning change,
but a required change to be able to use content of
`odoo.tools.config`, which is loaded after `odoo/models.py`.
For instance, if you want to configure a conditional decorator based on
the config `odoo.tools.config['dev_mode']`, it is not possible
to do so in `odoo/models.py` because the config is parsed/loaded
after `odoo/models.py`.
The config is loaded here:
https://github.com/odoo/odoo/blob/31de2b0a7a0921cab3c6c54045d15da46c8e6d8a/odoo/cli/server.py#L127
While, within the same file, `odoo/models.py` gets loaded through the
`import odoo`
https://github.com/odoo/odoo/blob/31de2b0a7a0921cab3c6c54045d15da46c8e6d8a/odoo/cli/server.py#L26
And we would like to put such a decorator based on `odoo.tools.config['dev_mode']`
on `_get_view_cache`, to not cache the back-end views when `--dev xml` is
passed in the server arguments.
X-original-commit: 0901adc38a724aec75676285977f1905a84ed8ee
Part-of: odoo/odoo#102117
Bug
===
- create a new helpdesk ticket without properties
- create a new property
- change the customer
=> The added property is removed
The fix in the onchange of models.py needs to be more specific, and
should be used only when the definition record is changed.
Task-2965523
X-original-commit: 909ee004183068d9ffa94abe0d1472a8f0534f34
Part-of: odoo/odoo#101487
The cache currectly fails to correctly invalidate relational fields that depend
on a non-relational field. Two passes of invalidation are done, to reflect
dependencies on both the old and the new written values. In the first pass
only relational fields are considered, as explained in the comments:
> It is best explained with a simple example: consider two sales orders SO1 and
SO2. The computed total amount on sales orders indirectly depends on the
many2one field 'order_id' linking lines to their sales order. Now consider the
following code:
>
> line = so1.line_ids[0] # pick a line from SO1
> line.order_id = so2 # move the line to SO2
>
> In this situation, the total amount must be recomputed on *both* sales order:
the line's order before the modification, and the line's order after the
modification.
The written values can be seen as the roots of a dependency forest (a
collection of dependency trees). Before this commit all non-relational roots
and their corresponding trees were filtered out during the first pass. However,
this approach is wrong, as relational fields can also depend on non-relational
fields. Instead, the complete dependency forest has to be traversed, skipping
invalidation for non-relational fields during the first pass.
The test that was previously included accidentally succeeded because of a
separate and unrelated bug in the orm domain parser: in certain one2many or
many2many leafs the domain parser would not take into consideration the domain
included in the definition of the field. As a result, the test still passed
by accident, because the records that no longer matched the domain after the
write were still invalidated during the second pass.
The problem can clearly be demonstrated, however, when the dependency is
generated by a compute function.
closesodoo/odoo#101038
X-original-commit: d4a5827b42d80f0f830455dcd2056701eb09aed1
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
- In `unlink`, since https://github.com/odoo/odoo/pull/66938
modified is called on self for each batch of 1_000.
But it should be called on the batched records.
- In `write`, remove useless `records_to_inverse`
(there from ORM refactor but never used)
- make `_modified_triggers` more deterministic by
changing a `set` into `OrderedSet`.
closesodoo/odoo#100472
Signed-off-by: Raphael Collet <rco@odoo.com>
A feature allows to set the modifiers invisible, readonly, required
according to a key in the context
e.g.
```xml
<field name="date_approve" invisible="context.get('quotation_only', False)" optional="show"/>
```
Following odoo/odoo#99417,
back-end views are now cached.
These expressions are currently evaluated server-side,
before serving the view to the web client.
Hence, the modifier becomes for instance `invisible="1"` or `invisible="0"`
according to the context passed when calling `get_view`.
The key used to cache the views doesn't take into account such context keys.
Hence, when you asked for a view using for instance
`invisible="context.get('quotation_only')"`
A first time using the context `{'quotation_only': True}`
and a second time using the context `{'quotation_only': False}`,
on the second time, you received the view from the first time you
requested the view, where the modifier is evaluated as if the the
context was `{'quotation_only': True}`.
As we do not want to store a cached version of the view for each
possible key in the context, postprocess the evaluation of the
modifiers using the context after retrieving the view from the cache.
A better alternative would be to delegate this evaluation to the
web client, because it already has the information it needs,
it has the context value.
Modifiers using domains are already evaluated client-side,
it would make sense modifiers using context would too.
Nevertheless, as this bug has been introduced by odoo/odoo#99417,
and as we are close to the release, solve this server-side,
to keep a similar behavior than before.
We might re-consider the implementation of this later on.
closesodoo/odoo#100130
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Oversight in odoo/odoo#100376
The web client sends `list` rather than `tree` when requesting
views.
However, once they went through `get_views`, they are converted back
to `tree`.
https://github.com/odoo/odoo/blob/master/odoo/models.py#L1653
So, here, we should check against `tree`, not `list`.
Could be seen in Lunch > Manager > Today's Order.
With the `id` field missing, an error is raised
```
Uncaught (in promise) Error: The following error occurred in onWillStart: "field is undefined"
```
closesodoo/odoo#100474
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Following odoo/odoo@4636620004
`get_views` only pass the model fields included in the view architecture,
except for the main model when the search views is requested.
Because the search views requires all fields for the user to be able
to make advanced filters and advanced group by using any fields of the
model.
However, other views requires more field descriptions as well than
just the fields included in their architecture:
- the graph view requires all integer and float fields,
to automatically add suggestions of measures in the measures dropdown
menu. It's a bit like the search view, the user should be able
to choose any measure available in the model
(as long as this is integer or float fields)
- the pivot view requires all groupable fields,
so the user can group by any groupable fields of the model.
The JS MockServer `getViews` is adapted to include the changes added by the above
revision as well as the current revision,
for the qunit tests suite to be able to reflect these API changes
from the server side.
closesodoo/odoo#100376
Related: odoo/enterprise#31427
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The main goal is to reduce the number of KB sent by the `get_views`
method to the web client by getting rid of things not used by the web
client.
This revision aims to only pass the `fields_get` info of fields
actually required by the web client.
Before this revision, for each view, the list of all fields
of all models implied in the view are passed.
For the main view, it's required to pass all the fields,
for the search view: The advanced search needs to know all fields
of the model to be able to let the user to do advanced filters and
groupbys on the model fields.
But, for the models of the subviews (one2many/many2many embbeded views),
this is not required. Only the fields included in the view are required.
Also, if the search view is not requested by the web client,
then it's also not required to pass all the fields of the main model.
After this revision, for each view, pass the `fields_get`
info of only the fields actually implied in the view.
For the search view, info of all fields are passed,
not only the ones implied in the view.
For instance, on the `get_views` of `account.move`,
this allows a gain of an extra 18,05KB,
reducing from 165.82 to 147.77KB.
In addition, before this revision, the whole content of `fields_get`
was stored in the cache of `_get_view_cache`, meaning per view.
If 3 views were requested on `get_views`
let's say kanban, list, form,
the same whole `fields_get` info, of all fields,
was stored 3 times in the cache.
This revision removes the content of fields_get of the cache
for `get_views`, to avoid storing multiple times the exact same
information in the cache, therefore gaining tremendous memory,
while not loosing that processing time.
The decrease of speed is only 1~3ms per call to `get_views`.
This is mainly because all the information gathered by `fields_get`
is already in the cache. For instance, it doesn't require any SQL
request (at the moment). e.g. the translations (string, help, selection)
are already cached.
closesodoo/odoo#99834
Related: odoo/enterprise#31167
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Instead of fetching all field attributes sent with the field list
to the web client,
restrict the attributes to the ones actually required by the web client
This allows, for instance,
to gain 44,75KB on each call on `get_views` for `account.move`,
from 208.78KB to 164.03KB,
with only `account_accountant` installed.
closesodoo/odoo#99660
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
The master plan finally comes to an end.
Thanks to:
- odoo/odoo#87522 refactoring `load_views`,
- odoo/odoo#94337 refactoring `common.Form` to prevent changing
invisible fields in unit tests using `Form` instances,
- odoo/odoo#95729 refactoring the behavior of `groups=` in views,
- odoo/odoo#98551 removing the need of the `groups_id` many2many field
on back-end views.
The result returned by `get_view`/`get_views` can now finally be easily
and efficiently cached, in order to cache back-end views.
The goal of this revision is to cache the model views and fields
already post-processed for the web client
(with the modifiers, etc., already computed)
without group restriction.
Then, from this cached version, post-process group related features,
such as removing nodes restricted with a `groups=` attribute,
set the create/write button according to the user access rights to models, ...
Not including the groups in the cache key allows:
- to have less cached versions,
(otherwise it would be one cached version per different group combination)
- to not have to fetch the user groups to compute the key
(with the current cache key,
there is nothing to fetch from the database to compute the key)
Besides, post-processing the groups features
after taking the view from the cache of the view doesn't take a tremendous time:
- parsing arch from/to string with etree is fast,
- removing the `groups=` nodes using etree is fast,
- adding the `create="False"`, `write="False"`, `delete="False"` on the view
root node according to the access right of the user on the model is fast.
This allows way faster calls to `get_views` by the web client,
as the server no longer need, for each call, to fetch the views in database,
combine the inherited views, post-process the modifiers attributes, etc.
Timing tests are available on the pull request of this revision.
Part-of: odoo/odoo#99417
The cache key of _get_bindings was not super efficient.
The result of _get_bindings is cached,
but its performance was altered by the cache
key which requires to fetch the user groups for each call to
_get_bindings.
Besides, as there is a lot of possible group
combination, this resulted in a lot of possible cache keys,
and therefore a lot of cached values.
This revision aims to make _get_bindings more efficient
by:
- do not use the groups in the cache keys (less cached values)
- filter out actions not available to the user groups after
retrieving them from the cache
- use has_group to do the above, which is itself cached as well,
and therefore do not need to fetch the user groups
at each call to get_bindings.
In addition, move get_bindings from `get_view`
to `get_views`. If there was 3 views asked by `get_views`
(let's say kanban, list, form)
`get_bindings` was being called 3 times, through `get_view`
with each time the same arguments and therefore the same result :-).
Moving it to `get_views` allows to call it only once for all view types
requested, and for the web client it doesn't change much,
as it always request the toolbar/get_bindings through `get_views` only.
In addition, add the lang to the cache of _get_bindings.
it was actually a bug not to put it: if you had 2 users
with the same group set, using 2 different languages,
the user accessing first the get_bindings would cache
the action names within his language, and then the second
user would see the action name within the language of the first user
:-).
Before
```py
In [1]: %time for i in range(1000): self.env['ir.actions.actions'].get_bindings('res.partner'); self.env.invalidate_all();
CPU times: user 790 ms, sys: 104 ms, total: 893 ms
Wall time: 1.7 s
```
After
```py
In [1]: %time for i in range(1000): self.env['ir.actions.actions'].get_bindings('res.partner'); self.env.invalidate_all();
CPU times: user 23.5 ms, sys: 9.12 ms, total: 32.7 ms
Wall time: 36.9 ms
```
Part-of: odoo/odoo#99417
For non-stored computed fields, `_modified_triggers` will traverse the
tree (at the cost of extra queries) only to know which record to
invalidate in cache. But in most cases, these fields have no data in
cache, so they can be ignored from the start, which allows us to prune
entire subtrees from the merged tree.
By example:
With simple write on `show_operations` of one `stock.picking.type`, the
`_modified_triggers` will fetch every ids (from database) of `stock.picking`
and `stock.move` related to this `stock.picking.type`
(For `stock.move`, it is because of the
`show_operations = fields.Boolean(related='picking_id.picking_type_id.show_operations'`))
In that case, there isn't any data of `show_operations` (`stock.move`)
in cache, then there are nothing to invalidate and
the `_modified_triggers` cost
is high (extra queries/processing) for nothing.
Then we cut parts of the tree when we know that
they won't invalidate anything (=> if the cache is empty for the field).
Also refactor the way to merge trees to be more efficient.
Performance improvements:
- For all tests at-install done by the runbot, we gain -+ 3.5% of
queries.
- In the example above, we reduce the number of queries (potentially
bottleneck queries in large DB) from 7 to 4.
- In term of CPU (without counting time in SQL), the new version is -+
20 % faster (on install of stock,purchase,mrp and with
--test-tags=/stock,/purchase).
closesodoo/odoo#76322
task-2780812
closesodoo/odoo#99274
Related: odoo/enterprise#30951
Signed-off-by: Raphael Collet <rco@odoo.com>
`_check_concurrency` was used to check if 2 people were modifying the
same record at the same time. It was doing so by setting a special
`__last_update` value inside the context that was later evaluted to
prevent some concurrency issues.
It was mostly unused, wasted a lot of cpu cycles and was not covering
all cases (e.g. pending write).
closesodoo/odoo#87756
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Purpose
=======
Make the onchange work for the properties fields. When changing the
container field, we need to update the properties definition.
Task-2852259
Part-of: odoo/odoo#95184
Purpose
=======
Add a new field "Properties" to be able to light customization of workflows
based on a parent model. Those properties acts in some ways like Odoo fields
without requiring specific columns e.g. add new properties on tasks of a
specific project.
Usage
=====
Define properties on a parent model (e.g. project) with
```
attributes_definition = fields.PropertiesDefinition('Message Properties')
```
It defines properties available on children: types, default, value, model for
relational properties,...
Use it on children records (e.g. task) with
```
attributes = fields.Properties(
string='Properties',
definition='parent_id.attributes_definition',
)
```
Technical
=========
Parent | Properties definition
------------------------------
The properties definition is stored on the parent, on a JSON field.
This definition contains the type of the properties, the default value,
the model of the many2one,...
```
[
{
'name': 'name',
'string': 'Name',
'type': 'char',
'default': 'Default Name',
}, {
'name': 'partner_id',
'string': 'Partner',
'type': 'many2one',
'comodel': 'res.partner',
},
]
```
Child | Properties values
-------------------------
The value is stored on the child, using a Properties field.
```
{
'name': 'Mitchel',
'partner_id': 1337,
}
```
When we read this field, we will automatically read the definition on
the parent, and merge both JSON into one, so the web client has the
value of each property, and their definition.
```
[
{
'name': 'name',
'string': 'Name',
'type': 'char',
'default': 'Default Name',
'value': 'Mitchel',
}, {
'name': 'partner_id',
'string': 'Partner',
'type': 'many2one',
'comodel': 'res.partner',
'value': 1337,
},
]
```
Integrity
---------
If we remove a property on the parent, we won't update the child value.
Instead, when we read the child properties, we will filter them based
on the parent. So the removed properties will be removed the next time
we write on the field.
In the same logic, the many2one existence is checked when we read the
field. There's no foreign key between the integer stored in the JSON
in the SQL row corresponding to the record in database.
Write
-----
We can write on the Properties field with a list of field definition
+ value.
Some types are not JSONifiable (like the date, datetime), they are
stored as string in database and parsed when we read the value.
In order to update the parent definition by writing on the child,
you need to add the dict key `definition_changed` or
`definition_deleted`. This is because we need to be able to know
if the definition has been changed without doing extra SQL queries.
Access rights
-------------
A user can add a many2one / many2many property to a model only if he
has the access rights to it.
Many2one / Many2many
--------------------
The model choice of a many2one / many2many properties was subject to
changes.
First implementation stored models in both parent and children to easily
spot changes and avoid complex queries when fetching records, trying to
synchronize them, ...
As this leads to storing a lot of duplicated content we choose to instead
reset the value on the child if the model has been change. We generate a
new name for the property. So it behaves like if we removed the property
and created a new one.
To be able to restore the old value (e.g. if by mistake we changed the
model, and go back to the old model), we store the initial states.
Task-2852259
Part-of: odoo/odoo#95184
The sequence field is special-cased early on in read_group (_raw): if
the caller requests the aggregation of ``sequence`` that request is
ignored:
if fspec == 'sequence':
continue
This was added a long time ago, probably due to the special-ish status
of `sequence` (summing sequence number doesn't really make much
sense).
The issue is that it's also possible to request ordering by
`sequence`, which requires `sequence` to be one of the aggregated
fields. This is checked in `_read_group_prepare` and triggers a
warning.
This leads to an inconsistent behavior, where the user requests
aggregating & ordering by `sequence`, we remove it from the aggregated
fields, then warn that they didn't aggregate on the field.
Make the behavior consistent by also ignoring requests to order by
`sequence`.
closesodoo/odoo#97409
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
In case several xml_ids target the same record, now we show an icon with
other xml_ids (not the first one) as tooltip.
task-2954293
closesodoo/odoo#98139
Signed-off-by: Jérémy Kersten <jke@odoo.com>
Consider two models A and B, where
- model A has a many2one_reference field 'res_id' with model field 'res_model';
- model B has an auto-join one2many field 'stuff_ids' to A using field 'res_id';
- the field 'res_model' is not flushed on some record.
model | A | B
-----------+-------------------+-------------------
memory | res_model = B |
-----------+-------------------+-------------------
database | res_model = NULL | id = 42
| res_id = 42 |
| foo = 'bar' |
Now, perform a search on model B that should return record with id=42 by
matching some condition on the unflushed record in model A, like:
B.search([('stuff_ids.foo', '=', 'bar')])
Before this patch, the search method would not flush the field
'res_model', which causes the method to return incorrect results. This
patch fixes the issue by ensuring that searches on one2many fields flush
all the fields on which the one2many field depends.
The issue was discovered while working on task 2735672.
closesodoo/odoo#96115
Signed-off-by: Raphael Collet <rco@odoo.com>
Raphael Collet