[IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
`NULL`
`{"en_US": "Foo"}`
`{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}`
`{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}`
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
` NULL`
`{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}`
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
`SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...`
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
closesodoo/odoo#97692
Task-id: 2081307
Signed-off-by: Raphael Collet <rco@odoo.com>
The current asset_bundle version uses the last modified date.
This can be problematic in some case.
Even if it is a long time issue, the problem was rediscovered on runbot
with a commit being in the future. Runbot will export all file and set
the write date of the file to the commit date. The main purpose is to
have deterministic bundle version between different builds. This also
allows to generate assets bundle once at install for all post install
subbuild.
The issue here is that the commit date was greater than now(), meaning
that some tests setting custom css in attachment won't trigger the
regeneration of assets bundle. This wasn't really noticeable before
assets pregeneration.
This is not the first time strange issues occurs because of the
last modified logic.
This commit combined all last_modified to generate the bundle
version.
The current adaptation is quick and dirty and this will be reworked in
another post 16.0 freeze assets refactoring and cleanup.
closesodoo/odoo#100160
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Website: the context install_filename='dummy' is used to prevent
arch_updated from becoming True while updating translations of
ir_ui_view.arch_db (if arch_update becomes True, test_inherit_specific
fails)
Fuzzy search for jsonb translated fields has been adapted in the case of
website. It may require some refactoring later.
Previously, 'Edit Translations' button is used to translate
the English field ir_ui_view.arch
A dialog for list view of all translations is opened
After this commit, a new 'EN' button is used to translate
the Engligsh field ir_ui_view.arch
A translation dialog (like other translatable fields) for all translations is opened
General idea of the implementation:
1. add the invisible ir_ui_view.arch_db field to the form view before ir_ui_view.arch
2. show the 'Lang' button in the edit mode
and make it look like a button for its next field ir_ui_view.arch
3. Forcely change the 'Lang' button to 'EN'
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The purpose of this PR is to inline the "Validate" button
with the `country_id` field on the partner form view
when using Avatax.
task-2947705
closesodoo/odoo#100093
Related: odoo/enterprise#31277
Signed-off-by: Olivier Colson (oco) <oco@odoo.com>
This commit changes the node modifiers domain evaluation in the Form test class to allow strings.
Domains as strings are used by the web client to evaluate special values such as 'uid'.
Part-of: odoo/odoo#99735
XML files are now declared in python module manifests. During the qweb
't-call-asset' directive, assetbundle will fetch the declared xml files,
apply the inheritance (t-inherit) and create a javascript service (for
eg: 'web.assets_backend.bundle.xml') which is added at the end of the
*.js mimifier file.
When the debug mode is activated, comments are added in the template
indicating which file the template comes from as well as the
inheritances applied to it.
****
JavaScript:
assets.js (module @web/core/assets) takes care of loading libraries,
javascripts and styles.
`loadJS(url)` (loads the javascript and returns a resolved promise when
the templates are also loaded via the '*.bundle.xml' service)
`loadCSS(url)` (loads the style a resolved promise when the file is
loaded)
`loadXML(xml, app=assets.defaultApp)` (load template into
application/owl, used by the `*.bundle.xml` services)
`getBundle(bundleName)` (get the bundle descriptor)
`loadBundle(desc)` (load the files and bundle from a descriptor)
templates (XML element content all owl templates)
A new `ready(serviceName)` method on boot.js lets you know when a
service is loaded are the require.
The xmlDependencies attribute no longer exists.
Python:
The xmls taken into account by assetbundle.py, applying `t-inherit`
inheritances and adding an `name_of_the_bundle.bundle.xml` service in
the generated JavaScript file.
****
Every manifest changes is into the next commit, except 'web_tour' in
this current commit as example.
Part-of: odoo/odoo#95500
When a file is immutable the different network layers can cache it.
Services receiving this header should never invalidate these files.
Part-of: odoo/odoo#95500
Also get rid of {'column_invisible': False} in modifiers
e.g. in the account move list
Before:
```xml
<field name="invoice_partner_display_name" string="Customer" modifiers="{"readonly": true, "column_invisible": false}"/>
```
After:
```xml
<field name="invoice_partner_display_name" string="Customer" modifiers="{"readonly": true}"/>
```
closesodoo/odoo#100084
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
The pregenerate of assets_bundle should be done maximum two times
- at the end of an install/update if the module test_assetsbundle is
installed
- at the beginning of post_install tests
Currently, during the nightly (install and all tests are ran at the
same time), the pregeration of assets is triggered at the end of some
tests
File "/home/xdo/osrc/master/odoo/odoo/tests/common.py", line 319, in _tearDownPreviousClass
super()._tearDownPreviousClass(test, result)
File "/usr/lib/python3.8/unittest/suite.py", line 300, in _tearDownPreviousClass
previousClass.doClassCleanups()
File "/usr/lib/python3.8/unittest/case.py", line 731, in doClassCleanups
function(*args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 704, in reset_changes
self.setup_models(cr)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 306, in setup_models
model._register_hook()
File "/home/xdo/osrc/master/odoo/odoo/addons/test_assetsbundle/models/ir_qweb.py", line 13, in _register_hook
The registry.updated_modules is set to allow post_install tests
to select the right tests to execute. This is a way to define if we
updated some modules loading this registry. This list is unfortunatelly
not emptied before running the test, meaning that if register hook are
called on the same registry again, pregenerate will be executed
a second time.
A solution here is to check if the registry is not ready,
meaning that this is the load_modules call to register_hook
and not the setup_models one
closesodoo/odoo#100079
Related: odoo/enterprise#31271
Signed-off-by: Raphael Collet <rco@odoo.com>
The goal is to easily re-use these 3 common methods
for unit tests in module implementing new type of views,
such as `web_cohort`
closesodoo/odoo#100021
Related: odoo/enterprise#31239
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Visible_menu check per_read model per model leading to a lot of queries
We can get all readable fields with one query
Computation time goes from ~400 total to less than ~50 total
The different would be even bigger on a server with a distant database.
(higher ping)
Part-of: odoo/odoo#99176
During tests on runbot, the main reason tours are slow to start is
because the first loading of "/web" need to generate assets bundles.
Generation can take up to 5 seconds time the number of tour.
Before this, the first requests to /web takes around 7 seconds on
runbot and the next ones less than 1 second.
After this commit, the first request to /web takes around 1.5 seconds
Note that the main difficulty is to choose when to generate the assets.
(With optimisations from following commits) the generation time is
arround 30 seconds (21 css + 12 js) the first time, for all modules.
If the attachements exists the generation time is arround 3 seconds
(2.5 js + 0.5 css) mainly because of globs to find usefull files.
In practice for runbot the ideal would be to generate them at
the end of the install so that it is shared for all post_install builds.
Doint it at the end of an install is not wanted in all cases, saas
pregenerated templates and upgrade may avoid doing that.
Doing it in a special runbot step (subcommand) is not possible for niglty
execution (no split, in one go). This needs to be in the code.
It is also not practical for devs wanting to have the same behaviour on
a local machine.
A solution to make the test conditionnal at install is to base the
condion on an existing test_module. test_assetsbundle is a good
candidate here. On runbot, the at_install step (befor split) will
automatically generate assets with this solution.
Assets are also generated before post_install tests. This should be fast
if they already exists and ensure that assets are up to date after
modifying sources locally or after downloading a database on runbot.
It should be fast enough for small database and may be even faster
in the future for small diffs.
Part-of: odoo/odoo#99176
Right now the qunit will log all results at the end.
This means that the runbot may wait for all qunit before detecting the
failure.
This also mean that all failure are in one ir.logging on runbot, making
the automated parsing difficult if multiple modules fails during the
same build.
We could also log all failure immediately, but grouping them my qunit
module will avoid duplicating logs for linked causes (one failure
leading to a `Expected %s assertions, but %s were run` message)
closesodoo/odoo#99912
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Class ResPartnerBank modified to:
- Add field 'allow_out_payment', true iff account can be recipient
of out payments
(False by default for new accounts except when created from res.partner)
- Enable chatter tracking for base fields
- Remove barely used/useful view view_company_partner_bank_form
closesodoo/odoo#94801
See: odoo/enterprise#29916
Task: 2856231
Related: odoo/upgrade#3637
Signed-off-by: Olivier Colson (oco) <oco@odoo.com>
Emails got always the odoo logo even if the company logo has been changed. This
solves the problem.
Technical note: the problem was caused by an exception in the controller due to
invalid parameters used for send_file method causing web/static/img/nologo.png
to be returned.
Task-2920690
closesodoo/odoo#99073
Signed-off-by: Julien Castiaux <juc@odoo.com>
Changing the name of model payment.acquirer to payment.provider
and everything that it touches. It is technically incorrect to
use the term "acquirer" for systems that only provide a service
of payment.
After this commit the model payment.acquirer and all related to
it will be renamed to payment.provider.
Task - 2842088
closesodoo/odoo#90899
Related: odoo/upgrade#3542
Related: odoo/documentation#1981
Related: odoo/enterprise#27131
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
The main goal is to reduce the number of KB sent by the `get_views`
method to the web client by getting rid of things not used by the web
client.
This revision aims to only pass the `fields_get` info of fields
actually required by the web client.
Before this revision, for each view, the list of all fields
of all models implied in the view are passed.
For the main view, it's required to pass all the fields,
for the search view: The advanced search needs to know all fields
of the model to be able to let the user to do advanced filters and
groupbys on the model fields.
But, for the models of the subviews (one2many/many2many embbeded views),
this is not required. Only the fields included in the view are required.
Also, if the search view is not requested by the web client,
then it's also not required to pass all the fields of the main model.
After this revision, for each view, pass the `fields_get`
info of only the fields actually implied in the view.
For the search view, info of all fields are passed,
not only the ones implied in the view.
For instance, on the `get_views` of `account.move`,
this allows a gain of an extra 18,05KB,
reducing from 165.82 to 147.77KB.
In addition, before this revision, the whole content of `fields_get`
was stored in the cache of `_get_view_cache`, meaning per view.
If 3 views were requested on `get_views`
let's say kanban, list, form,
the same whole `fields_get` info, of all fields,
was stored 3 times in the cache.
This revision removes the content of fields_get of the cache
for `get_views`, to avoid storing multiple times the exact same
information in the cache, therefore gaining tremendous memory,
while not loosing that processing time.
The decrease of speed is only 1~3ms per call to `get_views`.
This is mainly because all the information gathered by `fields_get`
is already in the cache. For instance, it doesn't require any SQL
request (at the moment). e.g. the translations (string, help, selection)
are already cached.
closesodoo/odoo#99834
Related: odoo/enterprise#31167
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
- Remove `_default_log_exceptions` of our `Cursor` class (unused except
in one test)
- Deprecated `serialized` args of `__init__` (Cursor class) and
`cursor()` (of Connection class), our cursor is always serialized.
- Remove `sql_log` attribute of Cursor class and replace it with
appropriate code to do the same stuff dynamically.
- Simplify some code
- Update some docstring
- Clean import
task-2766494
closesodoo/odoo#85078
Signed-off-by: Rémy Voet <ryv@odoo.com>
- Remove depreciated `after` method of `Cursor`
- Remove `unbuffer`/`flush_env`/`clear_env` methods
- Remove specific code for psycopg version < 2.7 (psycopg version is
always >= 2.7 in our requirements)
- Depreciated autocommit
task-2766494
Part-of: odoo/odoo#85078
Since e014cc88394fb9c8f6cff556ca73d5efafea030f, our proxy
`Cursor` object doesn't return a understandable error when we
try to use it when it is already closed
(`NoneType` or `Recursion` error).
Instead of reverting e014cc88394fb9c8f6cff556ca73d5efafea030f,
add a check in `__getitem__` raising a IterfaceError (like psycopg)
if we try to use psycopg cursor (`self._obj`)
task-2766494
Part-of: odoo/odoo#85078
The changes in `auth_password_policy` are largely the owlification of
the password meter widget:
- modernize the password policy module and convert it to an
odoo-module (note: now exports a pseudo-abstract class which is
really a policy, for the sake of somewhat sensibly typing
`recommendations`)
- replace the implementation of the Meter and PasswordField widgets by
owl versions
The changes to web and base stem from taking a look at converting the
ChangePassword wizard, and finding that it would be a pain in the ass
but also... unnecessary? It seems to have been done as a wizard
completely in javascript despite being backend-only for legacy
reasons: apparently one of the very old web clients (v5 or v6
probably) implemented it as a "native action" which was directly part
of the client's UI, and so it had to be implemented entirely in the
client.
Over time it was moved back into the regular UI (and moved around
quite a bit), hooked as a client action to maintain access to the
existing UI / dialog.
But since it's been an action opened via a button for years it can
just... be a normal wizard, with password fields, which
auth_password_policy can then set the widget of.
So did that:
- removed the old unnecessary JS, and its dedicated endpoint (which is
*not* used by portal, portal has its own endpoint)
- used check_identity for the "old password check"
- split out `change_password` with an internal bit so we can have a
safer (and logged) "set user password" without needing to provide
the old password, which is now used for the bulk password change
wizard as well
- added a small wizard which just takes a new password (and
confirmation), for safety a given change password wizard is only
accessible to their creator (also the wizard is restricted to
employees though technically it would probably be fine for portal
users as well)
Rather than extensive messy rewrite / monkeypatching (the original
wizard was 57 LOC, though also 22 LOC of template, the auth_policy
hooking / patching was 33, plus 8 lines of CSS),
`auth_password_policy` just sets the widget of the `new_password`
field in the new wizard, much as it did the bulk wizard.
Also improve the "hide meter if field is empty" feature by leveraging
`:placeholder-shown`. This requires setting a placeholder, and while
empty works fine in firefox, it doesn't work in chrome. So the
placeholder needs to be a single space. Still, seems better than
updating a fake attribute or manipulating a class for the sake of
trivial styling.
Notes on unlink + transient vacuum
Although the wizard object is only created when actually calling
`change_password`, and is deleted on success, it is possible for the
user to get an error and fail to continue (it should be unlikely
without overrides since the passwords are checked while creating /
saving but...).
While in that case the `new_password` in the database is not the
user's own, it could be their *future* password, or give evidence as
to their password-creation scheme, or some other signal useful to
attack that front of the user's life and behavior. As such, quickly
removing leftovers from the database (by setting a very low transient
lifetime) seems like a good idea.
This is compounded by the `check_identity` having a grace period of 10
minutes. 0.1 is 6 minutes, but because the cron runs every 10 the user
effectively has 6~10 minutes between the moment they create an
incorrect / incomplete version of the wizard and the moment where it
is destroyed if they just leave it.
closesodoo/odoo#99458
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Instead of fetching all field attributes sent with the field list
to the web client,
restrict the attributes to the ones actually required by the web client
This allows, for instance,
to gain 44,75KB on each call on `get_views` for `account.move`,
from 208.78KB to 164.03KB,
with only `account_accountant` installed.
closesodoo/odoo#99660
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Since #84736 the access right check queries were merged into one
Unfortunately the query plan for cases of a group with tons of users
can be suboptimal.
Using a subselect will force the query plan in order to avoid an index
only scan on res_groups_users_rel.
Performances where tested in production, this new query being
around 300 and 1000 times faster for two tests cases (res.partner
and account.payment.term)
closesodoo/odoo#99747
X-original-commit: c7f6a88466878698a78ba26ceeb18247df6dc955
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This commit (and its enterprise equivalent) purpose is to smooth the
user experience and remove unnecessary steps.
The commits:
- Remove all stat buttons except the status and collaborators one from
the project edit form.
- Add multiple small changes to string/name fo fields in views
- Change the custom O2M widget for a standard M2M for the management of
child tasks
- Add the milestone field to the portal page of shared project.
- Remove the wizard 'marked_as_reached_milestone'
- Add the automatic generation of milestone when a SO is confirmed if
some SOL need it
task-2941835
closesodoo/odoo#98546
Related: odoo/enterprise#30628
Related: odoo/upgrade#3798
Signed-off-by: Laurent Stukkens (ltu) <ltu@odoo.com>
This technical module can be used to configure onboarding panels flows at
company or global level (required for new Appointment onboarding, see ENT PR).
This implementation, which should be able to support the migration of the
other onboardings, separates the closed state from the completion state
(allowing to know that steps are not completed even though the panel is
closed).
Basic tree and form view are included.
Adding an onboarding with this module requires to provide records data for the
models:
- `onboarding.onboarding` + action to close the panel
- `onboarding.onboarding.step` + actions for the 'opening' and 'saving' of each
step.
See related ENT PR for the `appointment` example, in particular
`onboarding_data.xml` and `onboarding_onboarding.py`
Several python tests are also included.
Misc. In base, we slightly refresh the wording/style of the modal closing the
onboarding panel.
Task-2852375
Part of Task-2900763
Part-of: odoo/odoo#97105
* = account, sale
- Fix typo
- Make private as very specific role
- Allows to grep similar functionality from new onboarding module and previous
common implementation of onboardings.
Task-2852375
Part of Task-2900763
Part-of: odoo/odoo#97105
Purpose:
Enable to add a banner to the calendar view. This is used to show the
appointment onboarding panel (see related ENT PR).
Task-2852375
Part of Task-2900763
See odoo/enterprise#29139
Part-of: odoo/odoo#97105
The domain used in `get_filters`, used in `get_views` is
```python
[('action_id', 'in', [action_id, False]), ('model_id', '=', model), ('user_id', 'in', [self._uid, False])]
```
Therefore filtering the filters on `action_id`, `model_id` and
`user_id`.
An index on `(action_id, model_id, user_id)` would therefore be welcome
in order to search quicker on filters.
The unique constraint `name_model_uid_unique` almost does it,
but it puts the name in first, and therefore the index is not used
when searching on the above domain.
By moving `name` to the end of the unique constraint,
the index created for this unique constraint becomes
usable for the above domain,
and it makes the SQL statement to search on filters way quicker.
Before:
```sql
EXPLAIN ANALYZE SELECT "ir_filters".id FROM "ir_filters" LEFT JOIN "ir_translation" AS "ir_filters__name" ON ("ir_filters"."id" = "ir_filters__name"."res_id" AND "ir_filters__name"."type" = 'model' AND "ir_filters__name"."name" = 'ir.filters,name' AND "ir_filters__name"."lang" = 'en_US' AND "ir_filters__name"."value" != '') WHERE (((("ir_filters"."active" = true) AND (("ir_filters"."action_id" in (225)) OR "ir_filters"."action_id" IS NULL)) AND ("ir_filters"."model_id" = 'account.move')) AND (("ir_filters"."user_id" in (2)) OR "ir_filters"."user_id" IS NULL)) AND TRUE ORDER BY "ir_filters"."model_id" ,COALESCE("ir_filters__name"."value", "ir_filters"."name") ,"ir_filters"."id" DESC;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sort (cost=3500.58..3500.59 rows=1 width=56) (actual time=12.940..12.942 rows=0 loops=1)
Sort Key: (COALESCE(ir_filters__name.value, (ir_filters.name)::text)), ir_filters.id DESC
Sort Method: quicksort Memory: 25kB
-> Nested Loop Left Join (cost=0.00..3500.57 rows=1 width=56) (actual time=12.933..12.934 rows=0 loops=1)
Join Filter: (ir_filters.id = ir_filters__name.res_id)
-> Seq Scan on ir_filters (cost=0.00..3499.02 rows=1 width=36) (actual time=12.932..12.932 rows=0 loops=1)
Filter: (active AND ((action_id = 225) OR (action_id IS NULL)) AND ((user_id = 2) OR (user_id IS NULL)) AND ((model_id)::text = 'account.move'::text))
Rows Removed by Filter: 100001
-> Seq Scan on ir_translation ir_filters__name (cost=0.00..1.54 rows=1 width=36) (never executed)
Filter: ((value <> ''::text) AND ((type)::text = 'model'::text) AND ((name)::text = 'ir.filters,name'::text) AND ((lang)::text = 'en_US'::text))
Planning Time: 0.311 ms
Execution Time: 12.972 ms
```
After:
```sql
EXPLAIN ANALYZE SELECT "ir_filters".id FROM "ir_filters" LEFT JOIN "ir_translation" AS "ir_filters__name" ON ("ir_filters"."id" = "ir_filters__name"."res_id" AND "ir_filters__name"."type" = 'model' AND "ir_filters__name"."name" = 'ir.filters,name' AND "ir_filters__name"."lang" = 'en_US' AND "ir_filters__name"."value" != '') WHERE (((("ir_filters"."active" = true) AND (("ir_filters"."action_id" in (225)) OR "ir_filters"."action_id" IS NULL)) AND ("ir_filters"."model_id" = 'account.move')) AND (("ir_filters"."user_id" in (2)) OR "ir_filters"."user_id" IS NULL)) AND TRUE ORDER BY "ir_filters"."model_id" ,COALESCE("ir_filters__name"."value", "ir_filters"."name") ,"ir_filters"."id" DESC;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sort (cost=14.44..14.44 rows=1 width=56) (actual time=0.086..0.087 rows=0 loops=1)
Sort Key: (COALESCE(ir_filters__name.value, (ir_filters.name)::text)), ir_filters.id DESC
Sort Method: quicksort Memory: 25kB
-> Nested Loop Left Join (cost=8.86..14.43 rows=1 width=56) (actual time=0.080..0.081 rows=0 loops=1)
Join Filter: (ir_filters.id = ir_filters__name.res_id)
-> Bitmap Heap Scan on ir_filters (cost=8.86..12.87 rows=1 width=36) (actual time=0.079..0.080 rows=0 loops=1)
Recheck Cond: ((((model_id)::text = 'account.move'::text) AND (user_id = 2)) OR (((model_id)::text = 'account.move'::text) AND (user_id IS NULL)))
Filter: (active AND ((action_id = 225) OR (action_id IS NULL)))
-> BitmapOr (cost=8.86..8.86 rows=1 width=0) (actual time=0.078..0.079 rows=0 loops=1)
-> Bitmap Index Scan on ir_filters_name_model_uid_unique (cost=0.00..4.43 rows=1 width=0) (actual time=0.075..0.075 rows=0 loops=1)
Index Cond: (((model_id)::text = 'account.move'::text) AND (user_id = 2))
-> Bitmap Index Scan on ir_filters_name_model_uid_unique (cost=0.00..4.43 rows=1 width=0) (actual time=0.002..0.002 rows=0 loops=1)
Index Cond: (((model_id)::text = 'account.move'::text) AND (user_id IS NULL))
-> Seq Scan on ir_translation ir_filters__name (cost=0.00..1.54 rows=1 width=36) (never executed)
Filter: ((value <> ''::text) AND ((type)::text = 'model'::text) AND ((name)::text = 'ir.filters,name'::text) AND ((lang)::text = 'en_US'::text))
Planning Time: 1.000 ms
Execution Time: 0.121 ms
```
I take the opportunity to add the populate for `ir.filters`,
which helped me for the above analysis.
closesodoo/odoo#99657
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
No need to check access rights if we have no intersection with private
fields.
This is a small performance improvement since this value should be in
cache most of the time, but still usefull for tests.
closesodoo/odoo#99644
X-original-commit: db59dbdd7be12c6ea266d5abaf076cd189a2e7d8
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The main purpose of the test is to provide statistics for the runbot, in
order to detect unexpected variations in the loading of translations.
closesodoo/odoo#99636
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
In back-end views, the attributes
- attrs
- states
- invisible
- readonly
- required
are transfered to the `modifiers` attribute.
The web client only uses this `modifiers` attribute.
Hence, the above attributes which gets transfered
to the `modifiers` attribute can be safely removed
from the architecture sent to the web client.
In addition, in case the modifiers were all falsy
e.g. `{'invisible': False, 'readonly': False, 'required': False}`
`simplify_modifiers` was simplifying these modifiers to
`{}` and `modifiers="{}"` was set on the node,
which is a bit useless and waste transferred bytes.
This allows to gain some KB for all `get_views` calls.
For instance, with only `account_accountant` installed,
`get_views` of `account.move` goes
from 228.01 KB to 208.78,
therefore sparing 10% of KB for each calls.
An example using the `res.partner` form:
Before:
```xml
<field name="is_company" invisible="1" on_change="1" modifiers="{"invisible": true}"/>
<field name="commercial_partner_id" invisible="1" on_change="1" modifiers="{"invisible": true, "readonly": true}" can_create="true" can_write="true"/>
<field name="active" invisible="1" on_change="1" modifiers="{"invisible": true}"/>
<field name="company_id" invisible="1" on_change="1" modifiers="{"invisible": true}" can_create="true" can_write="true"/>
<field name="country_code" invisible="1" modifiers="{"invisible": true, "readonly": true}"/>
<field name="company_type" widget="radio" options="{'horizontal': true}" on_change="1" modifiers="{}"/>
```
After:
```xml
<field name="is_company" on_change="1" modifiers="{"invisible": true}"/>
<field name="commercial_partner_id" on_change="1" modifiers="{"invisible": true, "readonly": true}" can_create="true" can_write="true"/>
<field name="active" on_change="1" modifiers="{"invisible": true}"/>
<field name="company_id" on_change="1" modifiers="{"invisible": true}" can_create="true" can_write="true"/>
<field name="country_code" modifiers="{"invisible": true, "readonly": true}"/>
<field name="company_type" widget="radio" options="{'horizontal': true}" on_change="1"/>
```
closesodoo/odoo#99619
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
The name "Payment Acquirer Test" of the acquirer bundled with the module
`payment_test` is confusing. It is actually the only acquirer that
doesn't connect to a test API, and its purpose is not to make test
transactions but to showcase the integration of other apps (Accounting,
Sales, eCommerce, Subscriptions) with demo payments.
Hence, the module is renamed to `payment_demo` along with its data and
technical keys to better make the distinction between acquirers' test
environment and demo payments.
task-2853481
closesodoo/odoo#99397
Related: odoo/upgrade#3846
Signed-off-by: Antoine Vandevenne (anv) <anv@odoo.com>
In order for websocket upgrade to work with firefox, the http version
must be set to `HTTP/1.1`.
Until now, this was done in the `send_response` method. The issue is
that this method is also used when sending an error. This is problematic
because we use environ to know whether or not the version should be changed
which means any error during `BaseHTTPRequestHandler.parse_request` (such as
wrong http version) would have led to an AttributeError being raised.
In order to solve this issue, this modification is done when making environ.
Moreover, we previously used the request uri to know whether or not the version
should be changed, this was not really reliable, we now use the upgrade header
for this purpose.
closesodoo/odoo#99535
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
We added trigram index for char fields since https://github.com/odoo/odoo/pull/83015.
But if `unaccent` is installed in the database (and isn't force to
`False` on field), these new trigram indexes are pointless and cost a
lot for nothing (almost nothing, it still can be used for equality
operator but in this case a btree will be far more efficient).
The simple way to fix it is to add `unaccent(<column>)` in the index
trigram definition, but unfortunately `unaccent` is not immutable and
may therefore not be indexed. In order to make `unaccent` indexable, we
must declare it as immutable (see
https://stackoverflow.com/questions/11005036/does-postgresql-support-accent-insensitive-collations/11007216#11007216
for more information and how to do that).
With this patch, trigram indexes are created with `unaccent(<column>)`
if the function `unaccent` is available in the database, and for the
fields that are not declared with `unaccent=False`. Moreover, we issue
a warning when `unaccent` is available but is not immutable, in which
case most trigram indexes will be useless.
odoo/upgrade#3736
task-2551518
closesodoo/odoo#95943
Signed-off-by: Rémy Voet <ryv@odoo.com>
The master plan finally comes to an end.
Thanks to:
- odoo/odoo#87522 refactoring `load_views`,
- odoo/odoo#94337 refactoring `common.Form` to prevent changing
invisible fields in unit tests using `Form` instances,
- odoo/odoo#95729 refactoring the behavior of `groups=` in views,
- odoo/odoo#98551 removing the need of the `groups_id` many2many field
on back-end views.
The result returned by `get_view`/`get_views` can now finally be easily
and efficiently cached, in order to cache back-end views.
The goal of this revision is to cache the model views and fields
already post-processed for the web client
(with the modifiers, etc., already computed)
without group restriction.
Then, from this cached version, post-process group related features,
such as removing nodes restricted with a `groups=` attribute,
set the create/write button according to the user access rights to models, ...
Not including the groups in the cache key allows:
- to have less cached versions,
(otherwise it would be one cached version per different group combination)
- to not have to fetch the user groups to compute the key
(with the current cache key,
there is nothing to fetch from the database to compute the key)
Besides, post-processing the groups features
after taking the view from the cache of the view doesn't take a tremendous time:
- parsing arch from/to string with etree is fast,
- removing the `groups=` nodes using etree is fast,
- adding the `create="False"`, `write="False"`, `delete="False"` on the view
root node according to the access right of the user on the model is fast.
This allows way faster calls to `get_views` by the web client,
as the server no longer need, for each call, to fetch the views in database,
combine the inherited views, post-process the modifiers attributes, etc.
Timing tests are available on the pull request of this revision.
Part-of: odoo/odoo#99417
The cache key of _get_bindings was not super efficient.
The result of _get_bindings is cached,
but its performance was altered by the cache
key which requires to fetch the user groups for each call to
_get_bindings.
Besides, as there is a lot of possible group
combination, this resulted in a lot of possible cache keys,
and therefore a lot of cached values.
This revision aims to make _get_bindings more efficient
by:
- do not use the groups in the cache keys (less cached values)
- filter out actions not available to the user groups after
retrieving them from the cache
- use has_group to do the above, which is itself cached as well,
and therefore do not need to fetch the user groups
at each call to get_bindings.
In addition, move get_bindings from `get_view`
to `get_views`. If there was 3 views asked by `get_views`
(let's say kanban, list, form)
`get_bindings` was being called 3 times, through `get_view`
with each time the same arguments and therefore the same result :-).
Moving it to `get_views` allows to call it only once for all view types
requested, and for the web client it doesn't change much,
as it always request the toolbar/get_bindings through `get_views` only.
In addition, add the lang to the cache of _get_bindings.
it was actually a bug not to put it: if you had 2 users
with the same group set, using 2 different languages,
the user accessing first the get_bindings would cache
the action names within his language, and then the second
user would see the action name within the language of the first user
:-).
Before
```py
In [1]: %time for i in range(1000): self.env['ir.actions.actions'].get_bindings('res.partner'); self.env.invalidate_all();
CPU times: user 790 ms, sys: 104 ms, total: 893 ms
Wall time: 1.7 s
```
After
```py
In [1]: %time for i in range(1000): self.env['ir.actions.actions'].get_bindings('res.partner'); self.env.invalidate_all();
CPU times: user 23.5 ms, sys: 9.12 ms, total: 32.7 ms
Wall time: 36.9 ms
```
Part-of: odoo/odoo#99417
Part of the overall v16 SCSS optimization/restyle, task-2704984
- Converts dropdown into "nav ul li" structure.
- Removing of '.o_burger_menu_user'
- Removing of '.o_burger_menu_app'
- Removing of '.o_menu_sections'
- Removing of '.o_burger_menu_section'
- Some 't-key' not necessary anymore (thx to OWL2)
task-2812594
Part-of: odoo/odoo#88073
Co-authored-by: Adrien Dieudonné <adr@odoo.com>
Not entirely sure about TestAllocationRights. For TestEsEdiCommon
issue is quite obviously that it's inherited by tests which are
external, so when the `post_install_l10n` tag gets applied those tests
get run during "normal" l10n and they break.
closesodoo/odoo#98814
Related: odoo/enterprise#30825
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Yannick Tivisse