Commit Graph
242 Commits
Author SHA1 Message Date
Damien Bouvy 203af43784 [IMP] base,base_automation,mail,sms: server action UX
- Slight reword of the 'type/state' field labels for server actions
- Re-ordering of 'type/state' values
- Form view changes
- Allow hiding model in display_name of ir.model.fields base on context
  key (avoid technical details when they are not needed)

Task-3450200

Part-of: odoo/odoo#138804
2023-10-26 13:41:07 +00:00
Damien Bouvy 6bab7a6ae5 [IMP] base: reflect html sanitization attributes
Up until this commit, HTML fields created from the UI or via data
modules or via Studio where alway sanitized, without any possibility of
control by the developper or db admin.

This commit makes it possible to override these attributes for
data-defined fields (python-defined fields cannot be overriden).

This makes it possible to create fields that can e.g. be used as 'block
targets' in the website editor for data modules.

closes odoo/odoo#130544

Related: odoo/enterprise#47116
Related: odoo/upgrade#5290
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
2023-10-26 12:10:08 +00:00
Xavier Morel 39be91b48e [IMP] core: clean up the ACL tracebacks some
Currently if a record is not in the cache it raises a `CacheMiss` the
handling of which leads to loading the record from the database (or
something).

If an issue occurs during that loading, because the loading is an
`except` scope the cache miss gets linked with the new one via

> During handling of the above exception, another exception occurred:

This is both noise (the cache miss is not actually relevant) and
misleading, because the wording makes it look like an unrelated error
occurred during handling.

- move the loading of the record out of the `except` to limit the
  scope of the `KeyError` and avoid "inheriting" it
- try to `from None` a few specific errors to remove implicit linkage,
  as the new error should have all relevant information, the key error
  / cache miss is just an implementation detail

closes odoo/odoo#139680

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-25 19:37:31 +00:00
Xavier Morel 9ebbfdac73 [FIX] *: incorrect translations markings
Fixes a large number of cases where strings are translated then
formatted, instead of letting `_()` do the formatting internally,
which allows it to recover from incorrect translations (missing,
broken, or extra placeholders).

Also

- removes translation markers entirely when there's nothing to
  translate e.g. `_("%s - %s")` is not useful
- fixes a few messes which lead to only partial translatability
  (DRY is generally a bad idea when translations are involved, even
  more so when you don't make the variable part translatable)
- fixes a few nearby issues noticed at the same time
- replaces a few `"%s"` by `%r`, which should automatically quote
  strings relatively appropriately
- fixes translated strings which use `\` to escape a newline (in order
  to fill-paragraph): `\` escapes only the newline, if the
  continuation string is indented this results in a bunch of spaces
  ending in the string to translate, which is pretty garbage for the
  translator, using implicit concatenation works much better

Note: some of the updates revert f-string parameters to %, because
babel (2.9) apparently has trouble with f-strings and blows up trying
to extract them.

Not in scope:

Helping translators fix translatable strings e.g. any translation
string with more than one placeholder probably should use keyword
placeholders

- Provides more context / data to the translator to make sense of the
  sentence.
- Allows reordering the translated terms, which can be necessary
  depending on the sentence and language.

closes odoo/odoo#139314

Related: odoo/enterprise#49311
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-23 16:45:09 +00:00
Lucas Perais 2b28009f5b [IMP] base_automation: better UX
This commit improves the general feel when creating or displaying base automations
The improvement relies mainly on correctly labelling the fields

task-id-3450200

closes odoo/odoo#135932

Related: odoo/enterprise#47609
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
2023-10-20 09:35:12 +00:00
Antoine (ande)andRémy Voet 2c377f4cb8 [FIX] base: external identifier traceback
Current behaviour:
Traceback when trying to create
a new external identifier

Steps to reproduce:
1. Activate the developer mode
2. Go to settings
3. Technical > External Identifiers
4. Click on "New"
5. Traceback

Cause of the issue:
Introduced by https://github.com/odoo/odoo/commit/3c62ca1eb96d571b2b686b5caee370324c589ab4
When computing display_name, the model
can be false, and not be in self.env

opw-3489581

closes odoo/odoo#136279

X-original-commit: 2462df26977acc9e3ccafd57863703aacd2f059e
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
Co-authored-by: Rémy Voet <ryv@odoo.com>
2023-09-22 15:06:09 +00:00
Chong Wang (cwg) 13957b6281 [FIX] core: support cr.execute_values
psycopg2.extras.execute_values was introduced in PR #101237
however it pypasses the override logic for cr.execute. As a result
1. --log-sql cannot log these queries
2. assertQueryCount cannot notice these queries
...

This commit create a new api cr.execute_values to support the same SQL feature
without losing the override logic for cr.execute

closes odoo/odoo#131190

Related: odoo/enterprise#47374
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
2023-09-14 11:22:42 +00:00
Chong Wang (cwg) 506d0cc4ec [FIX] core: fix cached translations
before this commit:
translations updated by `update_field_translation` api cannot be detected by
t-cache and some cached data whose model overrides `write` with an extra
'clear_caches()'

Step to reproduce:
- Create a mega menu, select any template, `Odoo Menu` for the example
- Install another language on the website
- Go to the translated version of your website and enter translate mode
- Change "Camera" in the mega menu to something else
- Save

The change won't be replicated, looking like it did nothing.
From there, removing or adding `edit_translations=1` in the URL will
use different cache version of the page's views and you will see the
outdated value on one and the correct on the other one.

after this commit:
`update_field_translation` will call `write`
it does the following 4 important things
1. mark field as modified
2. execute logics in the override `write` method
3. update write_date if needed to support t-cache

opw-3305117

X-original-commit: 2beb466668e4eb80d7c3ca3947445fb1cb141cff
Part-of: odoo/odoo#135277
2023-09-13 12:19:10 +00:00
MerlinGuillaume 41d07e9d38 [FIX] base: allow deletion of inherited custom field
The inherited field of a custom field cannot be deleted

Steps to reproduce:
1. Install Contacts and Studio
2. Go to Contacts and open any contact
3. Toggle Studio
4. Add a field of any type in the view, remove it and close Studio
5. Go to Settings > Technical > Database Structure > Fields and search
   for `x_studio`

Solution:
Mark the inherited field as manual if its parent field is manual and
allow the deletion of inherited custom field if we also delete its
dependency

Problem:
The inherited field was not marked as custom so it was impossible to
delete it

opw-3093581

closes odoo/odoo#133820

X-original-commit: 526f3407c7c5b4cd014a4d091ca01b9617e6e938
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Guillaume Merlin (megu) <megu@odoo.com>
2023-08-31 18:53:16 +00:00
Miquel Raïch fd53a6bac6 [FIX] base: deleting selection fields of non tabled models
When a model selection record is going to be deleted, a _process_ondelete method is called in order to delete all the records of the corresponding model that have that selection. These records are obtained by calling _get_records, which uses a query that needs a table. Thus, we should avoid cases for non-abstract models that have _auto = False.

closes odoo/odoo#133118

X-original-commit: 408175a727ecbca88957a9d63d8ec28f3e54c9df
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-08-25 10:59:48 +02:00
Jinane Maksoud 6021087f6c [FIX] models: add xmlid for inherited selection values
If model A has a selection field, and is inherited by 2 modules
B and C, B adds selection values to the field, and C inherits model
A under a different module and model names.
Based on the order of installation of B and C, we may end up with
different xmlids.
If B is installed before C, then B will add xmlids for the new
selection values added for model C. But if C is installed before
B, then C will have the selection values from B without xmlids.
The change here ensures that the selection values introduced by B
will always have correct xmlids.

closes odoo/odoo#132765

X-original-commit: c673d9db40cb91d4eb12cb787ff114068a924caa
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-08-23 12:30:45 +02:00
Alvaro Fuentes f7cd412725 [FIX] core: fix remove constraints at uninstall
This patch aims to fix multiple issues with the removal of table
constraints at module uninstall.

1. We cannot remove `ir.model.constraint` records before calling
   `_module_data_uninstall` on them. Otherwise we either won't find them
   when performing the search
   `self.env['ir.model.constraint'].search([('module', 'in',
   modules.ids)]` or, if we somehow keep the ids and use `browse`
   instead, would get an error because `_module_data_uninstall` tries to
   access field values of records already removed. Note, although not an
   issue, the removal is redundant for non FK constraints since
   `_model_data_uninstall` already unlinks the record.
2. When a constraint has a name longer than 63 characters (Postgres
   default) we would fail the check for the existence of the constraint
   since the names are truncated.
3. When checking for the presence of a constraint we assumed its type
   would be `u` in `pg_constraint` because for us that means non FK
   (i.e. not `f` type). That's incorrect since there are many more
   types. Here we propose to handle `c,u,x` types.

For bullet 2 we use `tools.make_identifier` that hashes the name and
ensures it fits in the 63 chars limit.

Revert "[IMP] models: warn if constraint key len exceed 63"

The check from commit 823d9e10dc is no
longer needed since the name is ensured to fit length limit.

[IMP] code: improve uninstall tests

Perform extra checks for removal of SQL constraints. Note the test is
commented out in `__init__.py`. It can be uncommented locally for
testing. It's kept commented out to avoid random errors in runbot.

closes odoo/odoo#129084

X-original-commit: af288b7178c25261329dd85a2e64b9dd635cd9e1
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-08-16 16:04:47 +02:00
Xavier-Do eb708e5cad [IMP] base: avoid cache_invalidation
A clear_cache was added in _update_xmlids in pr 119813.

If it was needed in case of update, it is not useful when creating an
xmlid or when the value is unchanged.

The initial idea was to detect if an update or an insert was done, maybe
using create_date and write_date. Unfortunately the create_date is the
same as the write_date in the same transaction. This is unlikely but in
this case, we could update the cache in place.

The final behavior is to update the cache in all case, and notify other
workers only if a model was updated.

This will help to avoid invalidating the cache too mush during module
loading. Even if the impact on time is small, the increase in queries
was visible.

This solution would even be a slight improvement on previous query count

closes odoo/odoo#129029

Related: odoo/enterprise#44349
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
2023-07-20 14:23:15 +02:00
Xavier-Do d5ed47a11f [IMP] base: simplify _xmlid_lookup cache
The _xmlid_lookup cache contains the id of the ir.model.data with is
unused, making cache pupulation and usage more complex than needed.

Removing this id in prevision of the next commit

Part-of: odoo/odoo#129029
2023-07-20 14:23:15 +02:00
Xavier-Do 4c9968397b [IMP] base: avoid invalidation on xmlid updates
Splitting the cached revealed a missing cache invaldation.
The cache invalidation added a a query in test_related_fields

The query can be avoid by not updating the xmlid if it is not useful.

closes odoo/odoo#119813

Related: odoo/enterprise#42527
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2023-07-18 11:42:27 +02:00
Xavier-Do 595aa24843 [IMP] registry: multiple ormcache
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.

Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.

The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.

Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.

The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.

We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.

Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.

Note that this poc is inspired from the long term cache but not all
use case where applie yet.

Part-of: odoo/odoo#119813
2023-07-18 11:42:26 +02:00
Xavier Morel ce1a4a2ee0 [IMP] base: deprecate global rules
ir.model.access rules without group will now log a warning

Part-of: odoo/odoo#125216
2023-07-11 22:33:47 +02:00
Denis Ledoux 3d2f76671c [FIX] core: missing escaping in sql constraint name_manual_field
Oversight in revision
8c38baee85

In SQL,
The underscore character ( _ ) represents a single character
to match a pattern from a word or string.

Meaning
`name LIKE 'x_%'`
allows names starting by `x`, and not names starting by `x_` as expected.

Add the escape in the constraint to enforce starting by `x_`
and not just `x`

closes odoo/odoo#127883
2023-07-10 16:59:24 +02:00
Denis Ledoux 8c38baee85 [IMP] core: enforce custom fields to starts with x_
The `api.constrains` `_check_name` on `ir.model.fields`
ensures users do not create custom fields not starting with `x_`.
It offers a user-friendly message when users try to do that.

However, this `api.constrains` can be bypassed,
for instance when updating the field name through raw SQL.

This revision aims to no longer load custom fields
if they do not starts with `x_`.

In addition, it replaces the Python constraint
with an SQL constraint in order to have a stronger constraint.
Following bugs or unexpected flows, people
were able to add custom fields not starting with `x_`
with the `api.constrains` alone.
Adding the SQL constraint will tighten the constraint.

However, even without that SQL constraint,
for instance if users drop the SQL constraint manually,
custom field not starting by `x_` must not be loaded.
A user would be able to drop that constraint
through a SQL shell or a `cr.execute` in a server action.

This is to prevent users to override class attributes.

closes odoo/odoo#127702

Related: odoo/upgrade#4908
2023-07-07 12:10:05 +02:00
Rémy Voet (ryv) c5cb357d90 [IMP] *: add dependencies to display_name field
The previous commit introduced an optimization to reduce the number of
queries and fields fetched when we call `name_search`. But it works much
better when the dependencies of `display_name` contain field names used
in the calculation (on the same record/model).

Then, to improve the performance and the cache coherency, add `depends`
and `depends_context` depending on the custom `_compute_display_name`.
Add only the first level of dependencies (never traverse relational
field) because only these have a positive impact on the previous
optimization and the cost is very low (see `modified`).
About `depends_context`, we don't include `lang` because (when `_` is
used by example) it is unlikely to get the same display_name in the same
request with two different lang.

closes odoo/odoo#122085

Related: odoo/documentation#4639
Related: odoo/enterprise#42599
Related: odoo/upgrade#4780
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-06-28 17:41:19 +02:00
Rémy Voet (ryv) 3c62ca1eb9 [REM] core: remove name_get API
Rationale
=========

Since v8, the `display_name` field is present on all models. By default,
`display_name` uses `name_get` which has pretty much the same purpose
(return record name used by the web client). Gradually, many (backend)
developers (and the ORM: https://github.com/odoo/odoo/commit/6da1c3ac4c036eac289597602976538e243cb939)
started using `display_name` (more convenient than
`record.name_get()[0][1]`) but it still had the `name_get` override.
It becomes more complex than necessary and poeple start to misunderstand
the two (and sometimes override both, leading to inconstiencies between
`display_name`/`name_get`).

To simplify the ORM and the API, we decided to keep only one of them,
the `display_name` field:
- It is much more convenient from a backend point of view
(`record.name_get()[0][1]` vs `record.display_name`)
- It is cached during the same transaction (and invalidated if
its dependencies change)
- It can be overridden like any other compute field (override
`_compute_display_name` with any extra dependencies)
- `name_get` is replaced by `read(['display_name'])`
(API perceptive), which can actually be more efficient
(if `display_name`'s depends are correct, the ORM will only fetch the
fields it needs instead of every prefetchable field)

Changes
=======

- Deprecates `name_get` for the v17 and based the method on
`display_name` (the opposite of before)
- Converts all usage of `name_get`
- Overrides of `name_get` are now overrides of `_compute_display_name`
- For `res.partner`, rename the field store `display_name` into
`complete_name` because `display_name` context-dependent and it makes
no sense to have a compute store that is context-dependent.
- Previously, it was possible to return multiple names for the same
record with `name_get`, but it was tricky and most of the usage of
this `name_get` didn't take this into account. The only example of
this is the `name_get` of `product.product`
(now use `", ".join(<names>)`).

Part-of: odoo/odoo#122085
2023-06-28 17:41:19 +02:00
fdardenne 71ab15e4d5 [IMP] base: store currency_field in the database
Before this commit, it was not possible to create a custom monetary
field without having currencies named `currency_id` or `x_currency_id`
in the model.

With this commit, we store the currency_field from the Monetary field
in the database. This allows to create custom monetary field with
currencies name we want.

A good use case for this IMP is for Studio, we can now create and edit
monetary field easily without storing it as a `float` in the ORM.

Part-of: odoo/odoo#118199
2023-06-19 10:40:38 +02:00
Rahul Prajapati 54b1cb8bb9 [FIX] base: validation for studio fields compute dependencies
Before this commit:

`id` cannot be set as the dependency for a field's compute method.
So, when a user configures `id` as a dependency for a studio field's compute,
there is no error shown to the user and
the field is saved with `id` as a dependency.

After this commit:

It will throw a `UserError` to the user.

sentry-3979435039

closes odoo/odoo#124660

X-original-commit: 9eab433e09aafafd3015f731b9ec449965d97748
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-06-13 10:13:29 +02:00
Rahul Prajapati 8d7e0cefc0 [FIX] base: unlink custom cron after uninstalling model
When user uninstalls a module it doesn't unlinks custom crons
which are created by user and it will keep running in the background
and will generate traceback.

So, to fix this we unlink all the custom crons which are related to the
module being uninstalled.

sentry-3929309220

closes odoo/odoo#124175

X-original-commit: b0202488220bd48dd177e9e0601f2f3ba8d7a0f1
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
Signed-off-by: Rahul Prajapati (rapr) <rapr@odoo.com>
2023-06-08 08:02:44 +02:00
Xavier Morel 7c78bb5e5c [FIX] base: uninstallation of project
Since the "dirty flag" refactoring of
384fda2c2a
`IrModelFields._prepare_update` did not cope well with fields missing
from the python-side models, which can during uninstallation for
custom fields (possibly because the script loads the registry
incorrectly, not entirely clear).

Because of a custom field created by worksheet linking to it, the
removal of the `project.task` table would fail, making the
reinstallation of project fail to restore several constraints.

This case was actually handled correctly just a few lines above when
trying to resolve field dependencies, both record and field would be
checked for their presence before actually trying to use them.

Getting the model from the registry / environment has not been noticed
to break uninstallations, but might as well do that too so everything
lines up, and just in case.

X-original-commit: 05aca6ee4ce795b85b90430efd761f0cd3a6d5a3
Part-of: odoo/odoo#121522
2023-05-16 15:55:37 +02:00
Xavier Morel 95fa9fadb8 [FIX] base, crm: uninstallation
Uninstallation does not cope well with `setup_models` being performed
unconditionally as those will dramatically alter registry states, and
resurrect computes which the uninstallation has disabled: rather than
try to update registry models in-place (which is rather fraught) the
uninstallation deletes the columns, tables, and `ir.*` reflection
records and only after all of that is done does it reset the registry.

This means while it does fix up the registry caches (`field_depends`
and `field_triggers`) as it goes, resetting those may cause the
recomputation of fields whose columns have been deleted, possibly
based on dependencies whose columns have also been deleted.

As such these kinds of manipulations should either be performed in
`@ondelete` methods which don't get executed during uninstallation, or
they should be gated behind an uninstallation check.

In crm the latter is necessary, as `ondelete` runs before `unlink`
actually executes, and the registry reset would run too early (and
unnecessarily).

In base, only the latter is possible as we're not in `unlink` itself,
instead `IrModelFields._prepare_update` is called *during*
uninstallation and its trailing `setup_models` causes the issue.

X-original-commit: 357b9f2c9fd44e14e5b7c9d3c17f1794691986f3
Part-of: odoo/odoo#121522
2023-05-16 15:55:37 +02:00
Chong Wang (cwg) 55c9ca692f [FIX] base: translate ir model fields
before this commit:
after #109858
The method `update_field_translations` won't directly call the `write`
As a result, when changing the translation of fields from translation dialog,
the orm cache won't be cleared, and translations won't be updated in views
even after refresh the page

after this commit:
when users translate fields and refresh the page, the new translation can be
updated in new views

opw-3267024

closes odoo/odoo#120602

X-original-commit: 8d8dbab203fe7c153522dcb2a420d97dc4adaadb
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2023-05-05 12:44:05 +02:00
Victor Feyens f4ea6d3226 [FIX] *: strict api for main orm methods
Enforce strict types for returned values for
* create
* write
* unlink
* default_get

to make those methods more consistent and reliable.
Also make sure they can be called with empty self/values,
i.e. that they follow the same behavior as the base methods
defined in the main orm Model.

closes odoo/odoo#116809

Related: odoo/enterprise#38880
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2023-04-25 15:20:43 +02:00
Xavier Morel cedc968f22 [FIX] base: better filter drop table warnings
Along with improving the table_kind API, #117439 added a warning when
dropping non-tables.

This warning turns out to trigger on many models, in two major cases:

- the hook is called on abstract models, which don't have a table in
  the first place, it probably should not be called for those but can
  easily be skipped
- the hook is also called on `_auto = False` which don't have a table
  anymore, this is because we use `DROP TABLE $table CASCADE`, which
  implicitly drops any view (or table) depending on the table, it
  might be possible to avoid this by ensuring we drop models in
  reverse topological order *and* drop all of a module's views then
  tables at once (by passing multiple names to `DROP`, but for now
  just ignore the case where we're trying to drop an object which
  can't be found

closes odoo/odoo#118194

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-11 15:17:32 +02:00
Rahul Prajapati 10620c0caf [FIX] base: fix validation for co-model field in ir_field view
The co-model field in `IrField` view is accepting Related Model name
even if it is invalid. And replaces the invalid model name with `_unknown`.
This causes KeyError `display_name` when trying to create or read a record.

To fix that we are adding Validations for the Related Model field when it gets
updated from UI side.

sentry-3933471439

closes odoo/odoo#118052

X-original-commit: e4f7798fd5a5cccd5d385f535fbc21264cb6948b
Signed-off-by: Rémy Voet <ryv@odoo.com>
2023-04-07 19:43:33 +02:00
Xavier Morel 8932d447aa [FIX] core: table_kind semantics (incorrect classification of toast as temp)
17c4f47b0a updated table_kind to return
`pg_class.relkind`, however the semantics are *not* the same, and that
was ignored: `relkind = t` is for *toast* tables, not *temporary*
tables. In `pg_class` the temporary-ness is instead signaled by
`relpersistence` (which applies to both tables and sequences), temp
(and unlogged) tables have `relkind = r`. `existing_tables` does that
correctly, possibly unwittingly.

While at it, upgrade `table_kind` to return an `enum` (whose value is
the old discriminant).

closes odoo/odoo#117444

Related: odoo/enterprise#39185
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-07 11:37:25 +02:00
Louis Wicket (wil) 9afe7c74c9 [IMP] *: remove "French spacing" 👺
According to Wiktionary, French spacing is "the archaic practice (though
still current in French) of inserting a space around colons, semicolons,
question marks, and exclamation marks". This is not standard practice in
English and most languages of the world.

The purpose of this commit is to start purging the code from this typo,
as it may reflect poorly on the software for some people.

closes odoo/odoo#114533

Related: odoo/enterprise#37853
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2023-03-14 15:52:10 +01:00
Raphael Collet 161e5fad3b [REF] core: make APIs on registry for computation triggers
Those APIs are aimed at hiding the implementation of trigger trees,
dependent fields and fields modifying relations.  Explicit APIs simplify
the profiling of executions and comparison of implementations for
building trigger trees.

X-original-commit: d12b9270375634e738f5288d0c523ac0eec2fa18
Part-of: odoo/odoo#111946
2023-02-04 22:39:45 +01:00
Denis Ledoux f498ab4b13 [IMP] base: remove unused check_group methods on ir.model.access
It looks like these methods where an alternative to `has_groups`,
but not used since a while.

In Odoo 10.0, there is only one hit, in point_of_sale,
which has been replaced by `user_has_groups`
in 11.0 with revision
34c661111b

In Odoo 9.0, there are a few more hits in base,
which have been replaced by `self.env.user.has_group`
in 11.0 with revision
4ddc323139

closes odoo/odoo#111622

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-02-02 21:25:12 +01:00
Florian Vranckxandxmo-odoo 7dc2190fa4 [IMP] test_lint, * : SQL injection detection
This commit brings a new way to detect sql injection.

Previously, this test was meant to push developers to use the second argument of cr.execute(query,args) for parameters.

However, this also meant that the test will always be green as soon as the second argument is used.

This commit aims to change that by tracking the source of information of all variables used to build a query. Parsing of the AST in reverse, starting from the query variable itself.

However these are some current limitations:
  -The hierarchy of Odoo modules is currently not taken into account. If two functions have the same name, they will both be evaluated to find if their return value is part of the query
  -Object mutation is not supported and is not evaluated
  -Whitelisted values are too wide in order to reduce the amount of false positives.

Even if this test is blocking, it can be disabled by adding #pylint: disable=sql-injection at the end of the line.

closes odoo/odoo#101237

Related: odoo/enterprise#35697
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Co-authored-by: xmo-odoo <xmo@odoo.com>
2023-01-26 14:19:00 +01:00
Rémy Voet (ryv) f6cf94d4bd [FIX] *: ormcache works with annotation
The `ormcache` decorator fails to create the key method
(`determine_key`) when the method signature contains any annotation.
Fix it by removing annotation of the signature.

closes odoo/odoo#109777

Signed-off-by: Raphael Collet <rco@odoo.com>
2023-01-23 14:25:14 +01:00
ef00294e71 [IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation.  Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table.  The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages).  Empty text is allowed in translation values, but not NULL.

Here are examples for a field with translate=True:

    NULL
    {"en_US": "Foo"}
    {"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
    {"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}

Like before, writing False to the field makes it NULL, i.e., False in all
languages.  However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.

Here are examples for a field with translate=xml_translate:

    NULL
    {"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}

Change for callable(translate) fields: one can now write any value in any
language on such a field.  The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values.  Basically the
structure of the value must remain the same in all languages, like before.

Reading a translated field is now both simpler and faster than the former
implementation.  We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:

    SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...

The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages).  For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.

Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language.  Before this change, only the source value of the
field ('en_US') could be indexed.

Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages.  We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.

Code translations are no longer stored into the database.  They become static,
and are extracted from the PO files when needed.  The worker simply uses a cache
with extracted code translations for performance.  This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker.  Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).

Performance summary:
 (+) reading 'model' translated fields is faster
 (+) reading 'model_terms' translated fields is much faster (no need to inject
     translations into the source value)
 (+) searching translated fields with operator 'ilike' is much faster when the
     field is indexed with 'trigram'
 (+) updating translated fields requires less ORM flushing
 (-) importing translations from PO files is 2x slower

Some extra fixes:
 - make field 'name' of ir.actions.actions translated; because of the PG
   inheritance, this is necessary to make the column definition consistent in
   all models that inherit from ir.actions.actions.
 - add some backend API for the web/website client for editing translations
 - move methods get_field_string() to model ir.model.fields
 - move _load_module_terms to model ir.module.module
 - adapt tests in test_impex, test_new_api
 - because env.lang is injected into SQL queries, its returned value is
   now guaranteed to correspond to a valid active language or None
 - remove wizard to insert missing translations (no longer makes sense)

task-id: 2081307

Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-09-15 22:37:50 +02:00
Xavier-Do 31b04aa44b [IMP] base: batch access rules for visible_menu
Visible_menu check per_read model per model leading to a lot of queries
We can get all readable fields with one query

Computation time goes from ~400 total to less than ~50 total
The different would be even bigger on a server with a distant database.
(higher ping)

Part-of: odoo/odoo#99176
2022-09-12 13:48:59 +02:00
Fabien Pinckaers 781a771295 [IMP] remove duplicated index
The following indexes were in double:
- ir_module_module_name_index, ir_module_module_name_uniq
- ir_model_obj_name_uniq, ir_model_model_index
- ir_config_parameter_key_index, ir_config_parameter_key_uniq
- decimal_precision_name_index, decimal_precision_name_uniq
- account_edi_proxy_client_user_id_client_index, account_edi_proxy_client_user_unique_id_client
- mail_channel_rtc_session_channel_member_id_index, mail_channel_rtc_session_channel_member_unique
- loyalty_card_code_index, loyalty_card_card_code_unique

Part-of: odoo/odoo#99795
2022-09-08 12:46:50 +02:00
Xavier-Do 87e357ef82 [FIX] base: speedup access rights check query
Since #84736 the access right check queries were merged into one
Unfortunately the query plan for cases of a group with tons of users
can be suboptimal.

Using a subselect will force the query plan in order to avoid an index
only scan on res_groups_users_rel.

Performances where tested in production, this new query being
around 300 and 1000 times faster for two tests cases (res.partner
 and account.payment.term)

closes odoo/odoo#99747

X-original-commit: c7f6a88466878698a78ba26ceeb18247df6dc955
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2022-09-08 01:08:28 +02:00
Rémy Voet (ryv) 713fc48693 [REM] core: remove _check_concurrency method
`_check_concurrency` was used to check if 2 people were modifying the
same record at the same time. It was doing so by setting a special
`__last_update` value inside the context that was later evaluted to
prevent some concurrency issues.

It was mostly unused, wasted a lot of cpu cycles and was not covering
all cases (e.g. pending write).

closes odoo/odoo#87756

Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
2022-09-01 10:30:10 +02:00
Raphael Collet 07744c64e9 [FIX] core: in ir.model.fields, discard deleted fields from registry
This patch discard the fields to delete from data structures like field
triggers, field inverses, and fields to compute.

This is quite useful when uninstalling modules.  This commit fixes the
uninstallation of modules base_setup, bus, mail, web_tour.

closes odoo/odoo#98668

X-original-commit: 49398a769b2ec4965bf148f10b5e13d0a6c1acff
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-08-25 22:46:04 +02:00
Raphael ColletandVincent Schippefilt 384fda2c2a [REF] core: replace towrite by dirty flag in cache
Merging both the memory of field values and suspended updates has
several advantages:
 - avoid inconsistencies between cache and towrite
 - cache updates can be made safer w.r.t. dirty flag

However, the dirty flag in cache does not go well with context-dependent
fields.  When a context-dependent field is dirty in cache, the value to
store in the database is accessible through some context values.  But
when the model is flushed, the context values on the current environment
may be different.  When this happens, the method flush() fails to
retrieve the data to flush.

The proposed solution is to store the "dirty" value in cache under
conventional context values, and to retrieve them under the same
conventional context values to flush them.  For instance, when storing
the value of a binary field, it will be stored once under the context
value `context.get('bin_size')`, and a second time under the context
value `None`.  The flush implementation will then retrieve the value
using the context value `None`.

Translated fields are also problematic when a value is put in cache with
an environment where lang=False, and the value is retrieved with another
environment where lang=None.  This issue is addressed by normalizing the
context key 'lang' to None when the context value is False.

Part-of: odoo/odoo#95325
Co-authored-by: Vincent Schippefilt <vsc@odoo.com>
2022-07-14 22:45:44 +02:00
Raphael ColletandVincent Schippefilt eb67feb590 [FIX] *: cache consistency
In module mail, invalidating 'message_ids' on a mail thread also
invalidates its inverse field 'res_id' on messages.  If you haven't
flushed it before, your cache will be inconsistent, as shown by the test
/mail:TestMailgateway.test_message_process_bounce_records_channel.

In module purchase_stock, add depends on report.stock.quantity.  This
ensures that when the model is queried after changes in other models,
the data on which the SQL view depends is flushed to the database before
querying that model's table.

closes odoo/odoo#66938

Related: odoo/enterprise#16722
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Vincent Schippefilt <vsc@odoo.com>
2022-07-05 11:35:01 +02:00
mreficent daae7fab63 [IMP] base: add modules in _reflect_fields warning message
closes odoo/odoo#69130

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-06-29 17:11:23 +02:00
Raphael Collet 60a1452a40 [REF] base: adapt code to new flush API
Part-of: odoo/odoo#87527
2022-05-25 18:00:47 +02:00
Jeremy Kersten 79d7d78b75 [FIX] base: allow to duplicate an External ID
Before this commit, the ir_model_data_module_name_uniq_index constraints avoid
to duplicate an XML ID.

Now on copy, we add a random suffix to allow the end user to update just the
record id (and rename the name).

closes odoo/odoo#92108

Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2022-05-24 12:48:01 +02:00
Victor Feyens 2296e1e216 [IMP] core: only translate what's needed
There is no need to translate 4 error messages if only one is used.

To do so, use the tooling dedicated to lazy translations.

Part-of: odoo/odoo#91743
2022-05-19 23:16:46 +02:00
william 3155c3e425 [IMP] core,*: add helper for name_search
Most of the extensions of `_name_get` are very similar and only want to
search for the given string in multiple fields.
A lot of extensions also don't take into account the negative operators.
Some implementations were also really outdated and needlessly
complicated.

closes odoo/odoo#86588

Related: odoo/enterprise#25608
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-04-06 18:37:39 +02:00
Raphael Collet 67f343ba6a [FIX] base: record count on ir.model defined by query
Models can be defined by an SQL table, an SQL view or an SQL query (with
attribute _table_query).  Counting records makes little sense when the
model does not correspond to a table, and actually fails when it is
defined by an SQL query.

We fix the compute method by counting records only for models where
_auto=True.

closes odoo/odoo#86913

X-original-commit: 956a0a1cc22f3f35fc128e32cb267e2baa793fb9
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-03-22 15:17:04 +01:00