This commit includes the exception
details when logging an error in the registry module.
By importing the sys module and using the exc_info() method,
the commit ensures that the complete exception information is captured
by Sentry.
This modification improves the error reporting functionality
by providing more comprehensive information about the encountered
exceptions. This will aid in debugging and diagnosing issues,
enabling faster resolution of potential problems.
closesodoo/odoo#124318
X-original-commit: 55118b726bcd2532d2a7e67322eccb1ea6b75b7f
Signed-off-by: Achraf Ben Azzouz (abz) <abz@odoo.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When inside an HttpCase, the end of a successful request will
`signal_changes` meaning that the registry_invalidated flag is removed.
A second issue is that this flag is thread local meaning that if a
request set the flag, it won't be visible from the test thread.
For those reasons, this commit ensures the registry sequences are
incremented as in production mode, and adds a check that the sequence
didn't change during the tests, calling `setup_models` the registry
manually if needed.
closesodoo/odoo#121268
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
odoo/odoo#111651 improved and optimised triggers but dropped the
in-place cleanup of field dependencies. As a consequence, during
module uninstallation if a stored computed field is removed (because
it's part of a module being uninstalled), and one of its dependencies
is subsequently altered (e.g. it's itself removed, or written to) the
second update will break as the query trying to find out which
dependent records to update will error, either because of trying to
select / filter on a missing column, or because of trying to fetch
in a missing table.
The simplest examples of this issue are computed fields with a
dependency on `ir.model`:
- In `calendar`, `calendar.event.res_model` is a related on
`res_model_id.model`, this prevents the removal of *any* `ir.model`
record if it gets uninstalled.
As a result the `calendar.attendee` and `calendar.event` tables
don't get removed (just emptied of all their non-automatic fields),
their records remain as well, and when the non-automatic fields get
re-added during installation re-instating the NOT NULL constraints
fails, breaking the uninstall/reinstall test.
- In `payment`, `payment.provider.module_state` is a related on
`module_id.state`, this breaks *during* uninstallation, as after
`module_uninstall` first calls `_module_data_uninstall` which
removes the field, then it *updates the modules being uninstalled*
(sets their state), which tries to find out which
`payment.provider`'s `module_state` is should update, which breaks
because the `module_id` column has been removed.
This second one was worked around in odoo/odoo#118900, by marking
modules as uninstalled before actually gutting them, but as it turns
out the "actual" fix is needed anyway. So revert the workaround, and
actually fix the issue.
closesodoo/odoo#119130
X-original-commit: 48a420efcf7c7b46416bab006003553cc9d23846
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The TriggerTree class does not have a specific __repr__(). It thus
falls back on dict's __repr__(), which does not show the root of the
tree. This makes debugging hard and confusing.
closesodoo/odoo#113521
Signed-off-by: Raphael Collet <rco@odoo.com>
Odoo Test environments requires to modify many parts of the unittest
TestCase, Suite and Result.
The main initial reason is to **avoid to postpone result at the end of
the test suite**, because even if it is convenient to have all errors
visible after the tests in some case, odoo logs adds information during
the execution that can be useful to debug when a test fail, to have
context for an error. (see **OdooTestResult**)
We are also fixing the stack trace comming from a unittest and since
there is no proper way to hook inside the TestPartExecutor, a dirty hack
injects anoter result on the outcome to manage the error and complete
the stack trace. This was also a way to avoid to postpone subtest logs
at the end of the test case (see _ErrorCatcher)
`_feedErrorsToResult` was used to test the test suite behavior since
there are many customization and this is quite fragile, especially if
unittest changes behavior in other python version.
**Python 3.11** introduced python/cpython#664448d8 That, in a way, goes
in the same direction of the changed introduced with _ErrorCatcher:
immediately feed errors to resut instead of postponing it. But this also
removes `_feedErrorsToResult` that was used to test this behaviors, as
well as other ones.
Since odoo should remain multi-version, this amount of changes on the
initial behavior become to complicate to keep cross-version and the
(already in our mind for a while) solution to **vendor unittest** will
help to simplify most of our test code base.
This commit modified the vendored unittest files to simplify them as
much as possible to suite our needs.
Since the runner is still the unittest one, we need to inherit from
unittest.Testcase in order to have the right type.
This also means that we still have access to all TestCase methods
without overriding them all. This is convenient for assertion methods as
an example but the initial idea is to vendor our own version of TestCase
to avoid having trouble to adapte our miscommunications to future python
versions. A trade-off must be done to chose what should remain in our
code base. The idea is to keep logic closely linked to our changes in
our code base, mainly around the run method, but also addClassCleanup
wich need to be vendored for python 3.7, but assertions methods are
independent. Any logic can be moved fom unittest to our
vendored version in the future if needed.
X-original-commit: 9a5d1ea54be49e4cc8208c33e76a6bbd2414d5d0
Part-of: odoo/odoo#113850
Consider field F in module X with parameter translate=False, and F is
overridden in module Y with parameter translate=True. During the
upgrade of module X, module Y hasn't been loaded yet, and the ORM
considers the field to be `translate=False`. Therefore it converts its
database column from type jsonb to varchar, and accidentally drops
non-en_US values:
{"en_US": "English value", "fr_FR": "French value"} (jsonb)
-> 'English value' (varchar)
As a result, translations are lost after upgrade.
This commit fixes the bug by checking whether the field is translated in
database and patches the field accordingly when loading the registry.
This avoids the ORM considering the field as non-translated while
upgrading modules.
In order to "force" translated fields to become non-translated ones, at
the end of the loading process the patch above is discarded, and fields
are checked again. We then adapt the schema of models that have such
fields. This extra step handles the uninstallation of modules like
module Y in the example above.
The patching of the fields has one potential issue. While upgrading
module X, field F is patched with translate=True. If module Y actually
overrides F with translate=xml_translate or so, this may cause the
behavior of the upgrade to be slightly incorrect. Because of the
complexity, we have chosen to not support this case.
closesodoo/odoo#112223
X-original-commit: 1e6e482a9763baa5fae70d9b6676d16a46458f4a
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The class is a simple extension of dict, and adds an explicit attribute
for the content of the root node of a tree. It makes the code much more
readable with very small performance overhead.
closesodoo/odoo#111946
X-original-commit: 8a01d74e5f7d035cfd05bc02596f075270e0fcf8
Signed-off-by: Raphael Collet <rco@odoo.com>
This patch optimizes the way field trigger trees are computed. Overall,
the resulting trigger trees are mostly identical, but they can now be
determined one by one, which enables an on-demand approach and partial
cache.
Before this patch, getting the first trigger tree proceeded as follows:
- resolve the dependencies of all fields;
- compute the transitive closure of the dependencies of all fields;
- store the transitive closure above as field triggers for all fields
in a cache.
After this patch, getting the first trigger tree proceeded as follows:
- resolve the dependencies of all fields;
- cache them as direct triggers for all fields;
- compute one trigger tree as the transitive closure of the field's
triggers, and cache it.
This optimization is quite effective during the installation of modules,
and is even more effective when the number of fields is large. For
instance, a complete installation with all community modules is now 25%
faster. For a complete installation with all enterprise modules, the
installation time is even 30% less! A medium installation is about 16%
less time.
The optimization also speeds up the first request on a new Odoo worker,
since the minimum time for computing a handful of trigger trees is much
smaller than before. We have measured times for a first request going
from 1.6 seconds to 1 second for posting a message.
We have observed slight differences in trigger trees, but they occur in
places where the tree has redundant branches, in particular with fields
having recursive dependencies. It therefore makes no difference in what
is being triggered or invalidated.
X-original-commit: 68f786d494c3c73a085cd919b348c019f77794e7
Part-of: odoo/odoo#111946
Those APIs are aimed at hiding the implementation of trigger trees,
dependent fields and fields modifying relations. Explicit APIs simplify
the profiling of executions and comparison of implementations for
building trigger trees.
X-original-commit: d12b9270375634e738f5288d0c523ac0eec2fa18
Part-of: odoo/odoo#111946
There are two issues with the index naming convention used by the ORM:
Problem 1: it is possible to have naming conflict for indexes. For
instance, the name 'slide_channel_tag_group_sequence_index' is used for
both fields slide.channel.tag.group.sequence and
slide.channel.tag.group_sequence. Only the first index will be created.
Solution 1: we separate the model and field names with a double
underscore instead of a single one, which is the same strategy as with
LEFT JOIN aliases. This is correct because model names don't contain
such double underscores or underscores as prefix or suffix (it is not
forbiden but model name should follow the 'dot notation'.)
Problem 2: index names can be longer than 63 chars, but PostgreSQL
silently truncates it. This doesn't actually break anything (PostgreSQL
also truncates values when we check the existence of indexes) but it can
lead to using the same name twice. There is hopefully not any example
in our code.
Solution 2: if the name is too large, we truncate it and pad it with a
hash of the complete name to match 63 characters, which is also the
strategy used for LEFT JOIN aliases.
task-2984730
closesodoo/odoo#100736
Related: odoo/upgrade#3957
Signed-off-by: Raphael Collet <rco@odoo.com>
The cache currectly fails to correctly invalidate relational fields that depend
on a non-relational field. Two passes of invalidation are done, to reflect
dependencies on both the old and the new written values. In the first pass
only relational fields are considered, as explained in the comments:
> It is best explained with a simple example: consider two sales orders SO1 and
SO2. The computed total amount on sales orders indirectly depends on the
many2one field 'order_id' linking lines to their sales order. Now consider the
following code:
>
> line = so1.line_ids[0] # pick a line from SO1
> line.order_id = so2 # move the line to SO2
>
> In this situation, the total amount must be recomputed on *both* sales order:
the line's order before the modification, and the line's order after the
modification.
The written values can be seen as the roots of a dependency forest (a
collection of dependency trees). Before this commit all non-relational roots
and their corresponding trees were filtered out during the first pass. However,
this approach is wrong, as relational fields can also depend on non-relational
fields. Instead, the complete dependency forest has to be traversed, skipping
invalidation for non-relational fields during the first pass.
The test that was previously included accidentally succeeded because of a
separate and unrelated bug in the orm domain parser: in certain one2many or
many2many leafs the domain parser would not take into consideration the domain
included in the definition of the field. As a result, the test still passed
by accident, because the records that no longer matched the domain after the
write were still invalidated during the second pass.
The problem can clearly be demonstrated, however, when the dependency is
generated by a compute function.
closesodoo/odoo#101038
X-original-commit: d4a5827b42d80f0f830455dcd2056701eb09aed1
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
When a module extends model 'base', the database schemas of all models are
checked, including indexes. And a log message appears for "unexpected index
mail_message_subtype_id_index on table mail_message_subtype". The index indeed
exists, but not for the table mentioned in the message. The ORM actually makes
a confusion between:
- the index mail_message_subtype_id_index for subtype_id on table mail_message
- the index mail_message_subtype_id_index for id on table mail_message_subtype
The fix consists in logging the message about the unexpected index only if the
index is on the expected table.
closesodoo/odoo#100297
X-original-commit: fe8d0024897208e876f097564d50e6795df2c1d3
Signed-off-by: Raphael Collet <rco@odoo.com>
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
We added trigram index for char fields since https://github.com/odoo/odoo/pull/83015.
But if `unaccent` is installed in the database (and isn't force to
`False` on field), these new trigram indexes are pointless and cost a
lot for nothing (almost nothing, it still can be used for equality
operator but in this case a btree will be far more efficient).
The simple way to fix it is to add `unaccent(<column>)` in the index
trigram definition, but unfortunately `unaccent` is not immutable and
may therefore not be indexed. In order to make `unaccent` indexable, we
must declare it as immutable (see
https://stackoverflow.com/questions/11005036/does-postgresql-support-accent-insensitive-collations/11007216#11007216
for more information and how to do that).
With this patch, trigram indexes are created with `unaccent(<column>)`
if the function `unaccent` is available in the database, and for the
fields that are not declared with `unaccent=False`. Moreover, we issue
a warning when `unaccent` is available but is not immutable, in which
case most trigram indexes will be useless.
odoo/upgrade#3736
task-2551518
closesodoo/odoo#95943
Signed-off-by: Rémy Voet <ryv@odoo.com>
Because method _read() no longer updates existing values in memory,
those values in memory must be consistent with the database.
On the other hand, if pending updates are not performed on the database
before fetching values, it means that the corresponding database values
cannot be put in cache. This implies that one cannot empty the cache
without flushing the corresponding fields.
In order to avoid mistakes, flush automatically before invalidating the
cache. This makes the invalidation methods safe by default, and avoids
cargo-culting which would systematically associate invalidation to
flushing, which may eventually be less performant.
Part-of: odoo/odoo#66938
This provides a new API for those operations, in order to make the
distinction between the use cases more explicit. The former API was
using obscure parameter combinations to correspond to various cases.
In the summary below, `fnames` is an iterable of field names. If the
parameter is not given, it means "all fields" in the given context.
Note that method recompute() is now mostly private, as it should not be
used in business code.
# process pending computations and updates
records.env.flush_all() # all fields of all models
records.flush_model(fnames) # the fields of all records of the model
records.flush_recordset(fnames) # the fields of the given records
# process pending computations, became non-public methods
records.env._recompute_all() # all fields of all models
records._recompute_model(fnames) # the fields of all records of the model
records._recompute_recordset(fnames) # the fields of the given records
# invalidate the cache of fields
records.env.invalidate_all() # all fields of all models
records.invalidate_model(fnames) # the fields of all records of the model
records.invalidate_recordset(fnames) # the fields of the given records
Part-of: odoo/odoo#87527
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
=> `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
=> `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.
Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.
Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.
PR: odoo#78857
Task: 2571224
The possible index names have been renamed "btree", "btree_not_null"
(instead of "not null") and "trigram" (instead of "gin").
Task 2742526
Part-of: odoo/odoo#83274
It's not entirely clear whether `@synchronized` is even useful, but
keep it for now. `locked` is just the default instance of
`@synchronised`.
- rewrite `@synchronized` using `decorator`, don't fold everything
into a single call as there's a potential for parametric conflict
- remove the independent `locked` in `sql_db.py`
- convert `lru` to `locked`
- move `Registry` over to `locked` where applicable
closesodoo/odoo#82718
Signed-off-by: Raphael Collet <rco@odoo.com>
Three supported types:
- btree (default for index=True)
- btree not null (when >90% of the data are null)
- gin trigram search (for char fields)
Review of indexes on all objects.
closesodoo/odoo#83015
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
The cursor features a "transaction" object to manage application-
specific data in relation with cursor operations. In its initial
implementation, the transaction object was created by the registry.
This created a requirement: in order to be used in environments, a
cursor had to be created by registry.cursor().
We now remove that unnecessary technical requirement by making
environments create the transaction object on demand.
closesodoo/odoo#80644
X-original-commit: e902713648bca329e6821859462b2128aad62c09
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Until now, stored compute fields were computed after database insertion.
This meant that required fields should not be computed, for instance,
unless some hackish code was added to make it work. Another trick was
to provide some default value, but this actually prevents the field for
being computed after insertion.
This commit provides a field parameter to specify that the field should
be precomputed: adding precompute=True on the field definition force the
method create() to compute its value before inserting the new record in
the database. For the reason explained below, precomputing fields is
not always correct, and therefore the default remains to not precompute
a field.
Some stored fields must be computed after insertion, for instance:
* statistics fields computed with search/read_group/...
* fields referencing the current record (res.partner.commercial_partner_id)
* fields referencing another record that does not exist yet (think about
records created by one2many fields)
* fields depending on the create_date/write_date/create_uid/write_uid
Those fields shouldn't be defined with precompute=True, which triggers
their computation post record creation. This is why, by safety, we
consider the default behavior to be precompute=False.
closesodoo/odoo#80449
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Yannick Tivisse <yti@odoo.com>
When a submodule overrides an old UNIQUE constraint with a new one,
records in that module may respect the new one and not the old one.
As the old module is updated, it would fail giving an ERROR message,
and therefore blocking the Odoo.sh deployment pipeline.
i.e. website_sale (old): res_users_login_key -> ['login', 'website_id']
base (new): res_users_login_key -> ['login']
With this patch, the error level is changed from ERROR to WARNING,
leaving Odoo.sh free to continue the build deployment, as the error
was not a blocking one.
closesodoo/odoo#79199
X-original-commit: 8ed3641f81502d3a7e9c9bb93771c6f5601a31a9
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: Paolo Gatti (pgi) <pgi@odoo.com>
The registry attributes 'registry_invalidated' and 'cache_invalidated'
are used to flag that the current request has modified the registry or
invalidated the ormcache, respectively. This provides a simple yet
efficient way to signal registry changes or cache invalidations to other
workers.
However, those flags were not meant to be used with multi-threaded
workers. For instance, a thread may signal registry changes that are
actually made by another thread. It can also happen that a thread
changes the registry, which makes another thread crash (like a thread
modifying a dict while another one iterates over it), and the latter
will reset the registry to its original state because it misinterprets
the registry changes as its own changes.
The situation can even get worse, making threads crash in cascade and
eventually leaving the registry in an inconsistent state. When this
happens, the worker is broken and has to be manually restarted.
The fix consists in making those flags thread-specific. This does not
prevent thread crashing because of concurrent changes, but at least it
avoids leaving the worker in a broken state.
closesodoo/odoo#77273
X-original-commit: 28adbfa5a9df9b7754529d45188c0eedeffdf783
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Commit cd12293386 introduced an
optimization in the way that models and their inheritances are loaded,
with this commit we defer the setting of each model class' bases
from the _build_model method to the _prepare_setup method.
This has the advantage of setting any given model class' `__bases__`
attribute only once, but it also means that when _add_manual_models is
called, the __bases__ for ir.model are not yet set (only the default
implementation exists), therefore any module overrides to ir.model do
not take effect when creating the custom models.
This meant that if one creates a custom model with chatter support (i.e.
custom ir.model behaviour implemented in mail) and one restarted the
server, the registry would not properly setup ir.model before creating
the custom model (yielding warnings about tracking and such not being
valid fields) and when creating a record of the custom model, the
registry would crash.
With this commit, ir.model's _prepare_setup is explicitly called before
_add_manual_models to ensure that all overrides to ir.model are taken
into account.
closesodoo/odoo#76325
X-original-commit: f3afb23cdf21f395855771a037c721e977dc93c8
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Refactor the Environments object into a Transaction object, which is
bound to one cursor, and is no longer shared among several cursors.
The following methods/properties have been changed:
- Environment.envs no longer works (because of the design change);
- Environment.manage() is deprecated (no longer useful);
- Environment.reset() is now an instance method;
- env.clear_upon_failure() is deprecated in favor of cr.savepoint().
closesodoo/odoo#75598
Related: odoo/enterprise#20451
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
Before this commit searched terms were the ones entered by the end-user.
After this commit if the searched term is not found then the search is
performed on a resembling term.
The similarity is obtained from a Levenshtein distance combined with
ratios of common and different letters.
By default the dictionary against which the search term is matched is
based on the content of the search fields containing a word that starts
with the first letter of the search term.
If the `pg_trgm` Postgresql extension is installed the dictionary is
built from matches using the `<%%` operator (similar to the
`word_similarity` function).
Several approaches were benchmarked during development, those results
are available through the task record.
task-2379555
https://github.com/odoo/odoo/pull/65871
Part-of: odoo/odoo#65871
When computing the foreign key name,
`check_foreign_keys` didn't take into account the limit of 63 characters
for constraint names.
Because of this, some constraints were dropped and recreated
over and over while they were correct, during install and upgrades.
For instance, when installing `base`
when adding the foreign key for which the name was computed
`base_partner_merge_automatic_wizard_res_partner_rel_base_partner_merge_automatic_wizard_id_fkey`
Postgresql created the constraint under the name
`base_partner_merge_automatic__base_partner_merge_automatic_fkey`
and therefore, as the name did not match,
the constraint was dropped and re-created.
closesodoo/odoo#72234
X-original-commit: 43a4738ebf8a74a389b99f8f58330b3044beaa0c
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
This makes the union of many dicts into a single dict.
On a registry with 296 modules, this saves 300 kilobytes of memory,
which is about 3% of the registry's memory footprint.
When a registry is loading, if a custom field cannot be set up, it will
be removed from the dependencies of the field 'display_name', which do
not exist yet on the registry. This patch makes sure that the dict
registry.field_depends always exists, even as an empty dict.
closesodoo/odoo#70373
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
The attributes field.depends and field.depends_context are problematic
for sharing fields across registries, because they depend on the model's
registry class, which may vary from one registry to another. In order
to make computed fields shareable, we have to move those values away
from fields.
For the same reason, field.recursive should not be inferred, because its
value may depend on the registry, although it is generally not the case.
Moreover, the flag recursive=True is set on a field when field triggers
are determined (on the registry). A compute method may be called before
the flag is set (if no update has been done yet), and that can lead to
incorrect computations.
This happened in test TestUsers2.test_reified_groups in module 'base'.
The user groups view was apparently determined without the flag being
set, and the view depends on the recursive field 'trans_implied_ids',
which was not correctly computed.
We thus force developers to be explicit about recursive computed fields.
The code now logs a warning when the flag is not set up properly.
CRM install query count can vary from one execution to another, leading
to difficulties when analysing performances evolution.
The main reason for this is that some compute methods were called in
different order. Even if compute order shouldn't have any effect on the
final result, making it well defined will help finding other causes of
non-determinism.
The initial observation was that sorting Environment.fields_to_compute
leads to a fixed number of query when installing crm.
The main cause of non-determinisim is the usage of `set` impacting
Field.compute_value and BaseModel._modified_triggers.
Transforming all these `set` to `OrderedSet` solves the problem.
The query count is now deterministic when installing a database from
scratch, but not when updating a database with -i crm.
OrderedSet is also slightly optimised by using a dict instead of an
closesodoo/odoo#68692
Ordereddict: dict order is deterministic since python3.6
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When an index has name 'tablename_fieldname_index', this index is
dropped if the field has index=False. This lead to dropping a
user-created index or dropping/recreating the index when index=True is
set in a dependent module. Some info is logged instead.
closesodoo/odoo#67602
X-original-commit: 6a87318df296a11e3300723ca0c8b49c0dcf5fa7
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When table name is long enough, the foreign key constraint name computed
by odoo and the one by PostgreSQL are different, for the same table,
column and definition, are different. As an example,
base_partner_merge_automatic_wizard_res_partner_rel(partner_id) foreign
key is named
base_partner_merge_automatic_wizard_res_partner_rel_res_partner_id_fkey
in Odoo while it's
base_partner_merge_automatic_wizard_res_par_res_partner_id_fkey in
PostgreSQL. This difference trigger a useless foreign key drop/create.
closesodoo/odoo#67601
X-original-commit: 353b415c74da057d5e81cce4d309bebb5f0f2927
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This patch complements the previous one by ensuring that any
`cache_invalidated=True` flag inherited from the master process
(pre-fork mode) cannot trigger a cache clear.
This could occur when the first request is served, because the flag was
never clear in the master process, which never server any request.
At the end of `check_signaling()`, the local cache has either been
cleared because a (real) increment of the cache sequence was detected,
or it is considered still valid. The final state of the
`cache_invalidated` flag should reflect this, by being `False`.
closesodoo/odoo#65346
X-original-commit: 3585c2c38955ece4292da77de7173fb31a6992ba
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
During loading, the registry clears all `ormcache` data multiple
times, in order to ensure consistency with the newly loaded module
data.
Since 083c70bbb6, this was done by
calling `self.clear_caches()`, with the side-effect
of signalling to all other worker processes that the cache
*needs* to be invalidated, which is actually untrue.
If the other workers have any reason to reload their own registries,
they will also clear their own cache in the process - there is no
need to forcefully invalidate it globally.
One could think that combining the pre-fork mode with the `-d <db>`
parameter would mitigate this issue, by making all workers inherit from
a fully loaded registry, In reality it doesn't work, because they also
inherit from the `cache_invalidated=True` flag, that was never cleared
in the master process. So despite having a fully loaded registry, the
newly forked workers will signal a cache invalidation upon serving
their first request.
Further, in a multi-tenant setup with large numbers of databases,
registries may be recycled and loaded much more frequently than
new workers are starting, due to the limited registry LRU, amplifying
this effect a bit.
~~
This patch directly clears the cache LRU without going through
`clear_cache()`, avoiding setting the `cache_invalidated` flag of the
registry, and thus not signalling to other workers.
This is similar to what was being done before 083c70bbb6,
where the LRU was dropped like all other lazy properties.
X-original-commit: 87aef4e3a36d92462454f51960abf7215c5ab7f1
Currently in 13.0 when creating a new model through studio and convert
the x_name field to a computed fields with a dependency set on a
custom field from a native model, user get a Key_error when upgrading
or installing a new module
due to the custom field with an invalid depends raise error through a
transitive dependency.
The loading of the registry completely fails,
because the loading of the field custom_field raises a `KeyError` exception
in `def transitive_dependencies` @ `dependencies[field]`
it happens here because the custom_field was skipped at
`dependencies[field] = set(field.resolve_depends(model))`
task - 2366502
closesodoo/odoo#61663
X-original-commit: 92f6908bae4a6678f76a3ce9e29ac3907803ceb2
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this patch, adding a field on a custom model discards all
automated actions on that model. The explanation is relatively simple.
When models are set up in the registry, the classes of custom models are
dropped then recreated. Given that automated actions are implemented as
monkey-patches on model classes, the setup of models simply loses those
monkey-patches, which explains why they stop working on custom models.
The fix introduces an `_unregister_hook()` method, that is expected to
clean up what has been done in `_register_hook()`. When the registry is
ready (i.e., not being loaded), the setup of models first invokes
`_unregister_hook()` on models, proceeds with the setup, and finally
invokes `_register_hook()` to reinstall the hooks.
OPW 2362308
closesodoo/odoo#60833
X-original-commit: 67152bf82da2674179297d32e4cec9dd534fa0c9
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
By adding a parameter `_table_query` on the Model, we can now have views
that depend on the context. The query is used instead of the table name
in ORM operations. This allows to pre-compute some values to improve
performance, instead of storing context values in the database.
Co-authored-by: william-andre <wan@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
That's a not-very-useful subset of OdooTestResult, so:
* make results merge-able (aka add ability to update a result with the
contents of another)
* remove support for test data files, and transmission of the
assertion report thing through the data-files loading
* replace "legitimate" uses of assertion report by test result
* have run_unit_tests manipulate and return a result instead of weird
flags & ternaries
Before this, invalidations to the UID cache is not synchronised
between workers because it's an ad-hoc solution (so a user changing
their password or an admin disabling a user would only lock out an
attacker currently using the API of one of possibly several
workers). Shift the entire thing to ormcache which already has proper
support for synchronising cache invalidation between workers.
Also simplify the cache invalidation mess in Users.write because the
caches have been unified into a single registry-level LRU, so the
half-dozen cache clears on specific ormcached methods & models is
pretty much the same as repeatedly calling clear_caches on the current
model.
**However** registry.cache is trivially accessible from server actions
and safe_eval as long as they provide access to a model (through
`model.pool.cache`). Which is common, and an issue given we're very
much putting sensible data in there.
Fix this by renaming `Registry.cache` to `Registry.__cache`, this
requires few editions and mangled names are not accessible from
safe_eval contexts.
The alternative would have been to add more bespoke handling of the
uid cache to hook it into the cache invalidation propagation
machinery.
After discussion with (@)odony, fixing LRU access and using that seems
cleaner and less error-prone.
Note on lazy_property
=====================
Make Registry.cache / Registry.__cache into a regular attribute: the
overhead of the LRU is not that high (compared to that of the registry
itself), it's rare that we *don't* need it, and it's assumed to be a
persisted attribute (it's not just a cache) so making it a normal
attribute seems fine; and lazy_property doesn't work for mangled
names: the name of the property is mangled using the name of the
definition class, but the name of the symbol (fget) is not mangled so
lazy_property would set the __cache attribute but then Python would
lookup _Registry__cache, creating a new cache every access.
And we can't (always) mangle things correctly on `__get__(obj,
owner)`: `owner` is just `type(obj)`, meaning in the case of
inheritance the type we get is the type through which the property is
accessed rather than the one it's defined on. So it would work in the
cases where no inheritance is involved (such as Registry.__cache) but
not in general (lest we want to play around walking the MRO ourselves
to find the definition source, which doesn't seem worth it).
lazy_property *could* be made to work properly on Python 3.6+: the
descriptor protocol gains `__set_name__(name, owner)`, which is called
with the properly mangled name — and with the definition class to boot
(though there might still be issues when overriding lazy properties as
the override will be mangled & named differently... or maybe that's a
feature?). However we're still supporting 3.5 at this point, AFAIK, so
that's not an option. Plus it feels unnecessary / not very useful.
However add an assertion to `lazy_property` so it signals when we try
to use it on a mangled method (as otherwise it kinda sorta work in the
sense that the property / object is accessible but is in effect a
slower way to write a regular property).
This is a followup of commit bc2bb5e03c2b32d4ee1b0597ea5889c17d2b0e0e
When a module is updated, a constraint application may (temporarily)
fail because the existing data does not respect the constraint, this is
OK and can be fixed through hooks/migration scripts and was handled by
the aforementioned commit.
However when updating multiple modules, it is possible that an
inheriting module will try to re-apply the failed constraint and
succeed, if that is the case, when processing the `post_constraints` an
already-existing constraint will be applied and raise an error.
To fix this, a check is made before trying to apply the constraint, to
verify that it is not already in _constraint_queue, if it is not, then
we may attempt to apply it, if it is in the queue, then we may safely
ignore it as it will be applied further down the registry cycle.
closesodoo/odoo#55725
X-original-commit: 5225b9ce5178302af05b63029fb184e27b781815
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Achraf (abz)