Suppose a user who has CRM module and wants to import some leads
thanks to this CSV file:
```csv
name,recurring_plan
Coca-Cola,Plan01
SAP,Plan02
```
And, because the recurring plans are not yet created on his database,
he enables the option "Create new values" for that field. An error
will occur and here is the only info the user will have:
> current transaction is aborted, commands ignored until end of
> transaction block
When importing, we convert the data encoded by the user ([1]). To do
so, we convert the provided values, field by field ([2]). Since
`recurring_plan` is a `many2one` field, we go (through
`_str_to_many2one`) in `db_id_for`. In this method, we `name_search`
the record and then, if it does not exist and if the feature is
enabled, we `name_create` it ([3]).
Back to the above use case. We start with the first line and there is
not any recurring plan called "Plan01" so we try to create it. But
here is the issue: we only have a name to create the record although
there is another required field: `number_of_months`. Therefore, it
leads to a `NotNullViolation` error. This error is caught (see [3])
and, later in the same method, we will raise a `ValueError`. This
error will be caught by one of the except in [2]: we will save the
error and will then continue with the convertion of next values.
However, because of the `NotNullViolation`, the current SQL
transaction is broken. As a result, while trying to convert the
second line of the file, we will `name_search` "Plan02" and it will
simply lead to a `InFailedSqlTransaction`
[1]
https://github.com/odoo/odoo/blob/f3d7fdce608f692ecb08498ee158edf9dfbced5e/odoo/models.py#L1170-L1182
[2]
https://github.com/odoo/odoo/blob/b80d4294a000e748677a3a0c1849140bf46ddbe8/odoo/addons/base/models/ir_fields.py#L117-L118
[3]
https://github.com/odoo/odoo/blob/b80d4294a000e748677a3a0c1849140bf46ddbe8/odoo/addons/base/models/ir_fields.py#L472-L475
sentry-3969379125
closesodoo/odoo#132897
X-original-commit: 28373b9d261a48154b233fbbd895880680b0aed0
Signed-off-by: Adrien Widart (awt) <awt@odoo.com>
Rationale
=========
Since v8, the `display_name` field is present on all models. By default,
`display_name` uses `name_get` which has pretty much the same purpose
(return record name used by the web client). Gradually, many (backend)
developers (and the ORM: https://github.com/odoo/odoo/commit/6da1c3ac4c036eac289597602976538e243cb939)
started using `display_name` (more convenient than
`record.name_get()[0][1]`) but it still had the `name_get` override.
It becomes more complex than necessary and poeple start to misunderstand
the two (and sometimes override both, leading to inconstiencies between
`display_name`/`name_get`).
To simplify the ORM and the API, we decided to keep only one of them,
the `display_name` field:
- It is much more convenient from a backend point of view
(`record.name_get()[0][1]` vs `record.display_name`)
- It is cached during the same transaction (and invalidated if
its dependencies change)
- It can be overridden like any other compute field (override
`_compute_display_name` with any extra dependencies)
- `name_get` is replaced by `read(['display_name'])`
(API perceptive), which can actually be more efficient
(if `display_name`'s depends are correct, the ORM will only fetch the
fields it needs instead of every prefetchable field)
Changes
=======
- Deprecates `name_get` for the v17 and based the method on
`display_name` (the opposite of before)
- Converts all usage of `name_get`
- Overrides of `name_get` are now overrides of `_compute_display_name`
- For `res.partner`, rename the field store `display_name` into
`complete_name` because `display_name` context-dependent and it makes
no sense to have a compute store that is context-dependent.
- Previously, it was possible to return multiple names for the same
record with `name_get`, but it was tricky and most of the usage of
this `name_get` didn't take this into account. The only example of
this is the `name_get` of `product.product`
(now use `", ".join(<names>)`).
Part-of: odoo/odoo#122085
This API is much more sensible for making subqueries. Specifically, one
can generate a subquery without the clauses LIMIT and ORDER BY.
Part-of: odoo/odoo#112126
This simplifies the use of subqueries by avoiding some costly default
order on the model or the idiotic order='id'. Method _flush_search()
has been adapted accordingly.
Part-of: odoo/odoo#112126
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The import logging (ish) assumes that if an exception has at least 2
args the second arg is metadata added by the callee.
As it turns out, `UnicodeEncodeError` has *five* arguments, none of
which is added by us. So if encoding something fails during the
process (e.g. because the file contains a lone surrogate, which leads
to the database insert failing when psycopg2 tries to encode the query
to UTF8), then the `_log` function itself will fail, yielding a very
unhelpful error of:
dictionary update sequence element #0 has length 1; 2 is required
(because we tried to update a dict using a string).
This issue occurs only during *field conversion* and most fields have
no need to interact with the database (so don't need to encode the
value, which is what fails), however it is a problem when the invalid
string is used as a record name to look for (e.g. an m2o).
Further improve the experience by converting the UnicodeEncodeError to
a ValueError using the stringified UEE: `_log` assumes the first
argument to the exception is an error message of some sort, but for
UnicodeError subclasses it's just the encoding involved in the
error (here `utf-8`), which doesn't really serve as an error message.
Stringifying the exception generates a complete error message which is
quite a bit more helpful.
Specific update notes:
* avoid modifying the exception in-place, doesn't seem useful
* not sure why `field_name` was added as part of the augmentation
rather than up-front when `record` is created
Issue 2480064
closesodoo/odoo#72517
X-original-commit: 6c3c500929cd463cd3a1749f4ede8f3a8afd5748
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This provides a better API to search for records by name without the
formatting part (`display_name`).
This also simplifies all the overridings of `_name_search` that no
longer need to call `name_get()`. The call to `name_get()` is done in
method `name_search` in a generic way.
Before this change, we create an xid for every parent of a record
being imported, regardless of whether it already has an xid, or if
it's being created implicitly through the child.
This generates unnecessary extra xids on pre-existing objects
e.g. update 5 product variants -> the product gets 5 new xids despite
already having one.
We should *only* set a xid on parent records which are being
implicitly created by the creation of a child with a specified
xid. That is, we should never set a xid on the parent if it exists
before the child is created.
Update _process_end to try and see if "non-loaded" xids correspond to
an automatically generated "parent" xid: we're still setting a xid on
implicitly created records (if the child is created with a xid) so
they're properly removed if e.g. the module is uninstalled, but
because we're only doing so at creation these xids will not be visited
during update and _process_end will try to delete them.
A special case can be added to check that "unknown" parent xids don't
have children which _inherit them, in which case we want to protect them.
Task 2251039
closesodoo/odoo#53283
Related: odoo/enterprise#12023
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this change, if an error occurs while trying to match / import
a sub-field e.g. order_line/product_id, only the name of the top-level
field is showing when displaying the error e.g. "Order Line", which
lacks precision and makes understanding and fixing the issue more
complicated.
After this change, the entire field path should be displayed, e.g.
"Order Line / Product" in the example above.
Task-1906700
closesodoo/odoo#33031
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Co-authored-by: Mohammed Shekha <msh@odoo.com>
To avoid slowing down search like "{relational_field}" contains "{value}"
we always need to return the lazy name_get() for each [_]name_search method
closesodoo/odoo#36735
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Multi is the default api for methods, it is not necessary to explicitly
decorate methods with it, adds clutter and most people use it because
they see that the rest of the code uses it.
Done with `find . -type f -name '*.py' | xargs sed -i '/@api.multi/d'`
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
Purpose of this commit is to give description more "business oriented"
because those descriptions appears in Odoo Studio which is supposed to be used by end users, not only by developers.
Related Task ID : 37311
Adds a checkbox to import columns (in debug mode) allowing a user to
create records M2O and M2M records not found (via name_search).
Task ID: 1850633
* uses a context key to avoid altering basically all the import callstack
* attempted to lift the creation in the `_str_to_*` functions and create
m2m via commands, but that doesn't really work out
Purpose
=======
The method _name_search and _search add support to search as another user.
All the overrides of name_search and search redefine the behavior of the search.
This bring inconsistencies as the result of a call to _name_search and name_search
could differ on certain modules, which is not acceptable.
Specification
=============
Example:
~~~~~~~~
If the name_search method is overridden to search also on the driver name,
then calling name_search with a label 'JF' will return all the cars with a name containing
'JF' or all the cars with a driver name containing 'JF'. Let's say that we have a ir.rule
preventing a user to read the cars of another company. Then the call to name search only returns
the cars satisfying the previous condition AND belonging to his company.
Now, we want to overpass this constraint. We call _name_search with the attribute name_get_uid=1.
Then the call to _name_search returns all the cars from all the companies with a name like 'JF',
but nothing is done about the driver_name.
Example of wrong search redefinition on a model
-----------------------------------------------
@api.model
def name_search(self, name, args=None, operator='ilike', limit=100):
domain = args or []
domain = expression.AND([domain, [('name', 'ilike', name)]])
partners = self.env['res.partner'].search([('name', operator, name)])
if partners and name:
domain = expression.OR([domain, ['|', ('driver_id', 'in', partners.ids), ('driver_id', '=', False)]])
rec = self.search(domain, limit=limit)
return rec.name_get()
Example of correct search redefinition on a model
-------------------------------------------------
@api.model
def _name_search(self, name, args=None, operator='ilike', limit=100, name_get_uid=None):
domain = args or []
domain = expression.AND([domain, [('name', operator, name)]])
partner_ids = self.env['res.partner']._search([('name', operator, name)], access_rights_uid=name_get_uid)
if partner_ids:
domain = expression.OR([domain, ['|', ('driver_id', 'in', partner_ids), ('driver_id', '=', False)]])
rec = self._search(domain, limit=limit, access_rights_uid=name_get_uid)
return self.browse(rec).name_get()
The same logic should be applied on the overrides of the search method.
* remove references to basestring & unicode (use relevant pycompat
helpers)
* remove some str calls (either entirely or replaced by relevant
helper, either text or native)
* use better API to avoid unnecessary conversions
* remove some XML declarations in views
Previously availability of meta-information (failing constraint,
impacted table and column, …) in pg errors (e.g. constraint check
failure) was limited and only available through formatted error
messages, which could be localised (by PG itself) so not did getting
that information require string extraction it was brittle in the face of
localised instances.
Postgres 9.3 adds this meta-information to error diagnostic data
(`PQresultErrorField`), allowing easy programmatic access to it.
Psycopg2 [added a Diagnostic
object](http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.Diagnostics)
around the time pg9.3 was itself released.
Assuming Odoo now requires pg >= 9.3 we can remove the old string
munging extraction for diagnostic.
* funny story, the handling of 23505 didn't actually work because
"duplicate key value violates unique constraint" is a literal part
of the error message, I'd misunderstood "value" as a field name
because my test model's field was called "value". Makes sense too
since you can set UNIQUE constraints on multiple fields (and UNIQUE
indices on expressions)
* for both errors, it should be possible to use the table_name or
constraint metadata to provide messages about sub-model issues (a
constraint on an m2o), but that's not currently handled
Fixes#15323
Adrien Widart (awt)