The `=like`/`like` domain operators are accent-insensitive, but still
case-sensitive. This is not really coherent, since `unaccent` is a best
effort to find records from the client, and it is the same idea behind
being case-insensitive. Also, `=like`/`like` cannot be used from the web
client, and we use them in domain to search from the Python side.
Moreover, adding `unaccent` to `=like` can be very inefficient when
searching for a prefix ('prefix%'). In fact, PostgreSQL can use btree
index to find prefix matches, but because we create dy default btree
index without unaccent (when we put
`index=True/'btree'/'btree_not_null'` on the field), PostgreSQL cannot
use this index.
Part-of: odoo/odoo#136007
Add 'id' in partner ordering, to be sure searching on duplicated partners is
deterministic.
In mail, use standard ordering when searching on users to be deterministic.
As ordering on users is based on name then login which is unique, this is a
safe ordering and can be used as it.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1e6d50a141
Part-of: odoo/odoo#134934
Issues
======
- `_apply_ir_rule` applies `ir.rule` of the current model and also
`ir.rule` from the inherited model (via inherits). But
`check_access_rule` doesn't check the later one.
- `_flush_search` doesn't flush fields coming from the `ir.rule` of
the inherited model (via inherits). Then the filtering done by
`_apply_ir_rule` may be inconsistent with cached values.
Changes
=======
Because of https://github.com/odoo/odoo/blob/6ddcb448612f5d784c8e9ebb90f19077e65be3e1/odoo/osv/expression.py#L1073-L1073,
and https://github.com/odoo/odoo/blob/00e86b1552d1e5541a8dbf9411de5cfdb8990cc4/odoo/fields.py#L2895
leaf like `('<many2one_delegate>', 'any', [<sub-domain>])`,
will be translated in the same way as `_inherits_join_add` does.
We can remove `_inherits_join_add` and its usage in `_apply_ir_rule`
and change `ir.rule._compute_domain` to also return the inherited
(via inherits) `ir.rule` domain (with the new 'any' operator).
Since `_compute_domain` is used by `_apply_ir_rule` and
`_filter_access_rules_python`, everything is consistent.
Also fix `BaseModel._flush_search` to take in account 'any'/'not any'
operators (compulsory in order to flush correctly new domain
from `ir.rule._compute_domain` generated).
Part-of: odoo/odoo#125916
Goal:
* Simplified modifiers to only have one way to define modifiers;
* Remove states attributes on python field;
* Use python expression in view `required`, `readonly`, `invisible`;
* More accurate validation of xml views.
This commit change the syntax to python expression. The next commit
will update/convert all xml views.
Before this commit:
* the `required`, `readonly` and `invisible` attributes can only have
values of `True`, `False`, 1, 0 or a python expression to use the
context;
* the `attrs` attribute define a dict. The key of this dict was
`required`, `readonly` and `invisible` and the values are the domain or
a string representing a domain to be evaluate as python expression.
This python expressions was evaluate by the javascript with view fields
and other contextual values as: context, uid, parent, active_id,
active_ids, active_model, allowed_company_ids, current_company_id.
* the `states` attribute in the view was a comma separated list of the
state. This list was combined with the `invisible` attribute;
* the `invisible` attribute on python field is used as default value;
* the `states` attribute on python field was dictionnary with state as
key and list of tuple. This structure was combined with `readonly` view
attribute.
* After combining, the resulting domains of the different attributes
`required`, `readonly` and `invisible` are evaluated with the values of
the fields. The `invisible` attributes is splitted into two use:
`invisible` and `column_invisible`.
After this commit:
* The attributes `required`, `readonly`, `invisible` and
`column_invisible` define python expression. This python expressions
are evaluate by the javascript with view fields and other contextual
values as: context, uid, parent, active_id, active_ids, active_model,
allowed_company_ids, current_company_id.
The domains can contains contextual value and will be evaluate by the
javascript.
```xml
<field name="field_a" readonly="not context.get('show_a')" attrs="{'readonly': [('field_b', '!=', False), ('field_c', '=', parent.c)]}"/>
<field name="field_b" states="draft"/>
```
will be replaced by
```xml
<field name="field_a" readonly="not context.get('show_a') or field_b and field_c == parent.c"/>
<field name="field_b" invisible="state != 'draft'"/>
```
Some inherited views will be modified differently in order to maintain
the previous behavior:
```xml
<field name="field_a" readonly="not context.get('show_a')" attrs="{'invisible': [('field_b', '!=', False)]}">
```
```xml
<field name="field_a" position="attributes">
<attribute name="attrs">{'readonly': [('field_c', '=', False)], 'invisible': [('field_d', '!=', '3')]}<attribute>
</field>
```
will be replaced by
```xml
<field name="field_a" readonly="not context.get('show_a')" invisible="field_b">
```
```xml
<field name="field_a" position="attributes">
<attribute name="readonly" add="(not field_c)" separator=" or "/>
<attribute name="invisible">field_d != 3<attribute>
</field>
```
Validation:
A stricter control is made on the level of the attributes (modifiers)
and the fields necessary for these. The use of the previous attributes
'attr' and 'states' triggers an error (these no longer exist after the
application of the migration script)
task-2495504
Part-of: odoo/odoo#104741
`filtered_domain` didn't throw an exception when it is called with a
domain containing the new 'any' and 'not any' operators. Fix it.
X-original-commit: 05713a270bfad406daa9dbf739b80cf8c4d065c0
Part-of: odoo/odoo#127750
The `name_search` makes at least 2 SQL requests, one to find the records
and one to read fields needed to compute the `display_name`. With the
default `_compute_display_name`, it only needs to fetch the `_rec_name`
field (if there is one), but the ORM perfetch field mechanism will also
fetch every prefetchable field (see `_fetch_field`).
Then, to avoid running two queries and fetching too many fields, we add
the dependency fields (with `_determine_fields_to_fetch`) to the select
clause of the `Query` returned by `_name_search`.
Part-of: odoo/odoo#122085
Rationale
=========
Since v8, the `display_name` field is present on all models. By default,
`display_name` uses `name_get` which has pretty much the same purpose
(return record name used by the web client). Gradually, many (backend)
developers (and the ORM: https://github.com/odoo/odoo/commit/6da1c3ac4c036eac289597602976538e243cb939)
started using `display_name` (more convenient than
`record.name_get()[0][1]`) but it still had the `name_get` override.
It becomes more complex than necessary and poeple start to misunderstand
the two (and sometimes override both, leading to inconstiencies between
`display_name`/`name_get`).
To simplify the ORM and the API, we decided to keep only one of them,
the `display_name` field:
- It is much more convenient from a backend point of view
(`record.name_get()[0][1]` vs `record.display_name`)
- It is cached during the same transaction (and invalidated if
its dependencies change)
- It can be overridden like any other compute field (override
`_compute_display_name` with any extra dependencies)
- `name_get` is replaced by `read(['display_name'])`
(API perceptive), which can actually be more efficient
(if `display_name`'s depends are correct, the ORM will only fetch the
fields it needs instead of every prefetchable field)
Changes
=======
- Deprecates `name_get` for the v17 and based the method on
`display_name` (the opposite of before)
- Converts all usage of `name_get`
- Overrides of `name_get` are now overrides of `_compute_display_name`
- For `res.partner`, rename the field store `display_name` into
`complete_name` because `display_name` context-dependent and it makes
no sense to have a compute store that is context-dependent.
- Previously, it was possible to return multiple names for the same
record with `name_get`, but it was tricky and most of the usage of
this `name_get` didn't take this into account. The only example of
this is the `name_get` of `product.product`
(now use `", ".join(<names>)`).
Part-of: odoo/odoo#122085
This commit add a new `odoo.osv.expression.prettify_domain` function
that can be used to format a domain as a string with the correct
indentation.
Example:
['&', '|', ('name', 'like', 'Jack'), ('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
Becomes:
['&',
'|',
('name', 'like', 'Jack'),
('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
closesodoo/odoo#118067
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Rationale
---------
Given the following domain:
['|',
('company_id.name', 'like', 'BE'),
('company_id.city', 'like', 'Brussels')]
The ORM would generate the following SQL query:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."name" like 'BE'
) OR "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."email" like 'help@odoo.com'
);
Which is sub-optiomal as the WHERE clause of the two subqueries could be
regrouped inside of a single query like so:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company" WHERE (
"res_company"."name" like 'BE'
OR "res_company"."email" like 'help@odoo.com'
)
);
Postgres-wise, it is faster to execute the latter query than the former
one. This commit is about optimizing the ORM so that it generates
queries where the WHERE clause of compatible subqueries are grouped
together.
Technical solution
------------------
The solution explored by this work is to replace the regular relational
field accesses (over a dotted path) by a new `any` operator that look as
follow:
(relational_field, 'any', domain)
For instance, the above `('company_id.name', 'like', 'BE')` leaf becomes
`('company_id', 'any', [('name', 'like', 'BE')]` using `any`.
Having a domain as right-hand-side allow for the combination of the
domains of same compatible relations:
[('company_id', 'any', ['|',
('name', 'like', 'BE'),
('city', 'like', 'Brussels')])
Having combined domains allows for generating better SQL queries with
minimal changes to the expression parsing algorithm, which can only
translate a single leaf at a time to SQL. With the new `any` operator,
it is still a single leaf but the right-hand-side contains the merged
domain, thus the combined SQL WHERE clause is immediately generated.
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
This commit hads a few tests that report the existing behavior (before
merging subqueries WHERE clauses).
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
Goal: make _search() always return a Query object, in order to make
search_read() in a single query when possible
Adapt the overrides of _search() towards the given goal.
Part-of: odoo/odoo#112126
For non-translated field, when search(['name', 'in', list_right])
The behavior of False and the behavior of None in list_right should be the same.
Translated fields need customized process for 'in' operator
X-original-commit: a9a3c666f08c5612432dba40a69c7cfdc54ba96e
Part-of: odoo/odoo#103031
make searching translated field language dependent
add an extra filter using trigram index to speed up '=' and 'like' search
X-original-commit: 5b6f0a9e5f9d60735e306709a3e7d7ac46a586be
Part-of: odoo/odoo#103031
It is technically nearly impossible to delete a company
when it actually dealed with customers
(emitted invoices, received payments, ...).
Hence, if you want to get rid of a company,
for instance because you closed one of your subsidiaries,
giving the possibility to archive your company,
thanks to an active field, would be the best way to go.
We are hesitating to do a related field to the
`partner_id.active`, but we are a bit afraid
some people archive the partner linked to their company
for other valid reasons than get rid of their subsidiary
(such as avoid changing the address of their company
by mistake through the Contacts app),
while still wanting the company itself to be active.
So, we make it an independant column at the moment,
so we have the actual stored column in case we need it,
and will do a related stored to the partner
later on if we change our mind.
Manual forward-port of #102801closesodoo/odoo#102586
Signed-off-by: Olivier Dony <odo@odoo.com>
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The new feature (search count with limit) introduced by https://github.com/odoo/odoo/pull/95589 lacks of test and code readability:
- Add tests to check the result and query generate
- Increase the readability of SQL/python code.
- Change a little bit the SQL request generate: in the subquery, change `SELECT 1 FROM...` into `SELECT FROM` which avoid extra work in the postgreSQL side.
task-2761165
closesodoo/odoo#95641
Signed-off-by: Raphael Collet <rco@odoo.com>
Revert commit 54cd73a4f4 as it prevents customers from closing
their pos sessions without timeouting.
closesodoo/odoo#88941
X-original-commit: 43165dc3f7a8cc199219b8a32b65eb0ee7b5c2dd
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
Most of the extensions of `_name_get` are very similar and only want to
search for the given string in multiple fields.
A lot of extensions also don't take into account the negative operators.
Some implementations were also really outdated and needlessly
complicated.
closesodoo/odoo#86588
Related: odoo/enterprise#25608
Signed-off-by: Raphael Collet <rco@odoo.com>
This fixes a problem that occurs with conditions on one2many fields,
where most conditions are translated with a subquery like
"model".id IN (
SELECT "comodel"."model_id" FROM "comodel" WHERE ...
)
The issue occurs when the subquery return NULL values, i.e., when the
field "model_id" in the subquery above can be NULL. The fix simply
consists in adding the condition "model_id IS NOT NULL" in the WHERE
clause of the subquery.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
closesodoo/odoo#87285
X-original-commit: 6d680bac0333c6c7ce9f056cba0e1f7e9fa22f9c
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This fixes two problems that occur with conditions on many2many fields:
the subtle bug caused by NULL values that are returned by subqueries,
and the performance of those subqueries.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
Regarding performance, consider a domain like [('tag_ids', '=', False)].
It is currently translated in SQL as
"model".id NOT IN (
SELECT "model_id"
FROM "model_tag_rel"
WHERE "model_id" IS NOT NULL
)
PostgreSQL does not handle well this "NOT IN" expression when the
relation table is large, and uses some very inefficient query plan in
that case. The inefficient query plan is possibly related to the
special handling of NULL values, by the way. This commit changes the
above expression to a "NOT EXISTS" expression (as below), which is
executed with a much more efficient query plan, even with large tables.
NOT EXISTS (
SELECT 1
FROM "model_tag_rel"
WHERE "model_tag_rel"."model_id" = "model".id
)
This change was motivated by a use-case with 2.7M account moves, 11M
account move lines and a many2many relation table with 2.3M lines
(`account_move_line_account_tax_rel`). The query was timing out (15
minutes), and it now takes 15 seconds.
We have replaced all the conditions on many2many fields to use the
relational operators EXISTS and NOT EXISTS.
A test has been added to reflect a case where the many2many table
contains NULL values, like the one for channel_partner in regards to
partner_id and guest_id.
task-2753782
X-original-commit: 334b2aa2aa437d2ed86130f2ed426025f8fe4b86
Part-of: odoo/odoo#87285
Co-authored-by: Didier Debondt <did@odoo.com>
The method filtered_domain() didn't conserve the order of `self`, which
may be confusing, and a subtle source of bugs. It now preserves the
order from `self`.
The processing of `child_of`/`parent_of` making a search with the
default order of the model, which may be slow.
Add documentation.
Part-of: odoo/odoo#83687
The `column_format` of the `Field` class was unused and create useless
noise in the ORM. Then remove it and simplify some flows.
task-2735546
Part-of: odoo/odoo#82727
Issue
-----
Via the field prefetch mechanism, when we need a value of one field
(not in cache of course), the ORM will prefetch all fields
(which has the attribute to `prefetch=True`, the default value of this
attribute is `True`) for all record ids in `_prefetch_ids`.
Then, for each translate fields (where translate is not a callable)
the ORM need to make a `LEFT JOIN` on the `ir_translation` to fetch the
translated value. For big model, it leads to a simple `SELECT` with
several `LEFT JOIN` on ir_translation but each LEFT JOIN have a cost
in the planner time (a small cost in the execution time) of PostgreSQL.
By example, for `product.template` (stock/sale/purchase installed),
there are 6 LEFT JOIN to get all translated fields (5 of this
fields are rarely used).
Proposed solution
-----------------
Deactivate the prefetch by default for all translate fields expect if
this field is the `_rec_name` of the model (which is more likely to
be used).
In the example on the `product.template`:
Without prefetching the translated fields, there is only one LEFT JOIN
(the name, which is translated but is the `_rec_name` of the model).
With the 6 translated fields to fetch, the
query takes 5 ms to plan and 2 ms to execute VS with 1 translate field,
it 1 ms to plan and 1.5 ms to execute.
Side change note
----------------
- All translate of fields of `website.seo.metadata` should be prefetch
to avoid lot of website errors (it is because, website put in cache data
in sudo before reading it without sudo)
- `description` (`mail.message.subtype`), `subject` (`mail.template`),
`body_html` (`mail.template`) should be prefetch to avoid lot of extra
query from mail module.
- `vat_label` (`res.country`) should be prefetch to avoid a extra query
for each website page.
- Increase some queryCount (when it is legit, due to `subtitle` of
`blog_post` or `description` of `event.type.ticket`, etc)
task-2738029
closesodoo/odoo#82896
Signed-off-by: Raphael Collet <rco@odoo.com>
Currently, the domain expression on a many2one field with
negative operator + string values leads to a extra SQL request on the
target model of the many2one which is not optimal and lead to
performance degradation.
Example:
--------
With a simple model A with a one2many (`b_id`) targeting model B (with
a field `b_name` and `_rec_name = b_name`):
The `A.search([('b_id', 'not like', 'OneName')])` will
generate two SQL requests:
- A _name_search on model B and returning ids who match
`('b_name', 'not like', 'OneName')`.
- A other on the model A with the application of the result
of the first one `('b_id', 'in', <returning ids> + [False])`.
Solution:
---------
Avoid the extra SQL request (the first one) and use subquery instead.
We do it by translate the `('b_id', 'in', <returning ids> + [False])`
expression into `['|', ('b_id', 'in', _subquery), ('b_id', '=', False)]`
task-2671476
Part-of: odoo/odoo#78948
This fixes a performance issue: ORDER BY clauses in subqueries can make
the query unexpectedly slow. We should avoid this situation, since
ORM-generated queries have an ORDER BY clause which is not relevant in
the context of a subquery.
The following example was found:
SELECT "pos_payment"."id" AS "id"
FROM "pos_payment"
WHERE ("pos_payment"."pos_order_id" in
(SELECT "pos_order".id
FROM "pos_order"
WHERE ("pos_order"."company_id" in (1))
ORDER BY "pos_order"."id"))
AND "pos_payment".id IN (1285508)
Here are the query plans made by PostgreSQL on this query with and
without the ORDER BY clause. The query time went from 1240ms to
0.402ms, which is 3000 times faster!
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order"."company_id" in (1)) ORDER BY "pos_order"."id" )) AND "pos_payment".id IN (1285508);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Merge Semi Join (cost=2.88..82726.85 rows=1 width=4) (actual time=1239.361..1239.364 rows=1 loops=1)
Merge Cond: (pos_payment.pos_order_id = pos_order.id)
-> Sort (cost=2.46..2.46 rows=1 width=8) (actual time=0.021..0.022 rows=1 loops=1)
Sort Key: pos_payment.pos_order_id
Sort Method: quicksort Memory: 25kB
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.014..0.015 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..66770.53 rows=1282120 width=4) (actual time=0.013..1148.194 rows=1182463 loops=1)
Filter: (company_id = 1)
Planning time: 0.272 ms
Execution time: 1239.396 ms
(11 rows)
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order
"."company_id" in (1)) )) AND "pos_payment".id IN (1285508);
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.85..4.89 rows=1 width=4) (actual time=0.047..0.049 rows=1 loops=1)
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.027..0.028 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..2.45 rows=1 width=4) (actual time=0.018..0.018 rows=1 loops=1)
Index Cond: (id = pos_payment.pos_order_id)
Filter: (company_id = 1)
Planning time: 0.322 ms
Execution time: 0.080 ms
(8 rows)
closesodoo/odoo#77357
X-original-commit: 947c3bd72ee84c3a298e0c9c365340ddf16b307d
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Stanislas Sobieski (sts@odoo.com)
For searches, Odoo uses `unaccent` if it's available. On some
technical fields this is completely unnecessary and precludes the use
of indexes.
This PR provides:
* an opt-out (`unaccent = False`) on `String` and `Text` fields
* a warning if `unaccent` is enabled on *parent_path* fields as their
performance can be rather critical and not using the index is quite
an issue (note: the check that `parent_path` fields have been moved
outside of the check for their existence as we want to check that
the field is declared and correctly configured in all cases,
probably)
Task 2627454
Part-of: odoo/odoo#76436
Automatically activate `group_multi_currency` if there is more than one active currency,
deactivate it if there is only one active currency
retain sale/pos feature of activating group_product_pricelist
adapt tests accordingly
Task 2610735
closesodoo/odoo#74396
Related: odoo/enterprise#20106
Signed-off-by: Laurent Smet <smetl@users.noreply.github.com>
Before this commit, the implementation crashes when the ancestors of a
record contain non-accessible records. This fixes the code to avoid it
to crash.
We also make the semantics of hierarchical searches more consistent in
this case: searching with operators 'child_of' and 'parent_of' returns
the subset of accessible records that satisfy the hierarchy operator.
We actually make it coincide with the results of the search when using
the "_parent_store" optimization.
closesodoo/odoo#73962
X-original-commit: 3e1b960acf3f6a726338476ba9e62e5ef5c84ef9
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Domain terms of the form `('field', 'not in', [...])` generate incorrect
queries for translated fields, example (model `res.country`, field `name`):
```
psycopg2.errors.SyntaxError: syntax error at or near "ARRAY"
LINE 1: ...M "res_country" WHERE "res_country"."name" not in ARRAY['No ...
```
The root cause is that the right part of the term is not converted to
tuple.
Observed during the upgrade request 16639
opw-2525553
closesodoo/odoo#73030
X-original-commit: 7c4db97e21f74a429e56f6cae4d4786ee74f7e22
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The method filtered_domain() is broken for domains with hierarchical
terms ('child_of'/'parent_of').
To see *one* of the ways the implementation is broken, let `A` be a
model with `parent_id` pointing to `A`, and `a1` a record of model `A`
without parent (`a1.parent_id` is `False`), then this fails:
assert a1 in a1.filtered_domain([("parent_id", "child_of", a1.id)])
The reason it fails is that on
https://github.com/odoo/odoo/blob/f5519586d214a9b34ad24683a7f97c47802a3bad/odoo/models.py#L5377-L5380
`data` is empty since `a1` has no parent, thus
https://github.com/odoo/odoo/blob/f5519586d214a9b34ad24683a7f97c47802a3bad/odoo/models.py#L5403-L5404
fails, therefore the result of `filtered_domain` is empty.
Note: the implementation of the hierarchical operators is full of quirks
that are hard to emulate otherwise than by reusing the original code.
As a consequence, the current implementation may be broken in more than
one way.
Let's see another way the implementation is broken: let `B` be a model
without a `parent_id` field and with a `friend_id` field pointing
to `B`, and let `b1` be a record of model `B`. Then
b1.filtered_domain([("friend_id", "child_of", b1.id)])
throws an exception of the form shown below:
ValueError: Invalid field 'parent_id' in leaf "<osv.ExtendedLeaf: ('parent_id', 'child_of', 1) ...
Meanwhile the following code is still valid and returs b1:
B.search([("friend_id", "child_of", b1.id)])
closesodoo/odoo#71237
X-original-commit: e7a5ba95d8b7df5bbf545ef8afe0a1f5d0f70272
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
A domain with a negated thruty leaf raises an exception here
https://github.com/odoo/odoo/blob/835197e48fbbee1c8ac35d52287537fa6f0bdd36/odoo/osv/expression.py#L614
due to
https://github.com/odoo/odoo/blob/835197e48fbbee1c8ac35d52287537fa6f0bdd36/odoo/osv/expression.py#L670
and the fact that `FALSE_LEAF != (1, "!=", 1)` and `TRUE_LEAF != (0, "!=", 1)`
Current wrong behaviour:
```
>>> distribute_not(['!', TRUE_LEAF])
[(1, '!=', 1)]
>>> is_leaf((1, '!=', 1))
False
```
This issue was noticed when adapting domains for fields that have been
removed during migration. Example (extract of) traceback:
```
Traceback (most recent call last):
File "/home/odoo/src/odoo/13.0/odoo/http.py", line 624, in _handle_exception
return super(JsonRequest, self)._handle_exception(exception)
...
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 677, in __init__
self.parse()
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 814, in parse
self.stack = [ExtendedLeaf(leaf, self.root_model) for leaf in self.expression]
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 814, in <listcomp>
self.stack = [ExtendedLeaf(leaf, self.root_model) for leaf in self.expression]
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 562, in __init__
self.check_leaf(internal)
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 618, in check_leaf
raise ValueError("Invalid leaf %s" % str(self.leaf))
ValueError: Invalid leaf (0, '!=', 1)
```
closesodoo/odoo#68665
X-original-commit: e20c23658d15a135e5f35b0e8dec6e3202cecaab
Signed-off-by: Christophe Simonis <chs@odoo.com>
The field that supports inheritance should be auto_join=True. This
guarantees that searching on an inherited field (X) or its explicit
related expression (foo_id.X) gives the same query.
Note that adding auto_join=True on that field does not weakens the
model's security, since searching on the model automatically injects the
parent model's security rules into the query (see _apply_ir_rules).
closesodoo/odoo#66925
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This is a minor issue, but determinism is better than non-determinism.
closesodoo/odoo#62904
X-original-commit: 1b0869554d8755f8fdeb73dd2aa2ae7426522e84
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
The now unique class behaves as the former class `SavepointCase`. It is
now up to the developer to use `setUp` or `setUpClass` for preparing the
tests.
Make the method `_search` return a `Query` object, and make that object
generate a subquery when used on the right-hand side of a condition.
closesodoo/odoo#52403
Related: odoo/enterprise#10945
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Introduce better methods for introducing tables, join clauses and where
clauses to the Query object, and other methods to generate a complete
SELECT query. The Query object can also behave as a tuple of ids, where
the query is made on demand, and the result is memoized.
Performance fix: when parsing domains using special operators
(`child_of`, ...), or with dotted field on the left value
(`['partner_id.name', ...]`), the ORM starts by a first call on
`search()` and builds a new domain so the final query has "IN (<ids
found by sub-search>)".
The sub-search issued on the comodel uses the default "order by".
The order of the ids is irrelevant here, we only need to know the ids.
A good example is when the sub-search happens on `product.product`,
which is ordered by `default_code, name, id`. The field `name` is
inherited and translated, so the ordering requires a JOIN on
`product_template` and a LEFT JOIN on `ir_translation`. Ordering by `id`
avoids these 2 JOINs.
Some discussion took place about modifying `_generate_order_by()` to
remove the `ORDER BY` clause when the `order` argument is False or ''.
Apart the fact that it would be a breaking change, @odony has shown that
sorting by `id` is so cheap that removing the `ORDER BY` might not
bring significant performance boost [0].
[0] https://github.com/odoo/odoo/pull/52368#issuecomment-646643773
opw-2270690
closesodoo/odoo#53629
X-original-commit: fc91f7b4e8f9f8a95862df1ce95a9639cf7d1363
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Replaces the auto_join scheme which is pretty fast but doesn't quite
work (it's *completely* broken for read_group) by a subquery which is
slower but has the same semantics as the ORM version.
Ideally we'd use `EXISTS (select 1 from ...)` but that has no support
inside the SQL compiler right now, so use `id in (select inverse_field
from ...)`.
Here are some measures of a `read_group` on a simple object with lines
filtered on line values, similar to
read_group([('order_line.product_uom', '=', xxx)],
fields=['total_amount'], groupby=['user_id'])
+-------+-------+-------+--------+
| | ORM | join |subquery|
+-------+-------+-------+--------+
| large | 160 | 10 | 20 |
+-------+-------+-------+--------+
| many | 36000 | 50 | 180 |
+-------+-------+-------+--------+
* values of data fields (filtering, grouping and aggregating) randomly
picked between 10 different values
* "large" is 1000 objects with 1000~15000 lines each
* "many" is 1000000 objects with 1~40 lines each
closesodoo/odoo#48494
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Use a LEFT JOIN on table `ir_translation` instead of a subquery, by
using the same implementation as the one used for ordering by a
translated field. In particular, this avoids having both the LEFT JOIN
and subquery when searching and sorting on a given translated field.
This fixes some cases where search results are actually missing. For
instance, searching with a domain like
['|', ('parent_id.name', '=', 'foo'), ('name', 'like', 'foo')]
where `parent_id` is auto-joined, misses results where `parent_id` is
NULL, because the actual query makes cartesian products for auto-joins:
SELECT "res_partner".id
FROM "res_partner", "res_partner" AS "res_partner__parent_id"
WHERE ("res_partner"."parent_id" = "res_partner__parent_id"."id")
AND ("res_partner__parent_id"."name" = 'foo' OR "res_partner"."name" == 'foo')
Using LEFT JOIN instead of cartesian product above, the query includes
all results where `parent_id` is NULL:
SELECT "res_partner".id
FROM "res_partner"
LEFT JOIN "res_partner" AS "res_partner__parent_id" ON
("res_partner"."parent_id" = "res_partner__parent_id"."id")
WHERE ("res_partner__parent_id"."name" = 'foo' OR "res_partner"."name" == 'foo')
Rémy Voet (ryv)