This commit add a new `odoo.osv.expression.prettify_domain` function
that can be used to format a domain as a string with the correct
indentation.
Example:
['&', '|', ('name', 'like', 'Jack'), ('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
Becomes:
['&',
'|',
('name', 'like', 'Jack'),
('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
closesodoo/odoo#118067
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Rationale
---------
Given the following domain:
['|',
('company_id.name', 'like', 'BE'),
('company_id.city', 'like', 'Brussels')]
The ORM would generate the following SQL query:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."name" like 'BE'
) OR "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."email" like 'help@odoo.com'
);
Which is sub-optiomal as the WHERE clause of the two subqueries could be
regrouped inside of a single query like so:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company" WHERE (
"res_company"."name" like 'BE'
OR "res_company"."email" like 'help@odoo.com'
)
);
Postgres-wise, it is faster to execute the latter query than the former
one. This commit is about optimizing the ORM so that it generates
queries where the WHERE clause of compatible subqueries are grouped
together.
Technical solution
------------------
The solution explored by this work is to replace the regular relational
field accesses (over a dotted path) by a new `any` operator that look as
follow:
(relational_field, 'any', domain)
For instance, the above `('company_id.name', 'like', 'BE')` leaf becomes
`('company_id', 'any', [('name', 'like', 'BE')]` using `any`.
Having a domain as right-hand-side allow for the combination of the
domains of same compatible relations:
[('company_id', 'any', ['|',
('name', 'like', 'BE'),
('city', 'like', 'Brussels')])
Having combined domains allows for generating better SQL queries with
minimal changes to the expression parsing algorithm, which can only
translate a single leaf at a time to SQL. With the new `any` operator,
it is still a single leaf but the right-hand-side contains the merged
domain, thus the combined SQL WHERE clause is immediately generated.
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
Purpose
=======
Allow to search properties fields.
For now, the properties fields are meant to be used with the frontend
top bar filter on views in a later commit, and so everything as been
kept as simple as possible for that particular use case.
Technical
=========
Relational properties
---------------------
We can not search on relational properties using the rec name, or
using one of their fields.
The following domains do not work
```
[('properties.partner_id', 'ilike', 'alice')]
[('properties.partner_ids', 'ilike', 'alice')]
[('properties.user_id.name', 'ilike', 'bob')]
[('properties.user_ids.name', 'ilike', 'bob')]
```
Instead, we should give the ids
```
[('properties.partner_id', '=', 55)]
[('properties.partner_ids', 'in', 55)]
```
IN operator
-----------
The "in" operator is used to either search specific tags / many2many
values, either doing a "OR like" condition on "non-array" value.
It can be used if the left or the right side is not an array
```
[('properties.my_char', 'in', ['a', 'b'])]
[('properties.my_tags', 'in', 'a')]
```
But it can't be used if both side of the condition are array, because
it will require to check type in SQL (which is technically possible,
but add extra complexity, so for now we keep the feature as simple as
possible).
So the following domain is not supported.
```
[('properties.my_tags', 'in', ['a', 'b'])]
```
Instead we should use the "OR" operator
```
['|', ('properties.my_tags', 'in', 'a'), ('properties.my_tags', 'in', 'b')]
```
Tags / many2many - "=" operator
-------------------------------
Tags and many2many have to be searched using the "in" operator.
So the following domain will return unexpected results and raise a warning.
```
[('attributes.mytags', '=', ['a', 'b'])]
```
The reason is that we don't want to dynamically check the type in SQL,
and so we need the "in" domain operator to know what to do in SQL.
Task-2980121
Part-of: odoo/odoo#101901
This API is much more sensible for making subqueries. Specifically, one
can generate a subquery without the clauses LIMIT and ORDER BY.
Part-of: odoo/odoo#112126
This simplifies the use of subqueries by avoiding some costly default
order on the model or the idiotic order='id'. Method _flush_search()
has been adapted accordingly.
Part-of: odoo/odoo#112126
The goal is to be able to use Query objects for both subqueries and
known ids tuples. This provides a single API for injecting either a
subquery or its resulting ids into another query.
Part-of: odoo/odoo#112126
For non-translated field, when search(['name', 'in', list_right])
The behavior of False and the behavior of None in list_right should be the same.
Translated fields need customized process for 'in' operator
X-original-commit: a9a3c666f08c5612432dba40a69c7cfdc54ba96e
Part-of: odoo/odoo#103031
The trigram index function jsonb_path_query_array("column_name", '$.*')::text
uses all translations' representations to build the indexed text. So the
original text needs to be JSON-escaped correctly to match it.
X-original-commit: 7547df664945dddcb839e4903068f7f25ecfc08c
Part-of: odoo/odoo#103031
make searching translated field language dependent
add an extra filter using trigram index to speed up '=' and 'like' search
X-original-commit: 5b6f0a9e5f9d60735e306709a3e7d7ac46a586be
Part-of: odoo/odoo#103031
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Move the definition of class Query to odoo.tools, in order to avoid
circular imports when importing Query in core Odoo modules.
Also reorganize imports in the impacted modules.
Part-of: odoo/odoo#66938
Revert commit 54cd73a4f4 as it prevents customers from closing
their pos sessions without timeouting.
closesodoo/odoo#88941
X-original-commit: 43165dc3f7a8cc199219b8a32b65eb0ee7b5c2dd
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
This fixes a problem that occurs with conditions on one2many fields,
where most conditions are translated with a subquery like
"model".id IN (
SELECT "comodel"."model_id" FROM "comodel" WHERE ...
)
The issue occurs when the subquery return NULL values, i.e., when the
field "model_id" in the subquery above can be NULL. The fix simply
consists in adding the condition "model_id IS NOT NULL" in the WHERE
clause of the subquery.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
closesodoo/odoo#87285
X-original-commit: 6d680bac0333c6c7ce9f056cba0e1f7e9fa22f9c
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This fixes two problems that occur with conditions on many2many fields:
the subtle bug caused by NULL values that are returned by subqueries,
and the performance of those subqueries.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
Regarding performance, consider a domain like [('tag_ids', '=', False)].
It is currently translated in SQL as
"model".id NOT IN (
SELECT "model_id"
FROM "model_tag_rel"
WHERE "model_id" IS NOT NULL
)
PostgreSQL does not handle well this "NOT IN" expression when the
relation table is large, and uses some very inefficient query plan in
that case. The inefficient query plan is possibly related to the
special handling of NULL values, by the way. This commit changes the
above expression to a "NOT EXISTS" expression (as below), which is
executed with a much more efficient query plan, even with large tables.
NOT EXISTS (
SELECT 1
FROM "model_tag_rel"
WHERE "model_tag_rel"."model_id" = "model".id
)
This change was motivated by a use-case with 2.7M account moves, 11M
account move lines and a many2many relation table with 2.3M lines
(`account_move_line_account_tax_rel`). The query was timing out (15
minutes), and it now takes 15 seconds.
We have replaced all the conditions on many2many fields to use the
relational operators EXISTS and NOT EXISTS.
A test has been added to reflect a case where the many2many table
contains NULL values, like the one for channel_partner in regards to
partner_id and guest_id.
task-2753782
X-original-commit: 334b2aa2aa437d2ed86130f2ed426025f8fe4b86
Part-of: odoo/odoo#87285
Co-authored-by: Didier Debondt <did@odoo.com>
Purpose
=======
Loading a view trying to retrieve the hierarchy of a record using the field
parent_path could lead to an access error if records are mixed up.
Note:
Easily achievable for an end user. It could happen in a multi company
environment (you activate 2 companies at the same time) while configuring
the departments (using _parent_store=True), and you say that you have:
R&D (company=1):
- R&D Belgium (company=1)
- R&D India (company=2)
Then you go back to a single-company environment, you click on the form
view of an employee and crack, since there is a multi-company rule on the
departments, and that the search panel is loading the hierarchy for
display purpose.
Use sudo to avoid access rights issues, as the forbidden records will be
filtered automatically by the constructed domain like this:
```py
parent_ids = [
int(label)
for rec in left_model.sudo().browse(ids)
for label in rec.parent_path.split('/')[:-1]
]
domain = [('id', 'in', parent_ids)]
```
This actually makes the search consistent with the case where
_parent_store=False.
closesodoo/odoo#85042
X-original-commit: a7e13b32e6c47a3fdcfad8a6c6337b89580ad70a
Signed-off-by: Raphael Collet <rco@odoo.com>
The `column_format` of the `Field` class was unused and create useless
noise in the ORM. Then remove it and simplify some flows.
task-2735546
Part-of: odoo/odoo#82727
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
fields) are out of scope for the lint but my editor catches
closesodoo/odoo#74604
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Currently, the domain expression on a many2one field with
negative operator + string values leads to a extra SQL request on the
target model of the many2one which is not optimal and lead to
performance degradation.
Example:
--------
With a simple model A with a one2many (`b_id`) targeting model B (with
a field `b_name` and `_rec_name = b_name`):
The `A.search([('b_id', 'not like', 'OneName')])` will
generate two SQL requests:
- A _name_search on model B and returning ids who match
`('b_name', 'not like', 'OneName')`.
- A other on the model A with the application of the result
of the first one `('b_id', 'in', <returning ids> + [False])`.
Solution:
---------
Avoid the extra SQL request (the first one) and use subquery instead.
We do it by translate the `('b_id', 'in', <returning ids> + [False])`
expression into `['|', ('b_id', 'in', _subquery), ('b_id', '=', False)]`
task-2671476
Part-of: odoo/odoo#78948
This fixes a performance issue: ORDER BY clauses in subqueries can make
the query unexpectedly slow. We should avoid this situation, since
ORM-generated queries have an ORDER BY clause which is not relevant in
the context of a subquery.
The following example was found:
SELECT "pos_payment"."id" AS "id"
FROM "pos_payment"
WHERE ("pos_payment"."pos_order_id" in
(SELECT "pos_order".id
FROM "pos_order"
WHERE ("pos_order"."company_id" in (1))
ORDER BY "pos_order"."id"))
AND "pos_payment".id IN (1285508)
Here are the query plans made by PostgreSQL on this query with and
without the ORDER BY clause. The query time went from 1240ms to
0.402ms, which is 3000 times faster!
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order"."company_id" in (1)) ORDER BY "pos_order"."id" )) AND "pos_payment".id IN (1285508);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Merge Semi Join (cost=2.88..82726.85 rows=1 width=4) (actual time=1239.361..1239.364 rows=1 loops=1)
Merge Cond: (pos_payment.pos_order_id = pos_order.id)
-> Sort (cost=2.46..2.46 rows=1 width=8) (actual time=0.021..0.022 rows=1 loops=1)
Sort Key: pos_payment.pos_order_id
Sort Method: quicksort Memory: 25kB
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.014..0.015 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..66770.53 rows=1282120 width=4) (actual time=0.013..1148.194 rows=1182463 loops=1)
Filter: (company_id = 1)
Planning time: 0.272 ms
Execution time: 1239.396 ms
(11 rows)
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order
"."company_id" in (1)) )) AND "pos_payment".id IN (1285508);
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.85..4.89 rows=1 width=4) (actual time=0.047..0.049 rows=1 loops=1)
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.027..0.028 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..2.45 rows=1 width=4) (actual time=0.018..0.018 rows=1 loops=1)
Index Cond: (id = pos_payment.pos_order_id)
Filter: (company_id = 1)
Planning time: 0.322 ms
Execution time: 0.080 ms
(8 rows)
closesodoo/odoo#77357
X-original-commit: 947c3bd72ee84c3a298e0c9c365340ddf16b307d
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Stanislas Sobieski (sts@odoo.com)
For searches, Odoo uses `unaccent` if it's available. On some
technical fields this is completely unnecessary and precludes the use
of indexes.
This PR provides:
* an opt-out (`unaccent = False`) on `String` and `Text` fields
* a warning if `unaccent` is enabled on *parent_path* fields as their
performance can be rather critical and not using the index is quite
an issue (note: the check that `parent_path` fields have been moved
outside of the check for their existence as we want to check that
the field is declared and correctly configured in all cases,
probably)
Task 2627454
Part-of: odoo/odoo#76436
`website` added its own sql-compatible version of
`get_unaccent_wrapper` in order to use `psycopg2.sql` for safety.
That is very sad.
Add a condition in the "standard" unaccent wrapper which checks
whether the input type is a `Composable`, and in that case return an
`SQL`. This increases the cost of the wrapper a tad but probably not
to a really sensible amount.
Task 2634851
Part-of: odoo/odoo#76436
Before this commit, the implementation crashes when the ancestors of a
record contain non-accessible records. This fixes the code to avoid it
to crash.
We also make the semantics of hierarchical searches more consistent in
this case: searching with operators 'child_of' and 'parent_of' returns
the subset of accessible records that satisfy the hierarchy operator.
We actually make it coincide with the results of the search when using
the "_parent_store" optimization.
closesodoo/odoo#73962
X-original-commit: 3e1b960acf3f6a726338476ba9e62e5ef5c84ef9
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Domain terms of the form `('field', 'not in', [...])` generate incorrect
queries for translated fields, example (model `res.country`, field `name`):
```
psycopg2.errors.SyntaxError: syntax error at or near "ARRAY"
LINE 1: ...M "res_country" WHERE "res_country"."name" not in ARRAY['No ...
```
The root cause is that the right part of the term is not converted to
tuple.
Observed during the upgrade request 16639
opw-2525553
closesodoo/odoo#73030
X-original-commit: 7c4db97e21f74a429e56f6cae4d4786ee74f7e22
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When building a domain `(=|!=)` with
a Binary field stored in attachment and
the right part of the domain is not null,
the full binary content is logged, giving
the logs a huge size.
Now, the content of the binary is cropped to
20 chars as it is useless to log more.
opw-2527629
closesodoo/odoo#70740
X-original-commit: 0ed46ae94b20b30c10025b2a458e1ecf92c2712d
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
From having to work with it recently, it was not super clear what each
argument did (especially the link argument), and more documentation =
more betterer code
A domain with a negated thruty leaf raises an exception here
https://github.com/odoo/odoo/blob/835197e48fbbee1c8ac35d52287537fa6f0bdd36/odoo/osv/expression.py#L614
due to
https://github.com/odoo/odoo/blob/835197e48fbbee1c8ac35d52287537fa6f0bdd36/odoo/osv/expression.py#L670
and the fact that `FALSE_LEAF != (1, "!=", 1)` and `TRUE_LEAF != (0, "!=", 1)`
Current wrong behaviour:
```
>>> distribute_not(['!', TRUE_LEAF])
[(1, '!=', 1)]
>>> is_leaf((1, '!=', 1))
False
```
This issue was noticed when adapting domains for fields that have been
removed during migration. Example (extract of) traceback:
```
Traceback (most recent call last):
File "/home/odoo/src/odoo/13.0/odoo/http.py", line 624, in _handle_exception
return super(JsonRequest, self)._handle_exception(exception)
...
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 677, in __init__
self.parse()
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 814, in parse
self.stack = [ExtendedLeaf(leaf, self.root_model) for leaf in self.expression]
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 814, in <listcomp>
self.stack = [ExtendedLeaf(leaf, self.root_model) for leaf in self.expression]
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 562, in __init__
self.check_leaf(internal)
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 618, in check_leaf
raise ValueError("Invalid leaf %s" % str(self.leaf))
ValueError: Invalid leaf (0, '!=', 1)
```
closesodoo/odoo#68665
X-original-commit: e20c23658d15a135e5f35b0e8dec6e3202cecaab
Signed-off-by: Christophe Simonis <chs@odoo.com>
Assume field B depends on A, and searching on B is rewritten as a domain
that mentions A. If A is modified, and then we search on B, one has to
flush A to the database before searching.
closesodoo/odoo#57963
X-original-commit: b242b7b1382ee39ebefa58c556931b7dae77f669
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Current behavior:
Auto-generated SQL fails with ERROR: missing FROM-clause entry for table when table alias names exceed the target length of 63 characters. This is because odoo.osv.expression.py handles shortening the table name alias in generate_table_alias, but not in get_join_conditions. In the resulting SQL, shortened aliases are used in the FROM clause, whereas in the join conditions, the full alias is used, which is both exceeding the Postgres limit and not matching the table name alias in the FROM clause.
Expected behavior:
auto-generated SQL runs without errors.
Implementation of fix:
- refactor alias shortening into new function _shorten_alias to make it reusable
- Use _shorten_alias in both generate_table_alias and get_join_conditions
closesodoo/odoo#57944
X-original-commit: 40b97b37e2188b082cd70eec1cdf14bf9366377e
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
By adding a parameter `_table_query` on the Model, we can now have views
that depend on the context. The query is used instead of the table name
in ORM operations. This allows to pre-compute some values to improve
performance, instead of storing context values in the database.
Co-authored-by: william-andre <wan@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
Make the method `_search` return a `Query` object, and make that object
generate a subquery when used on the right-hand side of a condition.
closesodoo/odoo#52403
Related: odoo/enterprise#10945
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Introduce better methods for introducing tables, join clauses and where
clauses to the Query object, and other methods to generate a complete
SELECT query. The Query object can also behave as a tuple of ids, where
the query is made on demand, and the result is memoized.
This avoids the formatting part of those methods (`browse` in the case
of `search`, `display_name` in the case of `name_search`), which is
useless in most cases.
This also enables further optimizations where `_search` and
`_name_search` return subqueries instead of record ids.
Steps to reproduce:
- install helpdesk, hr_expense (the demo data has a clear example of the bug)
- go to helpdesk > tickets > set a custom filter > "activities is set"
Previous behavior:
the results show the ticket with id=2 as if it had an activity set.
Because the search sql query collides with other activities set on other models
(hr.expense.sheet in this example)
Current behavior:
the sql query returns the proper result
opw-2273680
closesodoo/odoo#55297
X-original-commit: 6e90c41413f6a726c08071c71d01c738a6538079
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Performance fix: when parsing domains using special operators
(`child_of`, ...), or with dotted field on the left value
(`['partner_id.name', ...]`), the ORM starts by a first call on
`search()` and builds a new domain so the final query has "IN (<ids
found by sub-search>)".
The sub-search issued on the comodel uses the default "order by".
The order of the ids is irrelevant here, we only need to know the ids.
A good example is when the sub-search happens on `product.product`,
which is ordered by `default_code, name, id`. The field `name` is
inherited and translated, so the ordering requires a JOIN on
`product_template` and a LEFT JOIN on `ir_translation`. Ordering by `id`
avoids these 2 JOINs.
Some discussion took place about modifying `_generate_order_by()` to
remove the `ORDER BY` clause when the `order` argument is False or ''.
Apart the fact that it would be a breaking change, @odony has shown that
sorting by `id` is so cheap that removing the `ORDER BY` might not
bring significant performance boost [0].
[0] https://github.com/odoo/odoo/pull/52368#issuecomment-646643773
opw-2270690
closesodoo/odoo#53629
X-original-commit: fc91f7b4e8f9f8a95862df1ce95a9639cf7d1363
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Replaces the auto_join scheme which is pretty fast but doesn't quite
work (it's *completely* broken for read_group) by a subquery which is
slower but has the same semantics as the ORM version.
Ideally we'd use `EXISTS (select 1 from ...)` but that has no support
inside the SQL compiler right now, so use `id in (select inverse_field
from ...)`.
Here are some measures of a `read_group` on a simple object with lines
filtered on line values, similar to
read_group([('order_line.product_uom', '=', xxx)],
fields=['total_amount'], groupby=['user_id'])
+-------+-------+-------+--------+
| | ORM | join |subquery|
+-------+-------+-------+--------+
| large | 160 | 10 | 20 |
+-------+-------+-------+--------+
| many | 36000 | 50 | 180 |
+-------+-------+-------+--------+
* values of data fields (filtering, grouping and aggregating) randomly
picked between 10 different values
* "large" is 1000 objects with 1000~15000 lines each
* "many" is 1000000 objects with 1~40 lines each
closesodoo/odoo#48494
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Instead of merging Query objects (main domain and access rules domains),
which requires renaming aliases in query strings, make `expression` push
its result in an existing Query object. This removes tricky code.
This also avoids using `IrRule.domain_get()`, which has become unsafe,
since it returns a list of tables (for the FROM clause) which does not
include the joins from the generated Query object.
closesodoo/odoo#49999
Related: odoo/enterprise#10126
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Use a LEFT JOIN on table `ir_translation` instead of a subquery, by
using the same implementation as the one used for ordering by a
translated field. In particular, this avoids having both the LEFT JOIN
and subquery when searching and sorting on a given translated field.
This fixes some cases where search results are actually missing. For
instance, searching with a domain like
['|', ('parent_id.name', '=', 'foo'), ('name', 'like', 'foo')]
where `parent_id` is auto-joined, misses results where `parent_id` is
NULL, because the actual query makes cartesian products for auto-joins:
SELECT "res_partner".id
FROM "res_partner", "res_partner" AS "res_partner__parent_id"
WHERE ("res_partner"."parent_id" = "res_partner__parent_id"."id")
AND ("res_partner__parent_id"."name" = 'foo' OR "res_partner"."name" == 'foo')
Using LEFT JOIN instead of cartesian product above, the query includes
all results where `parent_id` is NULL:
SELECT "res_partner".id
FROM "res_partner"
LEFT JOIN "res_partner" AS "res_partner__parent_id" ON
("res_partner"."parent_id" = "res_partner__parent_id"."id")
WHERE ("res_partner__parent_id"."name" = 'foo' OR "res_partner"."name" == 'foo')
The management of the join contexts now relies on the `Query` object,
that has all the necessary logic to do so. The refactoring also removes
the class `ExtendedLeaf`, which is no longer necessary: the join
contexts are all managed by a single `Query` object, and the parsing
stack now only contains triples like (leaf, model, table_alias).
Also inline directly the SQL translation made in `_to_sql`, which
overall simplifies the management of SQL parameters in the domain
compilation. The algorithm is illustrated in the code itself.
Rémy Voet (ryv)