We take advantage of SQL objects to discard the use of domain operators
'inselect and 'not inselect'. Indeed, as we now support SQL objects at
the right-hand side of a condition, one can replace conditions like
(lhs, 'inselect', (query, params)) by (lhs, 'in', SQL(query, *params)).
This commit also adds tests that ensure the correct behavior of the
adaptation of the implementation.
Part-of: odoo/odoo#138019
In the context of branches, you need to be able to select taxes of
an ancestor company on an invoice/bill, even when you don't have access
to the ancestor company.
In order to do that, we relax the restriction when searching with a
`parent_of` or `child_of` on a related field, so that it also includes
ids of related field records you don't have access to. This should not
be a problem, since in the end the search will return records of a
model restricted by its own access rules.
task-3503204
closesodoo/odoo#138942
Signed-off-by: William André (wan) <wan@odoo.com>
Signed-off-by: Dylan Kiss (dyki) <dyki@odoo.com>
The `=like`/`like` domain operators are accent-insensitive, but still
case-sensitive. This is not really coherent, since `unaccent` is a best
effort to find records from the client, and it is the same idea behind
being case-insensitive. Also, `=like`/`like` cannot be used from the web
client, and we use them in domain to search from the Python side.
Moreover, adding `unaccent` to `=like` can be very inefficient when
searching for a prefix ('prefix%'). In fact, PostgreSQL can use btree
index to find prefix matches, but because we create dy default btree
index without unaccent (when we put
`index=True/'btree'/'btree_not_null'` on the field), PostgreSQL cannot
use this index.
Part-of: odoo/odoo#136007
We introduce a new class of objects to wrap SQL code together with its
parameters. It is designed to be easily composable and to discourage
SQL injections. Its API is similar to the methods of module 'logging':
the code is a format string, and the positional parameters are meant to
be merged into it using the string formatting operator.
# default and increment are parameters of the SQL code in first argument
term = SQL("COALESCE(value, %s) + %s", default, increment)
# term can safely be injected into another SQL, besides regular parameters
query = SQL("SELECT %s FROM mytable WHERE id = %s", term, id_)
The SQL wrapper can return the final SQL code string as query.code, and
the corresponding parameters as query.params (list). The cursor method
execute() can now take an SQL object, and execute it just like
cr.execute(query.code, query.params)
It is quite easy to make SQL objects safe against SQL injections: if the
code is a string literal, then the SQL object is guaranteed safe,
provided the SQL objects within its parameters are themselves safe.
Part-of: odoo/odoo#134677
Goal:
* Simplified modifiers to only have one way to define modifiers;
* Remove states attributes on python field;
* Use python expression in view `required`, `readonly`, `invisible`;
* More accurate validation of xml views.
This commit change the syntax to python expression. The next commit
will update/convert all xml views.
Before this commit:
* the `required`, `readonly` and `invisible` attributes can only have
values of `True`, `False`, 1, 0 or a python expression to use the
context;
* the `attrs` attribute define a dict. The key of this dict was
`required`, `readonly` and `invisible` and the values are the domain or
a string representing a domain to be evaluate as python expression.
This python expressions was evaluate by the javascript with view fields
and other contextual values as: context, uid, parent, active_id,
active_ids, active_model, allowed_company_ids, current_company_id.
* the `states` attribute in the view was a comma separated list of the
state. This list was combined with the `invisible` attribute;
* the `invisible` attribute on python field is used as default value;
* the `states` attribute on python field was dictionnary with state as
key and list of tuple. This structure was combined with `readonly` view
attribute.
* After combining, the resulting domains of the different attributes
`required`, `readonly` and `invisible` are evaluated with the values of
the fields. The `invisible` attributes is splitted into two use:
`invisible` and `column_invisible`.
After this commit:
* The attributes `required`, `readonly`, `invisible` and
`column_invisible` define python expression. This python expressions
are evaluate by the javascript with view fields and other contextual
values as: context, uid, parent, active_id, active_ids, active_model,
allowed_company_ids, current_company_id.
The domains can contains contextual value and will be evaluate by the
javascript.
```xml
<field name="field_a" readonly="not context.get('show_a')" attrs="{'readonly': [('field_b', '!=', False), ('field_c', '=', parent.c)]}"/>
<field name="field_b" states="draft"/>
```
will be replaced by
```xml
<field name="field_a" readonly="not context.get('show_a') or field_b and field_c == parent.c"/>
<field name="field_b" invisible="state != 'draft'"/>
```
Some inherited views will be modified differently in order to maintain
the previous behavior:
```xml
<field name="field_a" readonly="not context.get('show_a')" attrs="{'invisible': [('field_b', '!=', False)]}">
```
```xml
<field name="field_a" position="attributes">
<attribute name="attrs">{'readonly': [('field_c', '=', False)], 'invisible': [('field_d', '!=', '3')]}<attribute>
</field>
```
will be replaced by
```xml
<field name="field_a" readonly="not context.get('show_a')" invisible="field_b">
```
```xml
<field name="field_a" position="attributes">
<attribute name="readonly" add="(not field_c)" separator=" or "/>
<attribute name="invisible">field_d != 3<attribute>
</field>
```
Validation:
A stricter control is made on the level of the attributes (modifiers)
and the fields necessary for these. The use of the previous attributes
'attr' and 'states' triggers an error (these no longer exist after the
application of the migration script)
task-2495504
Part-of: odoo/odoo#104741
Allow some models to be delegated to the main company of the object.
For instance:
* Have company Parent and Child
* Child has access to all the accounts of Parent
* When creating a document in Child using an account of Parent, there
shouldn't be a consistency error.
In order to do that, a new method `_check_company_domain` has been added.
This method is used when `check_company=True` is set on the field
declaration to:
* filter relational fields in the UI [^1]
* validate relational fields when writing on them in `create` and `write`
That method should also be used in the business code for models likely
to be shared amongst company branches instead of hardcoding the domain
based on `company_id`.
The same applies on domain restriction on the field declaration: the
flag `check_company` should be prefered instead of providing the domain
explicitly based on `company_id`.
Since a lot of validation and filtering will now be done based on
`parent_of`/`child_of` of the company, some support has been added for
that special case in `filtered_domain` in order to avoid doing
additional queries.
task-3371677
[^1]: because of that change, support has been added on the python
interpreter of the client to allow concatenatng domains.
Part-of: odoo/odoo#125642
Let's assume that
* Company S is a sub company of it's parent company P
* Company S has access to all the accounts and taxes of company P
* Some taxes are archived, but used
Because of the needed access rules, there will be a `parent_of` on the
record rules of accounts and taxes.
If we consider that we should consider the context key `active_test` to
add a implicit `('active', '=', True)` clause in the domain when
evaluating `parent_of` and `child_of` clauses, an access error will be
raised instead of hiding the archived records, even when simply trying
to read an archived record.
The archive feature and the security rules should be independent; if a
security rules needs to depend on the fact that a record is archived, it
should be explicit in the domain and not rely on side effects of the
implementation of `parent_of`/`child_of`
Part-of: odoo/odoo#125642
Since 5a998694a6,
`[('<many2one>', 'not any', [<domain>])]` matches rows where the
<many2one> is set and the corresponding many2one row matches the
<domain>. This is incorrect. 'not any' should be the inverse of
the 'any' operator and domains such as
`['!', ('partner_id.name', '=', 'System')]` are incorrectly converted
to "Return every record with a partner name != 'System'"
when it should be "Return every record with a partner name != 'System'
OR without partner at all".
Fix semantic and add tests to avoid any future regressions.
X-original-commit: c8c1ef45f24482e380529daf2de55b9091338a83
Part-of: odoo/odoo#127750
We have many issues with this log, except that with a `logger.error` we
do not have exception information (exc_info).
As mentionned in the documentation above, we cannot use `logger.exception`
in this case because we are not within an exception handler.
However, we can still retrieve the exception information using `sys`
module and the `exc_info()` method.
https://docs.python.org/3/library/logging.html#logging.Logger.exception
```
exception(msg, *args, **kwargs)
...
This method should only be called from an exception handler.
```
https://docs.python.org/3/library/sys.html#sys.exc_info
```
sys.exc_info()
This function returns the old-style representation of the handled exception.
...
If no exception is being handled anywhere on the stack, this function
return a tuple containing three None values.
...
```
closesodoo/odoo#124208
X-original-commit: b0844d2f3c0c9655746b4c16c9b612bae9954618
Signed-off-by: Christophe Simonis (chs) <chs@odoo.com>
1.
Infinite loop may happen on using `parent_of`\`child_of` when there is a
recursion in the tree (e.g. a record is marked as a parent of itself). Fix it by
excluding seen records from the next iteration.
2.
Another problem with `child_of` is `parent_id` that references to another model.
For example, the `parent_id` may come from inherited model. It's the case with
`res.users` and `res.partner` models. It may lead to a random search results.
Avoid that by raising exception in case of wrong usage of the `child_of`
operator.
STEPS:
In demo data, there is a partner called "Wood Corner" that is `res.partner(9,)`
that has 3 sub-contacts. If we give Portal access to two of them, we end up with
a database, where we have a `res.users(9,)` record that has a partner, which has a
`parent_id` to "Wood corner". So this way, the user id is the same as the user's
partner's parent contact id.
After that open a shell and type:
```
env['res.partner'].search([["user_ids", "child_of", 9]])
```
BEFORE: infinite loop (without change n.1) or random search results (when change
n.1 is applied)
AFTER: ValueError exception
---
opw-2729740
closesodoo/odoo#123353
X-original-commit: 2e1adc0c3e33fcf7989d27bb4d1c2e3c019faf2b
Signed-off-by: Ivan Elizaryev (iel) <iel@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This commit add a new `odoo.osv.expression.prettify_domain` function
that can be used to format a domain as a string with the correct
indentation.
Example:
['&', '|', ('name', 'like', 'Jack'), ('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
Becomes:
['&',
'|',
('name', 'like', 'Jack'),
('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
closesodoo/odoo#118067
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Rationale
---------
Given the following domain:
['|',
('company_id.name', 'like', 'BE'),
('company_id.city', 'like', 'Brussels')]
The ORM would generate the following SQL query:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."name" like 'BE'
) OR "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."email" like 'help@odoo.com'
);
Which is sub-optiomal as the WHERE clause of the two subqueries could be
regrouped inside of a single query like so:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company" WHERE (
"res_company"."name" like 'BE'
OR "res_company"."email" like 'help@odoo.com'
)
);
Postgres-wise, it is faster to execute the latter query than the former
one. This commit is about optimizing the ORM so that it generates
queries where the WHERE clause of compatible subqueries are grouped
together.
Technical solution
------------------
The solution explored by this work is to replace the regular relational
field accesses (over a dotted path) by a new `any` operator that look as
follow:
(relational_field, 'any', domain)
For instance, the above `('company_id.name', 'like', 'BE')` leaf becomes
`('company_id', 'any', [('name', 'like', 'BE')]` using `any`.
Having a domain as right-hand-side allow for the combination of the
domains of same compatible relations:
[('company_id', 'any', ['|',
('name', 'like', 'BE'),
('city', 'like', 'Brussels')])
Having combined domains allows for generating better SQL queries with
minimal changes to the expression parsing algorithm, which can only
translate a single leaf at a time to SQL. With the new `any` operator,
it is still a single leaf but the right-hand-side contains the merged
domain, thus the combined SQL WHERE clause is immediately generated.
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
Purpose
=======
Allow to search properties fields.
For now, the properties fields are meant to be used with the frontend
top bar filter on views in a later commit, and so everything as been
kept as simple as possible for that particular use case.
Technical
=========
Relational properties
---------------------
We can not search on relational properties using the rec name, or
using one of their fields.
The following domains do not work
```
[('properties.partner_id', 'ilike', 'alice')]
[('properties.partner_ids', 'ilike', 'alice')]
[('properties.user_id.name', 'ilike', 'bob')]
[('properties.user_ids.name', 'ilike', 'bob')]
```
Instead, we should give the ids
```
[('properties.partner_id', '=', 55)]
[('properties.partner_ids', 'in', 55)]
```
IN operator
-----------
The "in" operator is used to either search specific tags / many2many
values, either doing a "OR like" condition on "non-array" value.
It can be used if the left or the right side is not an array
```
[('properties.my_char', 'in', ['a', 'b'])]
[('properties.my_tags', 'in', 'a')]
```
But it can't be used if both side of the condition are array, because
it will require to check type in SQL (which is technically possible,
but add extra complexity, so for now we keep the feature as simple as
possible).
So the following domain is not supported.
```
[('properties.my_tags', 'in', ['a', 'b'])]
```
Instead we should use the "OR" operator
```
['|', ('properties.my_tags', 'in', 'a'), ('properties.my_tags', 'in', 'b')]
```
Tags / many2many - "=" operator
-------------------------------
Tags and many2many have to be searched using the "in" operator.
So the following domain will return unexpected results and raise a warning.
```
[('attributes.mytags', '=', ['a', 'b'])]
```
The reason is that we don't want to dynamically check the type in SQL,
and so we need the "in" domain operator to know what to do in SQL.
Task-2980121
Part-of: odoo/odoo#101901
This API is much more sensible for making subqueries. Specifically, one
can generate a subquery without the clauses LIMIT and ORDER BY.
Part-of: odoo/odoo#112126
This simplifies the use of subqueries by avoiding some costly default
order on the model or the idiotic order='id'. Method _flush_search()
has been adapted accordingly.
Part-of: odoo/odoo#112126
The goal is to be able to use Query objects for both subqueries and
known ids tuples. This provides a single API for injecting either a
subquery or its resulting ids into another query.
Part-of: odoo/odoo#112126
For non-translated field, when search(['name', 'in', list_right])
The behavior of False and the behavior of None in list_right should be the same.
Translated fields need customized process for 'in' operator
X-original-commit: a9a3c666f08c5612432dba40a69c7cfdc54ba96e
Part-of: odoo/odoo#103031
The trigram index function jsonb_path_query_array("column_name", '$.*')::text
uses all translations' representations to build the indexed text. So the
original text needs to be JSON-escaped correctly to match it.
X-original-commit: 7547df664945dddcb839e4903068f7f25ecfc08c
Part-of: odoo/odoo#103031
make searching translated field language dependent
add an extra filter using trigram index to speed up '=' and 'like' search
X-original-commit: 5b6f0a9e5f9d60735e306709a3e7d7ac46a586be
Part-of: odoo/odoo#103031
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Move the definition of class Query to odoo.tools, in order to avoid
circular imports when importing Query in core Odoo modules.
Also reorganize imports in the impacted modules.
Part-of: odoo/odoo#66938
Revert commit 54cd73a4f4 as it prevents customers from closing
their pos sessions without timeouting.
closesodoo/odoo#88941
X-original-commit: 43165dc3f7a8cc199219b8a32b65eb0ee7b5c2dd
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
This fixes a problem that occurs with conditions on one2many fields,
where most conditions are translated with a subquery like
"model".id IN (
SELECT "comodel"."model_id" FROM "comodel" WHERE ...
)
The issue occurs when the subquery return NULL values, i.e., when the
field "model_id" in the subquery above can be NULL. The fix simply
consists in adding the condition "model_id IS NOT NULL" in the WHERE
clause of the subquery.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
closesodoo/odoo#87285
X-original-commit: 6d680bac0333c6c7ce9f056cba0e1f7e9fa22f9c
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
This fixes two problems that occur with conditions on many2many fields:
the subtle bug caused by NULL values that are returned by subqueries,
and the performance of those subqueries.
The problem with NULL values returned by subqueries is the fact that
they make the SQL condition undetermined instead of false, and this
discards some of the results returned by a query. Simply consider:
id NOT IN (1, 2, 3)
The value id=3 makes the condition false, while i=4 makes it true. Now
consider that the set also includes a NULL value, like:
id NOT IN (1, 2, 3, NULL)
The value id=3 makes the condition false, but the value id=4 makes it
undetermined. Therefore this condition is actually undetermined for all
possible values of id, and a query containing that condition simply
returns nothing!
Regarding performance, consider a domain like [('tag_ids', '=', False)].
It is currently translated in SQL as
"model".id NOT IN (
SELECT "model_id"
FROM "model_tag_rel"
WHERE "model_id" IS NOT NULL
)
PostgreSQL does not handle well this "NOT IN" expression when the
relation table is large, and uses some very inefficient query plan in
that case. The inefficient query plan is possibly related to the
special handling of NULL values, by the way. This commit changes the
above expression to a "NOT EXISTS" expression (as below), which is
executed with a much more efficient query plan, even with large tables.
NOT EXISTS (
SELECT 1
FROM "model_tag_rel"
WHERE "model_tag_rel"."model_id" = "model".id
)
This change was motivated by a use-case with 2.7M account moves, 11M
account move lines and a many2many relation table with 2.3M lines
(`account_move_line_account_tax_rel`). The query was timing out (15
minutes), and it now takes 15 seconds.
We have replaced all the conditions on many2many fields to use the
relational operators EXISTS and NOT EXISTS.
A test has been added to reflect a case where the many2many table
contains NULL values, like the one for channel_partner in regards to
partner_id and guest_id.
task-2753782
X-original-commit: 334b2aa2aa437d2ed86130f2ed426025f8fe4b86
Part-of: odoo/odoo#87285
Co-authored-by: Didier Debondt <did@odoo.com>
Purpose
=======
Loading a view trying to retrieve the hierarchy of a record using the field
parent_path could lead to an access error if records are mixed up.
Note:
Easily achievable for an end user. It could happen in a multi company
environment (you activate 2 companies at the same time) while configuring
the departments (using _parent_store=True), and you say that you have:
R&D (company=1):
- R&D Belgium (company=1)
- R&D India (company=2)
Then you go back to a single-company environment, you click on the form
view of an employee and crack, since there is a multi-company rule on the
departments, and that the search panel is loading the hierarchy for
display purpose.
Use sudo to avoid access rights issues, as the forbidden records will be
filtered automatically by the constructed domain like this:
```py
parent_ids = [
int(label)
for rec in left_model.sudo().browse(ids)
for label in rec.parent_path.split('/')[:-1]
]
domain = [('id', 'in', parent_ids)]
```
This actually makes the search consistent with the case where
_parent_store=False.
closesodoo/odoo#85042
X-original-commit: a7e13b32e6c47a3fdcfad8a6c6337b89580ad70a
Signed-off-by: Raphael Collet <rco@odoo.com>
The `column_format` of the `Field` class was unused and create useless
noise in the ORM. Then remove it and simplify some flows.
task-2735546
Part-of: odoo/odoo#82727
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
fields) are out of scope for the lint but my editor catches
closesodoo/odoo#74604
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Currently, the domain expression on a many2one field with
negative operator + string values leads to a extra SQL request on the
target model of the many2one which is not optimal and lead to
performance degradation.
Example:
--------
With a simple model A with a one2many (`b_id`) targeting model B (with
a field `b_name` and `_rec_name = b_name`):
The `A.search([('b_id', 'not like', 'OneName')])` will
generate two SQL requests:
- A _name_search on model B and returning ids who match
`('b_name', 'not like', 'OneName')`.
- A other on the model A with the application of the result
of the first one `('b_id', 'in', <returning ids> + [False])`.
Solution:
---------
Avoid the extra SQL request (the first one) and use subquery instead.
We do it by translate the `('b_id', 'in', <returning ids> + [False])`
expression into `['|', ('b_id', 'in', _subquery), ('b_id', '=', False)]`
task-2671476
Part-of: odoo/odoo#78948
This fixes a performance issue: ORDER BY clauses in subqueries can make
the query unexpectedly slow. We should avoid this situation, since
ORM-generated queries have an ORDER BY clause which is not relevant in
the context of a subquery.
The following example was found:
SELECT "pos_payment"."id" AS "id"
FROM "pos_payment"
WHERE ("pos_payment"."pos_order_id" in
(SELECT "pos_order".id
FROM "pos_order"
WHERE ("pos_order"."company_id" in (1))
ORDER BY "pos_order"."id"))
AND "pos_payment".id IN (1285508)
Here are the query plans made by PostgreSQL on this query with and
without the ORDER BY clause. The query time went from 1240ms to
0.402ms, which is 3000 times faster!
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order"."company_id" in (1)) ORDER BY "pos_order"."id" )) AND "pos_payment".id IN (1285508);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Merge Semi Join (cost=2.88..82726.85 rows=1 width=4) (actual time=1239.361..1239.364 rows=1 loops=1)
Merge Cond: (pos_payment.pos_order_id = pos_order.id)
-> Sort (cost=2.46..2.46 rows=1 width=8) (actual time=0.021..0.022 rows=1 loops=1)
Sort Key: pos_payment.pos_order_id
Sort Method: quicksort Memory: 25kB
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.014..0.015 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..66770.53 rows=1282120 width=4) (actual time=0.013..1148.194 rows=1182463 loops=1)
Filter: (company_id = 1)
Planning time: 0.272 ms
Execution time: 1239.396 ms
(11 rows)
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order
"."company_id" in (1)) )) AND "pos_payment".id IN (1285508);
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.85..4.89 rows=1 width=4) (actual time=0.047..0.049 rows=1 loops=1)
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.027..0.028 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..2.45 rows=1 width=4) (actual time=0.018..0.018 rows=1 loops=1)
Index Cond: (id = pos_payment.pos_order_id)
Filter: (company_id = 1)
Planning time: 0.322 ms
Execution time: 0.080 ms
(8 rows)
closesodoo/odoo#77357
X-original-commit: 947c3bd72ee84c3a298e0c9c365340ddf16b307d
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Stanislas Sobieski (sts@odoo.com)
For searches, Odoo uses `unaccent` if it's available. On some
technical fields this is completely unnecessary and precludes the use
of indexes.
This PR provides:
* an opt-out (`unaccent = False`) on `String` and `Text` fields
* a warning if `unaccent` is enabled on *parent_path* fields as their
performance can be rather critical and not using the index is quite
an issue (note: the check that `parent_path` fields have been moved
outside of the check for their existence as we want to check that
the field is declared and correctly configured in all cases,
probably)
Task 2627454
Part-of: odoo/odoo#76436
`website` added its own sql-compatible version of
`get_unaccent_wrapper` in order to use `psycopg2.sql` for safety.
That is very sad.
Add a condition in the "standard" unaccent wrapper which checks
whether the input type is a `Composable`, and in that case return an
`SQL`. This increases the cost of the wrapper a tad but probably not
to a really sensible amount.
Task 2634851
Part-of: odoo/odoo#76436
Before this commit, the implementation crashes when the ancestors of a
record contain non-accessible records. This fixes the code to avoid it
to crash.
We also make the semantics of hierarchical searches more consistent in
this case: searching with operators 'child_of' and 'parent_of' returns
the subset of accessible records that satisfy the hierarchy operator.
We actually make it coincide with the results of the search when using
the "_parent_store" optimization.
closesodoo/odoo#73962
X-original-commit: 3e1b960acf3f6a726338476ba9e62e5ef5c84ef9
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Domain terms of the form `('field', 'not in', [...])` generate incorrect
queries for translated fields, example (model `res.country`, field `name`):
```
psycopg2.errors.SyntaxError: syntax error at or near "ARRAY"
LINE 1: ...M "res_country" WHERE "res_country"."name" not in ARRAY['No ...
```
The root cause is that the right part of the term is not converted to
tuple.
Observed during the upgrade request 16639
opw-2525553
closesodoo/odoo#73030
X-original-commit: 7c4db97e21f74a429e56f6cae4d4786ee74f7e22
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When building a domain `(=|!=)` with
a Binary field stored in attachment and
the right part of the domain is not null,
the full binary content is logged, giving
the logs a huge size.
Now, the content of the binary is cropped to
20 chars as it is useless to log more.
opw-2527629
closesodoo/odoo#70740
X-original-commit: 0ed46ae94b20b30c10025b2a458e1ecf92c2712d
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
A domain with a negated thruty leaf raises an exception here
https://github.com/odoo/odoo/blob/835197e48fbbee1c8ac35d52287537fa6f0bdd36/odoo/osv/expression.py#L614
due to
https://github.com/odoo/odoo/blob/835197e48fbbee1c8ac35d52287537fa6f0bdd36/odoo/osv/expression.py#L670
and the fact that `FALSE_LEAF != (1, "!=", 1)` and `TRUE_LEAF != (0, "!=", 1)`
Current wrong behaviour:
```
>>> distribute_not(['!', TRUE_LEAF])
[(1, '!=', 1)]
>>> is_leaf((1, '!=', 1))
False
```
This issue was noticed when adapting domains for fields that have been
removed during migration. Example (extract of) traceback:
```
Traceback (most recent call last):
File "/home/odoo/src/odoo/13.0/odoo/http.py", line 624, in _handle_exception
return super(JsonRequest, self)._handle_exception(exception)
...
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 677, in __init__
self.parse()
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 814, in parse
self.stack = [ExtendedLeaf(leaf, self.root_model) for leaf in self.expression]
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 814, in <listcomp>
self.stack = [ExtendedLeaf(leaf, self.root_model) for leaf in self.expression]
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 562, in __init__
self.check_leaf(internal)
File "/home/odoo/src/odoo/13.0/odoo/osv/expression.py", line 618, in check_leaf
raise ValueError("Invalid leaf %s" % str(self.leaf))
ValueError: Invalid leaf (0, '!=', 1)
```
closesodoo/odoo#68665
X-original-commit: e20c23658d15a135e5f35b0e8dec6e3202cecaab
Signed-off-by: Christophe Simonis <chs@odoo.com>
Assume field B depends on A, and searching on B is rewritten as a domain
that mentions A. If A is modified, and then we search on B, one has to
flush A to the database before searching.
closesodoo/odoo#57963
X-original-commit: b242b7b1382ee39ebefa58c556931b7dae77f669
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
By adding a parameter `_table_query` on the Model, we can now have views
that depend on the context. The query is used instead of the table name
in ORM operations. This allows to pre-compute some values to improve
performance, instead of storing context values in the database.
Co-authored-by: william-andre <wan@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
Make the method `_search` return a `Query` object, and make that object
generate a subquery when used on the right-hand side of a condition.
closesodoo/odoo#52403
Related: odoo/enterprise#10945
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Introduce better methods for introducing tables, join clauses and where
clauses to the Query object, and other methods to generate a complete
SELECT query. The Query object can also behave as a tuple of ids, where
the query is made on demand, and the result is memoized.
This avoids the formatting part of those methods (`browse` in the case
of `search`, `display_name` in the case of `name_search`), which is
useless in most cases.
This also enables further optimizations where `_search` and
`_name_search` return subqueries instead of record ids.
Steps to reproduce:
- install helpdesk, hr_expense (the demo data has a clear example of the bug)
- go to helpdesk > tickets > set a custom filter > "activities is set"
Previous behavior:
the results show the ticket with id=2 as if it had an activity set.
Because the search sql query collides with other activities set on other models
(hr.expense.sheet in this example)
Current behavior:
the sql query returns the proper result
opw-2273680
closesodoo/odoo#55297
X-original-commit: 6e90c41413f6a726c08071c71d01c738a6538079
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Saurabh Choraria