Currently, the domain expression on a many2one field with
negative operator + string values leads to a extra SQL request on the
target model of the many2one which is not optimal and lead to
performance degradation.
Example:
--------
With a simple model A with a one2many (`b_id`) targeting model B (with
a field `b_name` and `_rec_name = b_name`):
The `A.search([('b_id', 'not like', 'OneName')])` will
generate two SQL requests:
- A _name_search on model B and returning ids who match
`('b_name', 'not like', 'OneName')`.
- A other on the model A with the application of the result
of the first one `('b_id', 'in', <returning ids> + [False])`.
Solution:
---------
Avoid the extra SQL request (the first one) and use subquery instead.
We do it by translate the `('b_id', 'in', <returning ids> + [False])`
expression into `['|', ('b_id', 'in', _subquery), ('b_id', '=', False)]`
task-2671476
Part-of: odoo/odoo#78948
Seems like depending the pillow version, the output image size could differ
from one or two bytes. Now we check the expected size with a tolerance of
+/- 1 byte.
closesodoo/odoo#78965
X-original-commit: 42d3427975592a914b5a61aee304d499750e8afe
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
After some analyze on a lot of customer databases, seems like most of the time
their are performance probleme, and big store, it is due to a lot of big file
uploaded without reason. E.g. barcode, photo, ... a small one will be enough.
Now, we decided (in stable) to auto resize these pictures to 1920x1920px
by default and compress it with a quality of 80 when the source is bigger.
You can bypass this behaviour in your specific use case,
using a context key: 'image_no_postprocess' set to True.
You can disable the resize (and quality implicitely)
using an icp: 'base.image_autoresize_max_px' set to '0'.
You can change the default resize (1920x1920) format using an icp:
'base.image_autoresize_max_px' set to '<width>x<height>' (e.g. '1024x768')
You can change the default quality (80) using an icp:
'base.image_autoresize_quality' with a value between 0 and 100 where 0 skip it.
You can change the type of file that will be post process using icp:
'base.image_autoresize_extensions' (subtype of the mimetype comma separated).
Api of image has not be changed in this commit, only refactored to allow to
work with image directly without the need to encode/Decode in base64 the raw.
We decide to keep 1920x1920 by default instead of 1080p to avoid to resize
portrait picture in 1080px and stay consistent with field image_1920 that
return a 1920px image for width or height whatever the orientation.
+ fix some lint diff for ci style in master
closesodoo/odoo#78556
X-original-commit: d9ce0507960f247e1187baf7bd8399f90be237aa
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Before this commit, trying to create a field related to another field in
the same batch raises an error, because the computation of the field
'related_field_id' cannot find the target field in the registry.
The new approach consists in getting the target field from the database
instead. It works when creating fields in batch, because all records
are inserted into the database before the field is computated.
This costs extra queries, but it allows to batch field creation and
avoids multiple calls to setup_models() when creating a model with
partner field in Studio. (see odoo/enterprise#21746)
The creation of a simple custom related field (related="x.y") costs 3
additional queries to determine the target field:
- 2 queries to get a given field on a given model
- 1 query to get the first field's comodel
Part-of: odoo/odoo#78514
The real overhead of creating a related field is actually two queries:
- one query in update_db_related, the purpose of this test
- one more query for _compute_related_field_id in flush()
With the previous version of this test, one prefetching done in the
first create was not present in the second create since it was in cache.
The fix clears the caches to make the creation of both fields in the
same situation, such that query counts are compared in a fair way.
Part-of: odoo/odoo#78514
This commit fixes the case where you have some text after a comment.
Until know, we miss it. Now we render the tail part.
```
<t>
<!-- HIDE Text 1 -->
Text 1
<p>SHOW Text 2</p>
</t>
```
After the fix, Text 1 is correctly rendered
This commit fixes#76628closesodoo/odoo#78782
X-original-commit: 566360b07b4ff6c7290f264d9064400df2b06ca5
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
This is a followup on b6c688caa2, which
has introduced a new bug.
Consider two groups A and B in a given category, where B implies A and
also A.id > B.id. For the sake of simplicity, assume that A.id=2 and
B.id=1. Following the commit above, the name of the group selection
field will be 'sel_groups_1_2'.
Consider a user in group B. Because B implies A, this means that the
user now belongs to both A and B. A call to read() on that user returns
field 'sel_groups_1_2' with value 2, which on the form view appears as
the user belonging to A only.
The error is in read(), which assumes that the group ids in the field
name correspond to the implication order of the groups, which is no
longer the case since b6c688caa2.
closesodoo/odoo#78662
X-original-commit: e73c406671a51e9c1c4872a9e85eebe5656ea38e
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Because test cursors are implemented using savepoints, they should *at
most* be nested (ideally they would be strictly sequenced).
If upon being closed a test cursor finds a *different* un-closed
cursor at the top of the stack, one of its followers / children /
descendants was not closed before it, which is a problem.
The initial version of this would look for the cursor being closed in
the stack but this could lead to incoherent cursor stacks and errors
related to the management of the cursors stack, even though it only
exists for reporting reasons.
closesodoo/odoo#76243
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
This fixes a performance issue: ORDER BY clauses in subqueries can make
the query unexpectedly slow. We should avoid this situation, since
ORM-generated queries have an ORDER BY clause which is not relevant in
the context of a subquery.
The following example was found:
SELECT "pos_payment"."id" AS "id"
FROM "pos_payment"
WHERE ("pos_payment"."pos_order_id" in
(SELECT "pos_order".id
FROM "pos_order"
WHERE ("pos_order"."company_id" in (1))
ORDER BY "pos_order"."id"))
AND "pos_payment".id IN (1285508)
Here are the query plans made by PostgreSQL on this query with and
without the ORDER BY clause. The query time went from 1240ms to
0.402ms, which is 3000 times faster!
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order"."company_id" in (1)) ORDER BY "pos_order"."id" )) AND "pos_payment".id IN (1285508);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Merge Semi Join (cost=2.88..82726.85 rows=1 width=4) (actual time=1239.361..1239.364 rows=1 loops=1)
Merge Cond: (pos_payment.pos_order_id = pos_order.id)
-> Sort (cost=2.46..2.46 rows=1 width=8) (actual time=0.021..0.022 rows=1 loops=1)
Sort Key: pos_payment.pos_order_id
Sort Method: quicksort Memory: 25kB
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.014..0.015 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..66770.53 rows=1282120 width=4) (actual time=0.013..1148.194 rows=1182463 loops=1)
Filter: (company_id = 1)
Planning time: 0.272 ms
Execution time: 1239.396 ms
(11 rows)
EXPLAIN ANALYZE SELECT "pos_payment"."id" as "id" FROM "pos_payment" WHERE ("pos_payment"."pos_order_id" in (SELECT "pos_order".id FROM "pos_order" WHERE ("pos_order
"."company_id" in (1)) )) AND "pos_payment".id IN (1285508);
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.85..4.89 rows=1 width=4) (actual time=0.047..0.049 rows=1 loops=1)
-> Index Scan using pos_payment_pkey on pos_payment (cost=0.43..2.45 rows=1 width=8) (actual time=0.027..0.028 rows=1 loops=1)
Index Cond: (id = 1285508)
-> Index Scan using pos_order_pkey on pos_order (cost=0.43..2.45 rows=1 width=4) (actual time=0.018..0.018 rows=1 loops=1)
Index Cond: (id = pos_payment.pos_order_id)
Filter: (company_id = 1)
Planning time: 0.322 ms
Execution time: 0.080 ms
(8 rows)
closesodoo/odoo#77357
X-original-commit: 947c3bd72ee84c3a298e0c9c365340ddf16b307d
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Stanislas Sobieski (sts@odoo.com)
Because the field is stored it doesn't make much sense to have a
context dependency and that can cause significant issues. However
`name_get` looks pretty innocent and it's not necessarily insane to
have a context dependency *there* (or even in the average
`display_name` as they're normally non-stored computed fields).
`_compute_display_name` previously blanked (blacklisted) just a few
context values, but doing the opposite is probably the better idea.
See also: odoo/odoo#68946
OPW-2453956
Task 2500978
closesodoo/odoo#76469
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
For searches, Odoo uses `unaccent` if it's available. On some
technical fields this is completely unnecessary and precludes the use
of indexes.
This PR provides:
* an opt-out (`unaccent = False`) on `String` and `Text` fields
* a warning if `unaccent` is enabled on *parent_path* fields as their
performance can be rather critical and not using the index is quite
an issue (note: the check that `parent_path` fields have been moved
outside of the check for their existence as we want to check that
the field is declared and correctly configured in all cases,
probably)
Task 2627454
Part-of: odoo/odoo#76436
By activating the profiler debugger, the two qweb options is added. The
qweb templates that need to be rendered are compiled into a new function
to add instructions for saving data.
`Add qweb directive context`
It's a sub-option of "Record sql" or "Record traces", add some context on
thread at current call stack level. This context stored by collector
beside stack and is used by Speedscope to add a level to the stack with
this qweb directive information.
```
directive=t-call='website.layout', xpath=/t/t
t_call_content
directive=t-foreach='5' t-as="'a', xpath=/t/t/div/div
directive=t-esc='website.search([])', xpath=/t/t/div/div/t
execute
```
`Record qweb`
Add profiling data used by ProfilingQwebView widget. In the `ir.profile`
form view, the widget display the duration and number of sql of every qweb
directives with the xml of templates. Every xml is recorded to be
consulted even if the user change the xml templates.
```xml
<t t-call="website.layout">
<div>
<div t-foreach="5" t-as="a">
<t t-esc="website.search([])"/> <!-- will display 5 separate requests -->
</div>
</div>
</t>
```
closesodoo/odoo#74712
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Here is the idea: when creating a view V that inherits another view, and
V only adds elements in the parent view, only the elements added by V
must be validated. The other elements are not modified, and thus do not
need to be validated again. However partial validation cannot be done
when modifying an existing view, or when the view replaces elements in
the parent view. In such cases, the method _check_xml() validates the
whole combined architecture.
The time spent in method _check_xml() to install a database from scratch
with module sale_management went from 2.730s to 2.243s (-18%), and the
number of queries made in the method went from 2283 to 1614 (-29%).
closesodoo/odoo#75796
Related: odoo/enterprise#20633
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The XML-RPC interface has a compatibility shim for binaries as
historically Odoo has returned "binary" data as base64 strings. To
avoid breakages during the Python 3 transition, the shim was
introduced to decode the output binary data (under the assumption that
it'd be ASCII-compatible).
In the case where the data is *not* ascii-compatible, however, it can
generate invalid XML documents: "C0" control codes (with the exception
of tab, LF, and CR) are not valid in XML 1.0 (which XML-RPC is an
application of), however they're perfectly valid string characters and
the standard library's marshaller does not check for them, embedding
them directly in the output document and breaking the client's
decoding.
Work around the issue by replacing such binary data with an empty
string.
While at it, move the bytes shim to the customized marshaller, this
way everything's at the same place and it's not necessary to waste
time trying to understand why the marshaller is just not calling what
it's supposed to call.
Fixes#61919closesodoo/odoo#75973
Forward-port-of: #75952
Forward-port-of: #74699
X-original-commit: 1a0b3f7
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Since de077243 the iter(False) will occur when trying to render a
speedscope output with `use_context=False` since the context given in
parameter of `stack_to_ids` won't be an iterable.
This simple fix avoid this by always giving an empty iterable
in this case.
closesodoo/odoo#75851
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Purpose
=======
This commit improves the UI of the mail group module (backend and portal
view), fix some bugs and clean the code.
Bug 1
=====
If a user click 2 times on a confirmation link, an error is raised.
Bug 2
=====
A bug about the website snippets has been fixed;
- Add the snippet to a group in which you are member
- Change the group of the snippet to a group in which you are not member
- The snippet still show "Unsubscribe"
Bug 3
=====
In the portal view, when messages are filtered by month, messages which
are created in the last day of the month appear also in the next month.
Specification
=============
UI
--
Show the Allow / Ban buttons whatever is the state of the message, so
the user can ban / whitelist a message even if there's no pending email.
When we receive an email, remove the "Mailing List" footer. This was a
big issue because when some users reply many times, the footer was
duplicated on each answer.
Improve the form view of the mail group:
- On non-moderated group, show all messages instead of only pending
message
- Show the guidelines stuff on non-moderated groups
Improve the portal view
- show the attachments under every message (not only on opened message)
- improve the template to confirm the subscription of the user
- improve the portal view for mobile device
- do not show rejected messages, even for admin
- do not show the button "X replies" when the mode is not thread
Technical
---------
Split the route "/groups/subscription" into "/group/subscribe" and
"/group/unsubscribe"
Correctly quote the parent email for the emails sent by web version of
Outlook (with the HTML id "divRplyFwdMsg").
ACLs
----
Now only administrators or responsible / moderators can write / unlink
the group.
Fix an ACL issue on the group member where the responsible of a
non-moderated group could not access the members of the group.
Members with same email
-----------------------
Improve the behavior when multiple members have the same email address
(this situation is possible because there's no unique constraint on the
email field of the <res.partner> model.) We now return the most
appropriate member for the given email address and partner.
When unsubscribing by clicking on the confirmation link that we received
by email, remove all members with the same email address.
Links
=====
Task-2599676
See odoo/odoo/pull/73640
closesodoo/odoo#73640
Related: odoo/upgrade#2774
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The initial behaviour of ExecutionContext was to
- add the context for the current level of the stack on enter
- remove the context at this same level on exit.
When two ExecutionContext are nested at the same stack level:
- the first one adds it's context;
- the second one overrides the current context;
- the second one removes it at the end;
- the first one fails to remove the context at the same level a second time.
def call():
with ExecutionContext(foo=1):
with ExecutionContext(bar=2):
execute()
In this case we could imagine to combine both Execution managers in one:
def call():
with ExecutionContext(foo=1, bar=2)
execute()
But the semantics is different: ExecutionContext should add one level to
the stack, instead of two. In the first example above, we expect two
additionnal levels:
call
foo=1
bar=2
execute
A simple solution would be to transform the value of the context at some
level to a list of dict, but this would make the copy more difficult.
The decision was taken to change the context storage strategy, going
from a dict where the keys are the level to a tuple of tuples where the
first element is the level.
{3: {foo: 1}, 4: {bar: 2}} => ((3, {foo: 1}), (4, {bar: 2}))
The first benefit is to allow multiple contexts at the same level. The
second one is that we don't need to copy the data structure, since
tuples are immutables and the dicts are coming from kwargs. This means
that saving the context is faster, and the tuple is shared between
multiple samples. This is interesting if we assume that we do more
samples than exec_context mutations.
closesodoo/odoo#75687
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
PR #42723 changed the public method context_get on res.users return a
frozendict instead of a classic CPython dict.
Since the default XMLRPC Marshaller does not know how to serialize a
frozendict, this would result in errors when calling the method directly
via the XMLRPC layer.
This commit solves this issue by defining a marshalling procedure for
frozendict objects in our custom OdooMarshaller class, this simply
converts the frozendict into a dict and uses the default dict-to-xml
dump procedure (dump_struct).
Fixes#75412closesodoo/odoo#75478
X-original-commit: e80a299ed910cacac4aa4e64164909af36ce62ad
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Automatically activate `group_multi_currency` if there is more than one active currency,
deactivate it if there is only one active currency
retain sale/pos feature of activating group_product_pricelist
adapt tests accordingly
Task 2610735
closesodoo/odoo#74396
Related: odoo/enterprise#20106
Signed-off-by: Laurent Smet <smetl@users.noreply.github.com>
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
Purpose
=======
Revert the commit a757cab857
because the rewriting of the FROM of the emails sent will
be done in a smarter way in master.
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
This is the same behavior as javascript. Which had been changed during
the refactoring of qweb. t-esc asked for escaping whatever the content.
However, a lot of templates still use t-esc. It is easier to go back to
the previous functionality.
odoo/odoo#74678 fixed the universal traceback which would happen any
time an HTML field would be read, however it's incomplete because:
* it was tested on a field which wasn't HTML in 14.4
* and no markup was put in the field anyway
So the markup being embedded in the XML-RPC document unescaped was
missed, leading to:
* corrupted (partial) responses when the HTML content is XML-valid,
depending on the exact API of the client it might only return the
first or last text segment, or all the text without markup, or
something else
* outright deserialisation error on XML-invalid content (e.g. void
elements like <br>)
The cause being that `Markup` overrides all `str` methods to first
escape their parameters before actually applying on the object. This
means `xmlrpc.client.escape` would basically do nothing, then would
concatenate the `Markup` into the document (stringifying it) and send
the entire thing on its way.
Fixes#74884closesodoo/odoo#74957
X-original-commit: 8a0f7d820cb2296f410be215ffeadd1cefae174c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
In 01875541b1, HTML fields (and various
methods) were made to return markupsafe.Markup objects.
However at the time I didn't consider that XML-RPC serialization
remains based on type *identity*, and thus the `Markup` object would
not serialize outbound through XML-RPC, and would blow up instead.
This should fix the issue, by serializing Markup objects as str.
closesodoo/odoo#74684
X-original-commit: 165bd3bf88becc42416e49ad2f430e90675219fa
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes
closesodoo/odoo#68299
Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
* Remove AST in favor of pure Pyhon. This should make it easier for
developers to understand and create new directives because they do not
need to know AST.
* Remove `t-call-options` as it has been merged into `t-options` for more
consistency. Support for t-call-options is retained.
* Use generators for lists. This increases performances as the rendering
can be sent directly without having to wait for the creation of the
entire list.
* Optimize expressions runtime computation by pre-computing the static
parts.
Example:
'<' + 'div' + '>' + '<' + dynamic_value + '>'
Now compiles as:
'<div><' + dynamic_value + '>'
This commit adds a new attribute mode for position 'replace' to the xpath feature.
This mode can take 2 values:
- 'outer' (default mode if not provided) that will replace the sibling target
- 'inner' that will preserve the sibling target
If base arch is:
```html
<p>
<field name='x'>yyy</field>
</p>
```
`<field name='x' position="replace" (mode="outer")>zzz</field>`
```html
<p>
zzz
</p>
```
`<field name='x' position="replace" mode="inner">zzz</field>`
```html
<p>
<field name='x'>
zzz
</field>
</p>
```
Mode inner is useful for theme or render flat content, but not recommanded to
be used in page editable since we ignore the inherit-branding part until now.
task-2172208
closesodoo/odoo#48679
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Co-authored-by: Okan SUMER (osu) <osu@odoo.com>
Co-authored-by: Jérémy Kersten <jke@odoo.com>
`load_views` calls `get_bindings` to render Action and Print menu. Before this
update, it read all action fields, including heavy computed fields
like `search_view` (which calls fields_view_get). But in fact just few fields
are used.
This slighly improves response time and data size.
closesodoo/odoo#73983
Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
Before this commit, the implementation crashes when the ancestors of a
record contain non-accessible records. This fixes the code to avoid it
to crash.
We also make the semantics of hierarchical searches more consistent in
this case: searching with operators 'child_of' and 'parent_of' returns
the subset of accessible records that satisfy the hierarchy operator.
We actually make it coincide with the results of the search when using
the "_parent_store" optimization.
closesodoo/odoo#73962
X-original-commit: 3e1b960acf3f6a726338476ba9e62e5ef5c84ef9
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, some tests were indirectly using demo data.
For example `env['res.partner'].search([])` would return demo data in
addition to the partners created for the tests.
This commit changes that by ensuring that the records used are limited
to the records made for the tests. Some values have been adjusted to
account for the lower amount of available records.
closesodoo/odoo#72553
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
It seems quite easy to have a void content in the currently new editor
that actually holds a lot of undesired information, notably a font
tag. This is annoying when having behavior based on an html field
being empty or not. In this commit we therefore improve definition
of an 'empty' html field.
Task ID-2532529
PR odoo/odoo#71793
Domain terms of the form `('field', 'not in', [...])` generate incorrect
queries for translated fields, example (model `res.country`, field `name`):
```
psycopg2.errors.SyntaxError: syntax error at or near "ARRAY"
LINE 1: ...M "res_country" WHERE "res_country"."name" not in ARRAY['No ...
```
The root cause is that the right part of the term is not converted to
tuple.
Observed during the upgrade request 16639
opw-2525553
closesodoo/odoo#73030
X-original-commit: 7c4db97e21f74a429e56f6cae4d4786ee74f7e22
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
With commit odoo/odoo@83ffe8b we ensured that while creating a child contact, it
takes language from its parent by default if any, otherwise falls back to DB
lang.
The fix was done with help of `default_get` method. However after merge of
odoo/odoo#55995 `default_get` is now called through the onchange for o2m
fields. Now we don't get the default value of `parent_id` here which
re-introduced the bug.
This commit fixes the behavior by setting the language from onchange
while creating the child contact and thus (once again) making the
language from parent contact prevail on DB lang.
We also re-use default_lang coming from parent in form view, which partially
reverts odoo/odoo@83ffe8b .
TaskID-2416922
closesodoo/odoo#72960
X-original-commit: 4d9b85bd07e487e7843dc458332dede36c2889c2
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Before this commit: when partner is only created with name(i.e. without
address) and when 'address_inline' is passed in context then partner name shows
with unnecessary commas.
After this commit: partner name will not have unncessary commas, as we removes
unncessary '\n' in case of 'address_inline' passed in context.
task-2502597
closesodoo/odoo#72869
X-original-commit: c08082d89782b7675e8e649cba15ca9e03e40077
Signed-off-by: Simon Genin (ges@odoo) <ges@odoo.com>
The commit e123aa2f30 optimizes the call
to fields_get() made in the NameManager to post-process the combined
arch of a view. The optimization consists in avoiding translations,
with the assumption that the result is only used for validations. It
turns out that the regular post-processing actually returns the field
descriptions as part of the result of fields_view_get().
We fix this mistake by keeping the optimization only when the flag
'validate' is true, so that the regular post-processing of a view
returns field descriptions with the expected translations.
closesodoo/odoo#72216
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The `mode` field in used in `_combine` to distinguish `primary` views
from `extension` ones. The field has been retrieved as part of the query
done in `_get_inheriting_views` and manually placed in the cache so the
ORM doesn't perform an extra query just to retrieve the mode.
Using our benchmark (render_all_views test tag, website_sale, mrp,
contacts, crm and timesheet installed), the reported time was 18.46s
with 17890 queries performed without this improvement, it is 17.82 with
17340 queries with.
Closes#72096closesodoo/odoo#72127
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The view methods `get_inheriting_views_arch`, `apply_view_inheritance`,
`_apply_view_inheritance` and (previously `read_combined`)
`_get_combined_arch` were complicated and not optimal performance-wise.
The previous approach to retrieve views and combine them was to perform
it recursively, one primary view at a time. This new approach retrieves
in a single SQL query the whole tree of views to combine. The order of
view combination has been reproduced to be fully backward-compatible.
We also introduced various optimisations such as pre-fetching and
caching.
**get_inheriting_view_arch**
The method have been renamed `get_inheriting_views` as it returns a
recordset of views instead of the xml architecture of each view.
During an upgrade, only the views that have been fully upgrade already
are safe to be used. The views returned by the query are filtered to
remove any not-upgrade-yet view. Because this part is highly specific,
it has been isolated in a dedicated private method.
View extensions can be discarded according to user groups. The
retrieval of groups on views is also made by the same query. This
reduces the number of SQL queries, and makes the filtering faster.
**apply_view_inheritance**, **apply_view_inheritance**
The functions were responsible to build a hierarchy of views and to
iterate it to combine the views.
The hierarchy building have been moved to `_get_combined_view` and the
actual combination have been moved to a new method `_combine` and
documented.
Task: 2541579
Meta task: 2463632
Co-authored-by: Fabien Pinckaers <fp@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The method read_combined() has been deprecated in favor of explicit
calls to read() and get_combined_arch().
There are places where we call _get_combined_arch() instead, in order to
avoid parsing the XML that has just been serialized. This saves useless
serialization-deserialization.
Task: 2541577
Meta task: 2463632
Co-authored-by: Fabien Pinckaers <fp@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
A test that calls fields_view_get() on all models while measuring its
performances. Use it along with py-spy and load the generated file in
<https://www.speedscope.app/>
py-spy record -F -r 200 --format speedscope -o speedscope.json -- \
python3 odoo-bin -d <db> --test-tags render_all_views --stop-after-init
Task: 2463632
Co-authored-by: Raphaël Collet <rco@odoo.com>
Replace text fields to html fields as we have our own 'OdooEditor'.
Indeed, it gives more options to users in the way they format their
content without weighting too much on the UI
(tools appear on demand and not by default).
In this commit we replace report_footer and report_header field of
"base.document.layout, res.config.settings" model as
they are related fields of res.company models field
report_footer, report_header.
Task Id: 2499504
X-original-commit: 4c474d0d2055efe81fdc21502aa5957fe80af7ef
Before this commit there were few cases in which
is_html_empty method was not working as our expectation.
In cases such as "<p class=""><br/></p>", "<p id=""><br/></p>"
and many other cases.
In this commit we improves regex of is_html_empty method.
Mow from this commit all kind of attributes will be consider
in this method.
In this commit we also have added is_html_empty in mail template, portal
values and in report values also for rendering templates.
Task id: 2499504
X-original-commit: fd0a05f2955b9f7e9ae7233afebfd6240c9244dd
This commit adds tooling to profile performance and save execution by
saving stack traces and queries to a file/database in specific format.
----------
Collectors
----------
For now, three different profiling modes (aka Collectors) are available
even if a last once should be introduced by @Gorash to profile qweb
execution.
- SQLCollector (or 'sql'): Saves the current stack trace and the query
every time Cursor.execute() is called. Any query executed on the thread
will be collected, no matter the cursor.
- PeriodicCollector (or 'traces_async'): Saves the stack trace every
'interval' seconds using a parallel thread to profile the caller thread.
The python implementation was optimized to minimize impact on
performance while remaining portable and easy to enable/disable
inside a odoo execution. Higher the frequency (lower the interval),
more impactful the profiling will become on the execution and increase
memory usage. From last experiments, 1ms looks to be a good minimum for
short executions.
- SyncCollector (or 'traces_sync'): Saves the stack trace every function
call/return. This collector is obviously quite impactful on performance
and can quickly overload the memory for long executions, but this is
quite useful to understand the precise path followed by some short
executions. Any time related information will be almost irrelevant with this
collector.
A base Collector defining minimal collectors features can easily be
extended to create custom collectors if needed.
----------------
Profiler & Usage
----------------
Collectors are not supposed to be used by themselves, but should be
given to a Profiler. The Profiler will synchronize collectors starts and
stop, and manage saving them to a file of in a ir_profile in the
database.
Exemple of usage:
```
with Profiler():
do_stuff()
```
This simple example will use the default collectors (sql and
traces_async) and save them to the database. The database is defined
automatically from current_thread 'dbname' if available.
Example of usage:
```
with Profiler(collectors=['sql'], db=False, path=/home/user/logs/do_stuff_profile/{time}):
do_stuff()
```
This more complex example disable the default behavior consisting
to save to the database, gives a path where the profile will be saved
and specify to only use the 'sql' collector. Note that
collectors=[SQLCollector()] would have the same behavior since
Collectors can be either a Collector instance or a string describing the
desired collector. This allows to define custom params for the
collectors and use custom collectors if needed.
Note that it is always possible to get results after execution without
saving it since they are available on the profiler.
```
with Profiler(collectors=['sql'], db=False) as p:
do_stuff()
print(len([None for entry in p.collectors[0].entries if ...]))
```
Profiler will also save the stack below the profiler start point, and
collectors will only collect the part of the stack over this stack.
This is a good way to reduce collectors CPU and memory usage.
Collected entries will be saved as follows:
```
[{
'start': 2.0,
'context': {},
'stack': [
['path_to_file', lno, 'func_name', 'line_content'],
...
],
},
...
]
```
SQLCollector will add three additional keys on each entry:
- query (query without parameters)
- full_query (mogrified query with parameters)
- time (the 'exact' execution time of the query)
----------------
ExecutionContext
----------------
A last tool, ExecutionContext, allows to define some context on some block of code:
Example of usage:
```
def process_modules(modules)
for module in modules:
with ExecutionContext(module=module): # note the 'not linter frienldy but still convenient' 2 spaces indentation
do_stuff(module):
```
This context will automatically be added in the stack as a virtual frame between
process_modules and do_stuff in order to split do_stuff from one single frame to
one frame per module.
----------
Speedscope
----------
The saved data are in a simple json format easy to analyze, but can't be visualized in
speedscope as they are. A utility class `Speedscope` can be used to generate a format
readable by speedscope. The used format is actually the format defined by speedscope,
meaning that all features should be available using it.
The output format is evented, meaning that we need to transform a list of samples
(a list of stack) to a list of event (going in/out a frame).
This is the main task of the Speedscope, as well as combining samples from different
sources, to display SQLCollector and PeriodicCollector results mixed together.
When stored on an ir_profile, the default speedscope generation can easily be generated
with the speedscope computed field.
This class can be used as it is but will mainly be useful for the next commit.
Special thanks to @rco-odoo for the in depth review and @Gorash for support.
Florent de Labarre