Purpose
=======
On the res.config.settings model for example, calling execute without
having changed a single parameter will make several writes on
ir.default records.
However, even if the written and the current values are the same,
the cache will be invalidated several times.
Part-of: odoo/odoo#82999
Purpose
=======
Several actions are done even if nothing has changed on the configuration.
Example:
Writing on a cron the same value makes a dummy write-lock on the table
...
Part-of: odoo/odoo#82999
Gain of ~80% of the SQL queries & ~50% of execution time in a database
with test_main_flows & dependencies installed.
On a benchmark of 100 calls of (load_menus(True) + load_menus(False))
* 266 > 56 queries
* 0.38 > 0.21 sec of execution (mean)
Note that this only impacts the first request since this method is cached.
Unless user groups (or debug status) are different, the cached results
are returned.
closesodoo/odoo#83172
Signed-off-by: Raphael Collet <rco@odoo.com>
When we are only trying to know accepted raw values, there is no
need to fetch the translations for the description part of the
selection tuples.
This performance problem was first observed on res.config.settings
record creation, where the values are pruned to avoid writes and
invalidation for unmodified related values (cf create override
in base/models/res_config.py & 5a4e7b711e).
After more analysis, the 'problem' also impacts record updates on
any related field (readonly or not).
* Res.config.settings performance gains:
In a database with test_main_flows (& its diverse dependencies),
this reduces by ~50% the number of queries done when triggering
the creation of res.config.settings record through the interface
(aka with all the related updatable fields receiving their current
value as create value).
For the creation of 150 res.config.settings records, with full cache
invalidation between each record creation, we notice a gain of:
* 11853 to 6303 total queries
* 11.2s to 7.5s total creation time
Part-of: odoo/odoo#83104
* batch module data prefetch in res.config.settings flows.
* manipulate ir.module.module records instead of a strange list of tuples
* Only keep the missing module names logic where useful (useless for the
res.config.settings scope, modules must exists to be referenced by
a settings field).
* drop the support of module fields of type selection, they are not used anymore,
it's not even sure they still work and this feature doesn't bring much added value anyway.
Part-of: odoo/odoo#83104
The issue has been introduced by 1fe4b0cc1981382cc3d944ec44276f0aee90945d.
A simple example is an element with attribute "string" like
<field name="foo" string=" "/>
When the translation process is parsing the value of the attribute as
some HTML fragment html.from_string(" "), it raises an exception. This
was not a problem before the commit above, but it now is. For instance,
updating the view "event.view_event_form" triggers the error.
closesodoo/odoo#84144
X-original-commit: 5ffc1704162d8918985964b08f4b2b49e6054958
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Christophe Simonis <chs@odoo.com>
Whenever a record is modified in a demo/data file,
create will still be called, but with an empty list of values.
This flow is correctly managed in the core orm code, but a lot of
custom logic in create overrides is triggered for nothing.
This commit makes sure that create is not called with an empty list
when there is no record to create (i.e. only records to update)
during files imports (including modules demo/data loading).
Part-of: odoo/odoo#84055
Before this commit the phone fields inside forms did start in a blank
state.
After this commit the empty phone fields are pre-filled with "+" and the
country phone code.
The country is determined based on the IP address with the GeoIP
database. The phone code is obtained from `res.country` where it is
kept in cache.
task-2463563
closesodoo/odoo#78068
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Co-authored-by: Benoit Socias <bso@odoo.com>
When comparing existing translations to new source value
html tag would invalidate existing translations.
The html tags used for styling (coloring, italic, ...) should not
invalidate the translation as the meaning of the text as not changed.
Changed the logic to compare translations based on textual content only.
+ add some tests.
task-2667950
closesodoo/odoo#83895
X-original-commit: 1fe4b0cc1981382cc3d944ec44276f0aee90945d
Signed-off-by: Antoine Guenet <age@odoo.com>
Signed-off-by: Geelen Sébastien (sge) <sge@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
Co-authored-by: xmo-odoo <xmo@odoo.com>
QWeb is the primary templating engine used by Odoo. It is an XML
templating engine and used mostly to generate XML, HTML fragments and
pages.
To create new XML template, please see :doc:`QWeb Templates documentation
<https://www.odoo.com/documentation/15.0/developer/reference/frontend/qweb.html>`
In **input** you have an XML template giving the corresponding input
etree. Each etree input nodes are used to generate a python function.
This fonction is called and will give the XML **output**.
The ``_compile`` method is responsible to generate the function from the
etree, that function is a python generator that yield one output line at a
time. This generator is consumed by ``_render``. The generated function is
orm cached.
In the graphic below you can see theresume of the call of the methods
performed in the IrQweb class.
Odoo
┗━► _render (returns MarkupSafe)
┗━► _compile (returns function) ◄━━━━━━━━━┓
┗━► _compile_node (returns code string array) ◄━━━━━━━┓ ┃
┃ (add technical directives: t-inner-content, t-tag) ┃ ┃
┣━► _directives_eval_order (defined directive order) ┃ ┃
┃ ┃ ┃
┣━► _compile_directives (recursive) ◄━━━━┓ ┃ ┃
┃ ┣━► _compile_directive ┃ ┃ ┃
┃ ┃ ┗━► t-if ━━► _compile_directive_if ━┫ ┃ ┃
┃ ┃ ┗━► t-foreach ━━► _compile_directive_foreach ━┫ ┃ ┃
┃ ┃ ┗━► t-* ━━► ... ━┛ ┃ ┃
┃ ┃ ┗━► t-inner-content ━━► _compile_directive_inner_content ◄━━━━┓ ━┛ ┃
┃ ┃ ┗━► t-tag ━━► _compile_directive_tag ━┫ ┃
┃ ┃ ┗━► t-call ━━► _compile_directive_call ━┫ ━━━┛
┃ ┃ ┗━► t-out ━━► _compile_directive_out ◄━┓ ━┫
┃ ┃ ┗━► t-field ━━► _compile_directive_field ━┛ ┃
┃ ┃ ┃
┗━━┻━► _compile_static_node ━┛
Part-of: odoo/odoo#81024
Some html field are not in their ideal style.
As the style for html fields is now dependent of
where you are in the xml view ( in group or not),
we have to adapt some views and flags some fields
to ensure they have the correct look.
This change is only be a visual enhancement
and should not prevent the function of said html fields
even if the views are not updated.
task-2637488
# Conflicts:
# addons/mail/wizard/mail_compose_message_views.xml
# addons/website_slides/views/slide_channel_views.xml
# Conflicts:
# addons/sale/views/sale_views.xml
closesodoo/odoo#83792
Related: odoo/enterprise#23911
Signed-off-by: Antoine Guenet <age@odoo.com>
Signed-off-by: Geelen Sébastien (sge) <sge@odoo.com>
This commit brings the ability to use some feature detection
mechanism used in Odoo. The jquery.touchSwipe library
implementation requires the browser to get ontouch* events from
the window to be available to assert that the browser has touch.
The switch will make the mechanism to work as expected, and
allow tests to use touch events while being correctly executed
because the library expects such events.
closesodoo/odoo#83588
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Add the possibility to force the two-factor authentication for all users,
using a two-factor authentication by email
when the 2FA using an Authenticator app is not configured for the user.
Two possibilities:
- Force the 2FA only for employee users using the system parameter `auth_totp.policy=employee_required`
- Force the 2FA for all users, employees and portals, using the system parameter `auth_totp.policy=all_required`
closesodoo/odoo#83750
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Purpose
=======
A field `use_google_gmail_service` has been used in stable to define
a mail server which use Gmail authentication.
But now that the fields `smtp_authentication` exists, we want to use it
to simplify the mail server form view. For the incoming mail server,
the field `server_type` will be used for the same purpose.
Add a new field to have the option to install `google_gmail` in the
main settings page.
Task-2170676
closesodoo/odoo#83413
Related: odoo/upgrade#3199
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The `_create` don't batch the SQL request to insert of data which it
isn't efficient we there is a several record to create.
Batch the SQL insertion and measure performance
depending of batch size to avoid any regression (see result in the
PR: https://github.com/odoo/odoo/pull/80961).
task-2299314
closesodoo/odoo#80961
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
The `check` decorator in `sql_db.py` was on a lot of `Cursor` methods.
It checks if the cursor is close before be using a the method.
Remove it because:
- It is completly redundant because `psycopg2` do already the job
to check the cursor before usage.
- It complicated the call stack and lead to a small overhead of
highly use methods (can be more than 1% of the execute call)
- Also the nature of
the Error isn't correct: raise `OperationalError`
(https://www.psycopg.org/docs/module.html#psycopg2.OperationalError)
instead of `InterfaceError`
(https://www.psycopg.org/docs/module.html#psycopg2.InterfaceError).
Part-of: odoo/odoo#80961
It was only used one time on a unused field.
We don't want to keep useless fields anymore,
the migration is done for that.
task-2735546
odoo/upgrade#3194closesodoo/odoo#82727
Signed-off-by: Raphael Collet <rco@odoo.com>
Issue:
`_write` checked that the number of modified row in DB was equal to the
number of ids in the RecordSet (raise `MissingError` if not).
This behavior was only used by the ORM to retry
(with exists() on the RecordSet before) if the Missing Error raised.
Then it makes the job a second time (for no reason).
Then:
- Remove the check of number of row (and simplify the call of `_write`)
- Remove the useless return value (always `True`)
- Remove the `set(` before the `split_for_in_conditions`, it adds
randomness for nothing because the `_write` should be call without
duplicate id (even if it is not the case, we don't care).
task-2735546
Part-of: odoo/odoo#82727
The `column_format` of the `Field` class was unused and create useless
noise in the ORM. Then remove it and simplify some flows.
task-2735546
Part-of: odoo/odoo#82727
The `_sequence` was only used for the insertion of data in the DB for
the `id` value. But in Odoo, the `id` is always a `SERIAL`, then the
PostgreSQL fill already the value by the right SQL sequence. Then,
avoid doing the job of the DB by ourselves.
Sadly, we need to manage the case when we create an empty record to
always get a valid query.
task-2735546
Part-of: odoo/odoo#82727
As an overridable _neutralize model method was added in a previous
commit, the method is now implemented for various models.
closesodoo/odoo#67825
Related: odoo/enterprise#19042
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When debugging a database, it's sometimes desirable to neutralize it to
avoid undesired side effects like sending emails.
This commit adds a generic method `_neutralize` on models to neutralize
the current database. This method has to be overridden by modules models
that needs to be neutralized.
A new CLI command `neutralize`is now available that will permit to
neutralize a database from the command line before starting
investigations.
Pay attention that the neutralize operation cannot be reversed and should
only be used on a database copy.
Task id: 1818795
Part-of: odoo/odoo#67825
When creating a new user, add a default signature if none is given. It
should be a simple one like
```
--
<user_name>
```
Implement it using an editable computed field based on name. That way users
can set it explicitly as void, but changing user's name sets a default one
when it is not set.
Task-2712450 (Mail/Sale: Improve 'Pay Now' notification template)
Part-of: odoo/odoo#82167
Purpose of this commit is to make ``get_access_action`` private as it is not
necessary to expose it directly. Website management is also made explicit
using a ``force_website`` parameter instead of relying on context key of the
same name. This allows to better understand the method code flow.
Contains also some code fix / improvements :
* Website forum: update code to better skip the frontend redirection if the
forum is not active and frontend is not forced;
* Website slides: respect force website parameter in redirection;
Task-2710804 (Mail: Clean Mail.Thread API)
Part-of: odoo/odoo#82167
Update some counters at latest runbot results, just to have updated counters.
Note that some counters are somehow not deterministic, hence not updating
everything.
Task-2712450 (Mail/Sale: Improve 'Pay Now' notification template)
Part-of: odoo/odoo#82167
reportlab package is already in use to print barcodes, but it requires
extra packages (pylibdmtx and libdmtx) to print Data Matrix barcodes.
Unfortunately the python3-pylibdmtx package is not currently in the
Ubuntu version being used by Odoo SaaS and odoo.sh => we cannot install
this package automatically. Therefore we leave both packages as optional
extra installs for users since pylibdmtx is available via pip3 and
additional libdmtx package can be installed at that time (libdmtx0b on
linux, other OS info in pylibdmtx pip page).
We default to Code128 to avoid blocking stacktrace in case
Reportlab cannot print Data Matrices.
Part of task: 2494740
Part-of: odoo/odoo#82389
Whenever a user is modified, if at least one of a predefinet-field list
is being touched (e.g. companies or groups), cache is automatically
invalidated. However, that field list is harcoded, which prevents it
from being inherited and extended.
This commit moves such fields to a separate method, which makes
possible to inherit such method and extend the list.
closesodoo/odoo#79986
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Followup of #82838 and #82844
By default, every subclass of BaseModel is registered in a mapping for
the automatic discovery of model classes by registries. Prevent
registration of the mockup class used for rendering the database manager
templates.
closesodoo/odoo#83626
X-original-commit: 3bce74b20f8c15befee453382e3ab4697bee6960
Signed-off-by: Raphael Collet <rco@odoo.com>
When the assets are regenerated, the previous attachments are deleted.
This may happen during the execution of t-call-assets directives in a
qweb view.
However, some properties that have been accessed with a sudo() were
accessible in the cache. Clearing the cache during the view rendering
could lead to access errors, which wouldn't be present without this
directive.
This commit works around this problem by removing these attachments with
a SQL query, without relying on the classic unlink, so that the cache is
preserved in this case.
By the way, sanitize the filename when marking it for deletion.
closesodoo/odoo#83570
X-original-commit: 627d508edfb62449067a05b0e3c1a0004f0af32c
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
Signed-off-by: Paul Morelle <pmo@odoo.com>
The possible index names have been renamed "btree", "btree_not_null"
(instead of "not null") and "trigram" (instead of "gin").
Task 2742526
Part-of: odoo/odoo#83274
Issue
-----
Via the field prefetch mechanism, when we need a value of one field
(not in cache of course), the ORM will prefetch all fields
(which has the attribute to `prefetch=True`, the default value of this
attribute is `True`) for all record ids in `_prefetch_ids`.
Then, for each translate fields (where translate is not a callable)
the ORM need to make a `LEFT JOIN` on the `ir_translation` to fetch the
translated value. For big model, it leads to a simple `SELECT` with
several `LEFT JOIN` on ir_translation but each LEFT JOIN have a cost
in the planner time (a small cost in the execution time) of PostgreSQL.
By example, for `product.template` (stock/sale/purchase installed),
there are 6 LEFT JOIN to get all translated fields (5 of this
fields are rarely used).
Proposed solution
-----------------
Deactivate the prefetch by default for all translate fields expect if
this field is the `_rec_name` of the model (which is more likely to
be used).
In the example on the `product.template`:
Without prefetching the translated fields, there is only one LEFT JOIN
(the name, which is translated but is the `_rec_name` of the model).
With the 6 translated fields to fetch, the
query takes 5 ms to plan and 2 ms to execute VS with 1 translate field,
it 1 ms to plan and 1.5 ms to execute.
Side change note
----------------
- All translate of fields of `website.seo.metadata` should be prefetch
to avoid lot of website errors (it is because, website put in cache data
in sudo before reading it without sudo)
- `description` (`mail.message.subtype`), `subject` (`mail.template`),
`body_html` (`mail.template`) should be prefetch to avoid lot of extra
query from mail module.
- `vat_label` (`res.country`) should be prefetch to avoid a extra query
for each website page.
- Increase some queryCount (when it is legit, due to `subtitle` of
`blog_post` or `description` of `event.type.ticket`, etc)
task-2738029
closesodoo/odoo#82896
Signed-off-by: Raphael Collet <rco@odoo.com>
In debug mode, when a company ir.rule raise a exception
for one or multiple records, a feedback message is
display with limited records information.
Add company name to these information to help the user to change
his current company to the correct one directly.
task-2628865
Part-of: odoo/odoo#79073
Before this commit, changing the model on an existing xml-id may lead to
strange results.
create any record in xml, ie: id=my_xid, model=new_model
=> Odoo will create a new external_id, pointing to (new_model, id=1)
change only the model in the xml, ie: id=my_xid, model=ir.cron
=> Odoo will use the id of the existing external_id with the new model
and points now to (ir.cron, id=1)
AND REPLACE the ir.cron id=1 (autovacuum_job) with the data of the xml.
This commit simply prevent that by raising when trying to recycle the
same xmlid for another model. An explicit upgrade script to remove the
existing xmlid and corresponding records should be write.
closesodoo/odoo#83422
Signed-off-by: Raphael Collet <rco@odoo.com>
This prevents memory issues on creating massive amount of records.
For example, on populating `account.move` table,
the process takes 10 time less memory, while execution time per batch is almost the same: ~1 minute.
closesodoo/odoo#82875
Signed-off-by: Raphael Collet <rco@odoo.com>
Yannick Tivisse