Commit Graph
869 Commits
Author SHA1 Message Date
Xavier Morel a92e88d547 [IMP] base: make tests less sensitive to babel / CLDR updates
Babel's named / implicit formats (short, medium, long, ...) come from
the CLDR, which can get tuned as debates get settled, cultures shift,
etc... as a result using these formats can break tests on any babel or
even CLDR release (technically nothing stops distros from updating
their bundled babel with new CLDR data).

b77eb98bbf6bc937efb7941802c0794ae3622094 previously did some
mitigation of this issue, but even if they're not yet in distros
further Babel updates (e.g. 2.12) already affect some of the patterns
we're using.

Proactively mitigate this issue more by only using explicit datetime
patterns (and time patterns, date is fine because it retrieves the
pattern from the lang rather than default to babel patterns) in the
date/time formatting tests. This means only specific terms still vary,
and those should be a lot more stable / reliable than e.g. futzing
with separators and minor formatting issues.

closes odoo/odoo#137181

X-original-commit: 3547e980428cc1dd11c8e82446e7d7f824b9f6f8
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-02 05:56:06 +00:00
Jorge Pinna Puissant b89f53095c [REF] base: remove unused test
Since [1] qweb library is not used anymore, rendering this test useless.

1 : 4b0a951af6

closes odoo/odoo#136610

Signed-off-by: Raphael Collet <rco@odoo.com>
2023-09-27 18:56:19 +00:00
Rémy Voet (ryv) f9e75d19a9 [FIX] *: Fix bad usage of 'like'/'ilike' operator
The 'like'/'ilike' operators automatically add the wildcard character
(`%`) at the beginning and at the end of the value. This commit fixes
the incorrect usage.

closes odoo/odoo#136007

Related: odoo/enterprise#47886
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-09-27 09:11:44 +00:00
Rémy Voet (ryv) 51a795d363 [FIX] core: '=like'/'like' doesn't use unaccent anymore
The `=like`/`like` domain operators are accent-insensitive, but still
case-sensitive. This is not really coherent, since `unaccent` is a best
effort to find records from the client, and it is the same idea behind
being case-insensitive. Also, `=like`/`like` cannot be used from the web
client, and we use them in domain to search from the Python side.

Moreover, adding `unaccent` to `=like` can be very inefficient when
searching for a prefix ('prefix%'). In fact, PostgreSQL can use btree
index to find prefix matches, but because we create dy default btree
index without unaccent (when we put
`index=True/'btree'/'btree_not_null'` on the field), PostgreSQL cannot
use this index.

Part-of: odoo/odoo#136007
2023-09-27 09:11:44 +00:00
Raphael Collet 6a61d2b322 [IMP] core: use SQL wrapper in Query
Part-of: odoo/odoo#134677
2023-09-27 03:01:45 +00:00
Raphael Collet 020ddc3a6b [IMP] core: introduce SQL wrapper
We introduce a new class of objects to wrap SQL code together with its
parameters.  It is designed to be easily composable and to discourage
SQL injections.  Its API is similar to the methods of module 'logging':
the code is a format string, and the positional parameters are meant to
be merged into it using the string formatting operator.

    # default and increment are parameters of the SQL code in first argument
    term = SQL("COALESCE(value, %s) + %s", default, increment)

    # term can safely be injected into another SQL, besides regular parameters
    query = SQL("SELECT %s FROM mytable WHERE id = %s", term, id_)

The SQL wrapper can return the final SQL code string as query.code, and
the corresponding parameters as query.params (list).  The cursor method
execute() can now take an SQL object, and execute it just like

    cr.execute(query.code, query.params)

It is quite easy to make SQL objects safe against SQL injections: if the
code is a string literal, then the SQL object is guaranteed safe,
provided the SQL objects within its parameters are themselves safe.

Part-of: odoo/odoo#134677
2023-09-27 03:01:44 +00:00
Gorash ba1a5509fa [IMP] base: Remove context dependencies from get_views method
In this viewtiverse, the heroes remove the context dependencies for
`get_views`, from the views and python fields (such as domain). To reduce
inconsistencies and the number of rpc.

Current issues:
* There may be inconsistencies in views at the JavaScript level. Some
overrides modify the behavior of get_views or domains on fields via
context keys, therefore by changing the action, the rendering may be
different. However, these views are cached. However, the cache key
(Javascript) does not reflect the entire context, and requires additional
post-processing from the server.
* Multiple rpc for the same rendering. get_views being dependent on the
context, as soon as it changes, a new rpc is performed. In most cases,
when JavaScript needs the same view, there is no change depending on the
context, the rpc is useless.
* Inconsistency when rendering subviews, some views could be different
depending on the context, this context can be modified in the view itself
via the context attributes. However, the JavaScript client does not redo
an rpc for each change of these sub-contexts. Therefore the result may be
inconsistent.

Solution:
Limit as much as possible the number of context keys provided when calling
get_views, and use the context provided as a cache key. The authorized
keys are 'lang' and '*_view_ref'. For the cache key, options are added in
the get_views method.
Instead of using the context, it is inserted into python expressions.
This will be evaluated by JavaScript and thus avoids inconsistencies.

task-3414108
task-3414068

closes odoo/odoo#135145

Related: odoo/enterprise#47584
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-09-21 16:52:10 +00:00
Thibault Delavallée 8409ebe5fb [FIX] various: update query counters to runbot state
Update (some) query counters according to runbot state.

Also make some tests deterministic when involving company name.

Task-36879 (Mail: Support MultiCompany Aliases)

closes odoo/odoo#135288

Related: odoo/enterprise#47345
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-09-19 16:37:13 +00:00
Ivan Rasputin 1711e91257 [IMP] core: test exporting of translation files
Static qweb files are not tested for syntax errors.
This commits adds a hacky way to do this job: we test exporting of translation files.

Inspired by the following PRs, that were made as a response for errors caught by sentry

* https://github.com/odoo/enterprise/pull/43975
* https://github.com/odoo/odoo/pull/128221

closes odoo/odoo#128445

Related: odoo/enterprise#44074
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-09-15 09:22:04 +00:00
Yannick Tivisse c2ebe70ed4 [IMP] *: Remove todos we won't do
closes odoo/odoo#134836

Related: odoo/enterprise#47144
Related: odoo/upgrade#5129
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2023-09-12 17:16:23 +00:00
Chong Wang (cwg) ddfdba9424 [FIX] core: update model terms for en_US and other
For model_terms translated fields if a translation for a lang(fr_FR) has never
been defined

before this commit
when update translations for both en_US and fr_FR, the new translation for fr_FR
cannot be saved.

after this commit
new translations can be correctly saved when en_US and fr_FR are updated at the
same time.

closes odoo/odoo#135103

X-original-commit: ef4b195eeac178a14eb47f4e6cc61ddc97f78866
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Chong Wang (cwg) <cwg@odoo.com>
2023-09-11 23:59:51 +00:00
Thibault Delavallée 64aae8bce1 [FIX] tools, base, mail: add a fallback when parsing wrongly-formatted emails
With input 'name email@domain.com' (missing chevrons allowing to clearly spot
the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
whole input is considered as being the email.

To improve the heuristic we can add a fallback by recalling 'getadresses'
on the input with spaces replaced by commas when it found only an email and
no name. The new email will be split into sub pairs allowing to find the real
email and various name parts, allowing to make a new name / email pair.

Emails should not contain spaces thus this is coherent with email formation.
This fallback actually comes from a specific code done in '_parse_partner_name'
of Partner model. Supporting it directly at tools level make the behavior
coherent for all models.

Task-2612945 (Mail: Defensive email formatting)

X-original-commit: odoo/odoo@18c71edf59
Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
Thibault Delavallée e0207d1551 [IMP] tools, base, mail: better support non-ascii / IDNA when normalizing
PURPOSE

Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.

SPECIFICATIONS

As of rfc5322 section 3.4.1 local-part is case-sensitive. However most main
providers do consider the local-part as case insensitive. With the introduction
of smtp-utf8 within odoo, this assumption is certain to fall short for
international emails. We now consider that

  * if local part is ascii: normalize still 'lower' ;
  * else: use as it, SMTP-UF8 is made for non-ascii local parts;

Concerning domain part of the address, as of v14 international domain (IDNA)
are handled fine. The domain is always lowercase, lowering it is fine as it
is probably an error. With the introduction of IDNA, there is an encoding
that allow non-ascii characters to be encoded to ascii ones, using 'idna.encode'.

Also remove usage of 'email_re' in mailing email check. It is too restrictive
compared to real formatting we support (or try to). Valid outgoing emails
were directly canceled, notably when containing unicode.

Task-2612945 (Mail: Defensive email formatting)

X-original-commit: odoo/odoo@3ce5fb3072
Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
Thibault Delavallée 075f073006 [IMP] tools, base, mail: use first found email in 'email_normalized'
PURPOSE

Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.

SPECIFICATIONS

When having multi-emails input in an email field, 'email_normalized' field is
currently 'False', as they expect the field to contain a single email. This
has several drawbacks

  * searching partners or fetching information based on emails does not work as
    most tool methods use 'email_normalized' which is False (see e.g.
    '_message_partner_info_from_emails', '_mail_find_partner_from_emails'
    or 'find_or_create');
  * blacklist is not available as it is based on 'email_normalized';
  * mass_mailing wrongly considers those emails as invalid and cancel their
    mail and related trace, as it tries to skip sending emails to invalid
    emails;

Be more defensive and use first found email in case of multi-emails field.
Other emails are ignored. It is already an improvement that does not break
flows in stable and allow more emails to be sent.

  before
  -> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
  -> email_normalized: False
  after
  -> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
  -> email_normalized: raoul1@raoul.fr

A side effect is that it helps finding back some partners, as indicated in
tests where less phantom partners are created. It also helps suggested
partners / emails flow in discuss.

Task-2612945 (Mail: Defensive email formatting)

X-original-commit: odoo/odoo@90218186c5
Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
Thibault Delavallée 3191aa8207 [IMP] base: avoid double formatting in partner 'email_formatted' field
PURPOSE

Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.

SPECIFICATIONS

Main fix in this commit: fix multiple nested formatting in 'email_formatted'
computation for <res.partner>. Other use cases are mainly left untouched as
we let users deal with their input. In summary :

  * double format: if email already holds a formatted email, we should not use
    it to compute email_formatted, like

      name: Name / email: 'Format' <email@domain.com>
      -> before '"Name" <"Name" <email@domain.com>>"
      -> after '"Name" <email@domain.com>''

  * multi emails: sometimes this field is used to hold several addresses
    like email1@domain.com, email2@domain.com. We currently let this value
    globally untouched by extracting emails and joining them, as we do not
    expect email_formatted to be a list of emails. Extractin emails allows
    to filter out extra text stored in email field, like

      name: Name / email: text, email1@domain.com, email2@domain.com
      -> before: "Name" <text, email1@domain.com, email2@domain.com>
      -> after: "Name" <email1@domain.com,email2@domain.com>

  * invalid email: if something is wrong, better keep it in email_formatted
    than harcoding "False". Indeed this eases management and understanding
    of failures at mail.mail, mail.notification and mailing.trace level. This
    behavior does not change as it was already implemented like that even if
    not sure it was intended;

Task-2612945 (Mail: Defensive email formatting)

X-original-commit: odoo/odoo@9175bbd8e2
Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
Thibault Delavallée 31150660ca [MOV] base, mail: regroup partner and mail tools tests
Move tests currently in mail about helpers defined in 'tools/mail.py' directly
into base. No need to test it in mail, there is no specific override or
behavior change compared to base.

Regroup partner related tests in base into 'test_res_partner'. This breaks
part of test history but it helps knowing which features are tested.

Task-2612945 (Mail: Defensive email formatting)

Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
Thibault Delavallée 459c085c19 [IMP] base, test_(mass_)mail(ing): add tests for multi / formatted email fields
PURPOSE

Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.

RATIONALE

Add tests related to not standard usage of email field. Two main use cases
are tested here

  * formatted emails: `"Full Name" <email@domain.com>` stored into the 'email'
    field;
  * multi emails: `email1@domain.com, email2@domain.com` stored into a single
    'email' field;

Additional tests: tests with unicode / ascii / case / wrong formatting are also
added to check the support in normalize and format methods.

IMPLICATION

Email field is generally managed as "containing a valid email". This means
it is sometimes used as it in 'formataddr' as well as to perform searches or
identification checks. Example of issue: partner 'Raoul' has a formatted email
like "Raoul" <raoul@raoul.fr>. Using 'formataddr' in email_from leads to

  from: "Raoul" <"Raoul" <raoul@raoul.fr>>
  -> which is incorrect (but often dynamically corrected by email servers);

Email field holding multi-emails are not normalized, as current normalize
is done only if the field holds a single email. It means

  * no easy finding based on 'email_normalized', e.g. various tools like
    '_mail_find_partner_from_emails' or 'find_or_create' do not find partners
    based on this email;
  * no exclusion list management;
  * issue with formatting, like

  to: "Raoul" <raoul@raoul.fr,raoul.other@raoul.fr>
  -> which is incorrect (but often dynamically corrected by email servers);

USAGE: OUTGOING EMAILS

Those use cases currently generate faulty outgoing emails. This is valid for
recipients ('email_cc', 'email_to') as well as author ('email_from').

For formatted emails: `email_to` is formatted again based on name and email
which leads to sending emails to `"Full Name" <"Other"<email@domain.com>>`.

Note that multi emails without formatting may work as it leads to email_to
`"Full name" <email1@domain.com,email2@domain.com>`. Some outgoing email
servers correctly send multiple emails. It depends on their fault
tolerance.

USAGE: FIND BASED ON EMAIL (NORMALIZED)

When searching for partners (e.g. using '_mail_find_partner_from_emails' or
'find_or_create') normalized version of input is used.

In case of multi emails sanitize is 'False', as normalization expects a single
email in the field. Therefore no partner is found. In processes that do a
"search or create" (e.g. using a template on a record) this leads to creating
a new partner (or several partners in case of multi emails) each time.

USAGE: OTHER FLOWS

Other flows are build on top of '_mail_find_partner_from_emails' / 'create'
of outgoing emails and are impacted by formatted email / multi email usage.
Those include notably

  * mass_mailing: '_message_get_default_recipients' should be defensive to
    give correct values when creating mailing emails;
  * mass_mailing: faulty emails is based on normalize and multi-emails are
    considered as faulty and ignored;
  * after post hook: '_message_post_after_hook' tries to link messages without
    author (but email_from) with newly-created partners, when partners are
    created from chatter. It is therefore impacted by those corner cases;
  * marketing_automation: built on top of mass_mailing and suffers from the
    same issues;

USAGE: UNICODE

Unicode in emails should be supported. 'formataddr' and IrMailServer notably
received fixes to support unicode. Some check performed on email addresses
fail when unicode is involved, which leads to some emails not being sent
while they could.

SPECIFICATIONS

Add tests related to those corner cases. Also add tests for computation of
`email_formatted` field of Partner model. It currently generates wrong email
values for the same corner cases (multi emails, formatted emails).

Tests are also added for the computation of `email_normalized` field used
notably for blacklists. It is not computed currently when being in multi
email mode which prevents from any blacklist mechanism as well as make
email finding harder. `_mail_find_partner_from_emails` tool method is also
tested with multi email as it uses the same heuristic as normalized email
field.

Tests are also added for mass mailing, when having to mail documents that
have a partner with formatted emails / multi-emails, or that have an email
field with formatted emails / multi-emails.

Also restore a test removed at odoo/odoo@afcb734908 while it should have been
updated to state that email addresses containing non-ascii characters are
supported.

Add some tests for tools methods used in various email processing flows.
Unicode tests are also added.

In future commits we will try to make email usage a bit more defensive to
try to lessen issues with that kind of use cases.

Task-2612945 (Mail: Defensive email formatting)

X-original-commit: odoo/odoo@fc8442f133
Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
Thibault Delavallée 7f7440a599 [FIX] base, mail: ensure ordering of search on partner / user
Add 'id' in partner ordering, to be sure searching on duplicated partners is
deterministic.

In mail, use standard ordering when searching on users to be deterministic.
As ordering on users is based on name then login which is unique, this is a
safe ordering and can be used as it.

Task-2612945 (Mail: Defensive email formatting)

X-original-commit: odoo/odoo@1e6d50a141
Part-of: odoo/odoo#134934
2023-09-11 15:22:09 +00:00
bve-odoo 6af506e842 [FIX] base: avoid variable name collision with functions in QWeb
Task-2674716

Closes #78392

Signed-off-by: Christophe Matthieu (chm) <chm@odoo.com>
2023-06-27 08:35:21 +00:00
Damien Bouvy aa0833184e [FIX] base: writing selection values with server actions
Server actions that 'update the record' have a mechanism to allow users
to easily select a selection value if the field they want to update is a
selection field (instead of having to type the technical value
directly).

Before this commit, this mechanism incorrectly stored the selection's
name instead of the value (e.g. 'Done' instead of '01_done'), making the
write crash when the action was run.

closes odoo/odoo#134625

Signed-off-by: Bouvy Damien (dbo) <dbo@odoo.com>
2023-09-07 10:29:49 +00:00
0a744accc2 [IMP] base_automation,*: simpler edition workflow
*: base, crm, digest, mail, mass_mailing, sms, test_base_automation,
   website_forum, website_sale

This commit makes "Automated Actions" more discoverable and usable by:

- Adding a menu in the kanban header config dropdown to add/edit them.
- Creating a new custom kanban view for a clear understanding of each
  automated action record and its associated actions.
- Introducing new "smart" triggers that appear in the form view based on the
  chosen model:
  - Updated Values category:
    - "Stage is set to" when a `stage_id` field exists in the model,
      allowing users to select a specific stage value.
    - "State is set to" when a `state` field exists in the model,
      allowing users to select a specific state value.
    - "Priority is set to" (`priority`) where users can select a specific priority.
    - "User is set" (`user_id`, `user_ids` fields)
    - "Tag is added" (`tag_ids` field) where users can select a specific tag.
    - "On Archive"
    - "On Unarchive"
  - Timing Conditions:
    - "After creation"
    - "After last update"
- Deprecating previously known triggers "On Creation" (`on_create`) and "On
  Update" (`on_write`) to simplify the user experience. "On Creation & Update"
  (`on_create_or_write`) is retained and renamed to "On save".
- Changing the `ir.actions.server` Many2one relationship to a One2many
  relationship. Automated actions can now directly contain multiple actions,
  eliminating the need for an "Execute several actions" action in automation
  rules.
- Introducing a widget for the new `ir.actions.server` One2many field for a
  clearer understanding of multiple actions.

This commit also enhances the usability of "Server Actions" (`ir.actions`) by:

- Removing the `ir.server.object.lines` model and the associated `fields_lines`
  One2Many field. The attributes of the removed model are now merged into
  `ir.actions`. An action can now write to only one field, and the create action
  is now a name_create action.
- Adapting the form view when creating an "Update the record" action. The value
  field shown adapts itself based on the field to update; this field can be a
  `reference` field for a `one2many` `update_field_id`, a `one2many` field for a
  selection `update_field_id`, or a `text` field otherwise.
- Refactoring the form view to display only relevant details and other
  miscellaneous improvements.

Taskid: 3085360
Part-of: odoo/odoo#114352
Co-authored-by: Florent Dardenne <dafl@odoo.com>
Co-authored-by: Julien Carion <juca@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
2023-09-05 18:44:13 +00:00
niyasraphy 251e558ff2 [IMP] base: remove activate module server action
before this commit, from list view users can install
module using the button in list view and from the
action button.

initially the Install button was not available in the
list view and only option to install multiple apps was
from the action button.

but with the introduction of the button in list header
there is no need for an another server action to
perform the same.

after this commit, the activate modules server action
will be removed from the code and its related test
and also newly added Install button will be renamed
to "Activate" to align with the button in kanban and
form.

closes odoo/odoo#133544

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-04 05:49:50 +00:00
MerlinGuillaume 41d07e9d38 [FIX] base: allow deletion of inherited custom field
The inherited field of a custom field cannot be deleted

Steps to reproduce:
1. Install Contacts and Studio
2. Go to Contacts and open any contact
3. Toggle Studio
4. Add a field of any type in the view, remove it and close Studio
5. Go to Settings > Technical > Database Structure > Fields and search
   for `x_studio`

Solution:
Mark the inherited field as manual if its parent field is manual and
allow the deletion of inherited custom field if we also delete its
dependency

Problem:
The inherited field was not marked as custom so it was impossible to
delete it

opw-3093581

closes odoo/odoo#133820

X-original-commit: 526f3407c7c5b4cd014a4d091ca01b9617e6e938
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Guillaume Merlin (megu) <megu@odoo.com>
2023-08-31 18:53:16 +00:00
Rémy Voet (ryv) 013332d791 [FIX] core: add display_name fallback
Since 3c62ca1eb9, if the `_rec_name` value
is False, `name_search` and `name_create` will return a tuple of
(<id>, False). This is an invalid response for the web client,
which triggers a JS traceback.

Instead of using the old behavior (returning an empty string, resulting
in a partially invisible row in the Many2one selection),
use the same fallback as when the `_rec_name` doesn't exist.

Since `display_name` should never be Falsy anymore, remove part of the
test_mail_message_values_fromto_long_name that covers the
Falsy `display_name` case.

task-3424154

closes odoo/odoo#133691

X-original-commit: 0cb9e66edd9b7142a6e56bc6ee6491e6d6047e51
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
2023-08-31 09:05:02 +00:00
Thibault Delavallée a4fe51cb2e [MOV] base, mail: move mail config parameters usage to mail
RATIONALE

As multi-company tolerant alias domains will soon replace the usage of
configuration parameters, having them in base then replaced by more advanced
models in mail would be complicated to handle and not useful. Move those
ICP to 'mail' so that all mail configuration is done in that module.

SPECIFICATIONS

Move config parameter used for alias domains configuration in 'mail' module.
Base should be as simple as possible and let mail deal with mail server
complexity.

Move 'mail.{bounce/catchall}.alias' used with 'mail.alias.domain' to make
bounce and catchall emails. Move 'mail.default.from' as it will be integrated
into alias domains in some form.

Note that 'mail.default.from_filter' stays as an ICP in base as it is a
more global default parameter. It is used as default value in 'connect' when
no mail_server is used and no from_filter can be retrieved.

Some tests in 'base' are either fixed, either moved directly into 'mail'.
We now differentiate base behavior (without ICP) from configurable behavior
(with ICP in mail).

Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130750
2023-08-22 20:58:54 +02:00
Thibault Delavallée 29f7e6d894 [FIX] base: be defensive when computing parts of 'from_filter'
From filter could be ill-defined, like ' ' or ','. This commit just make
some code more defensive against those values.

Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130750
2023-08-22 20:58:53 +02:00
Thibault Delavallée 79677134d7 [IMP] base: split _get_test_email_addresses to distinguish from/to
Split '_get_test_email_addresses' into two methods allowing to generate the
'from' and 'to' when testing SMTP connection. As 'email_to' is always the
same better have a small method for it. Moreover it eases overrides if
some code wants to tune the from / to by overriding only the necessary one.

Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130750
2023-08-22 20:58:52 +02:00
Thibault Delavallée 17e5bbb38b [IMP] base, mail: improve and add tests about ir mail server
Prepares the move of ICP to mail before replacing them by dynamic alias
domains. Improve test coverage, notably for edge cases. Continue to make
tests more explicit after odoo/odoo#131492. Some tests are also merged to
lessen number of different tests when possible, notably when only a test
parameter differs (like giving an SMTP session or not).

Clean ICP and mail servers setup in test classes allowing to remove some
unnecessary extra initialization. Cleanup a mock in mail.

Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130750
2023-08-22 20:58:50 +02:00
Rémy Voet (ryv) 160acc0200 [FIX] core: fix inconsistencies between _apply_ir_rule and check_access_rule
Issues
======
- `_apply_ir_rule` applies `ir.rule` of the current model and also
`ir.rule` from the inherited model (via inherits). But
`check_access_rule` doesn't check the later one.
- `_flush_search` doesn't flush fields coming from the `ir.rule` of
the inherited model (via inherits). Then the filtering done by
`_apply_ir_rule` may be inconsistent with cached values.

Changes
=======
Because of https://github.com/odoo/odoo/blob/6ddcb448612f5d784c8e9ebb90f19077e65be3e1/odoo/osv/expression.py#L1073-L1073,
and https://github.com/odoo/odoo/blob/00e86b1552d1e5541a8dbf9411de5cfdb8990cc4/odoo/fields.py#L2895
leaf like `('<many2one_delegate>', 'any', [<sub-domain>])`,
will be translated in the same way as `_inherits_join_add` does.
We can remove `_inherits_join_add` and its usage in `_apply_ir_rule`
and change `ir.rule._compute_domain` to also return the inherited
(via inherits) `ir.rule` domain (with the new 'any' operator).
Since `_compute_domain` is used by `_apply_ir_rule` and
`_filter_access_rules_python`, everything is consistent.

Also fix `BaseModel._flush_search` to take in account 'any'/'not any'
operators (compulsory in order to flush correctly new domain
from `ir.rule._compute_domain` generated).

Part-of: odoo/odoo#125916
2023-08-21 19:56:55 +02:00
VAN BOSSUYT Nicolas ad06a7b2ad [IMP] base: improve PDF generation speed for large tables
This commit addresses performance issues when generating PDFs with large
tables using wkhtmltopdf. Processing time for such tables grows
exponentially with rows, causing significant delays.
Testing revealed a PDF with 250,000 rows took about an hour.

Previously, a workaround involving special XML template was provided to
users, inserting </table><table> tags every 500 rows.
This commit introduces a general solution at framework level.
Now, tables with >500 rows will automatically use this workaround,
enhancing PDF generation speed.

The number 500 is taken from opw-1689673 and seems to be a good
compromise between the number of split in tables and the processing
time by wkhtmltopdf

closes odoo/odoo#131933

Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
2023-08-21 12:39:19 +02:00
Raphael Collet 24747d112f [FIX] base: execute tests using Form post-install
Part-of: odoo/odoo#124614
2023-08-18 19:16:50 +02:00
Gorash cdaa761ced [REF] base: Update modifier syntax (invisible, required, readonly)
Goal:
* Simplified modifiers to only have one way to define modifiers;
* Remove states attributes on python field;
* Use python expression in view `required`, `readonly`, `invisible`;
* More accurate validation of xml views.

This commit change the syntax to python expression. The next commit
will update/convert all xml views.

Before this commit:
* the `required`, `readonly` and `invisible` attributes can only have
values of `True`, `False`, 1, 0 or a python expression to use the
context;
* the `attrs` attribute define a dict. The key of this dict was
`required`, `readonly` and `invisible` and the values are the domain or
a string representing a domain to be evaluate as python expression.
This python expressions was evaluate by the javascript with view fields
and other contextual values as: context, uid, parent, active_id,
active_ids, active_model, allowed_company_ids, current_company_id.
* the `states` attribute in the view was a comma separated list of the
state. This list was combined with the `invisible` attribute;
* the `invisible` attribute on python field is used as default value;
* the `states` attribute on python field was dictionnary with state as
key and list of tuple. This structure was combined with `readonly` view
attribute.
* After combining, the resulting domains of the different attributes
`required`, `readonly` and `invisible` are evaluated with the values of
the fields. The `invisible` attributes is splitted into two use:
`invisible` and `column_invisible`.

After this commit:
* The attributes `required`, `readonly`, `invisible` and
`column_invisible` define python expression. This python expressions
are evaluate by the javascript with view fields and other contextual
values as: context, uid, parent, active_id, active_ids, active_model,
allowed_company_ids, current_company_id.

The domains can contains contextual value and will be evaluate by the
javascript.

```xml
    <field name="field_a" readonly="not context.get('show_a')" attrs="{'readonly': [('field_b', '!=', False), ('field_c', '=', parent.c)]}"/>
    <field name="field_b" states="draft"/>
```
will be replaced by
```xml
    <field name="field_a" readonly="not context.get('show_a') or field_b and field_c == parent.c"/>
    <field name="field_b" invisible="state != 'draft'"/>
```

Some inherited views will be modified differently in order to maintain
the previous behavior:

```xml
    <field name="field_a" readonly="not context.get('show_a')" attrs="{'invisible': [('field_b', '!=', False)]}">
```
```xml
    <field name="field_a" position="attributes">
        <attribute name="attrs">{'readonly': [('field_c', '=', False)], 'invisible': [('field_d', '!=', '3')]}<attribute>
    </field>
```
will be replaced by
```xml
    <field name="field_a" readonly="not context.get('show_a')" invisible="field_b">
```
```xml
    <field name="field_a" position="attributes">
        <attribute name="readonly" add="(not field_c)" separator=" or "/>
        <attribute name="invisible">field_d != 3<attribute>
    </field>
```

Validation:
A stricter control is made on the level of the attributes (modifiers)
and the fields necessary for these. The use of the previous attributes
'attr' and 'states' triggers an error (these no longer exist after the
application of the migration script)

task-2495504

Part-of: odoo/odoo#104741
2023-08-18 09:49:08 +02:00
Alvaro Fuentes f7cd412725 [FIX] core: fix remove constraints at uninstall
This patch aims to fix multiple issues with the removal of table
constraints at module uninstall.

1. We cannot remove `ir.model.constraint` records before calling
   `_module_data_uninstall` on them. Otherwise we either won't find them
   when performing the search
   `self.env['ir.model.constraint'].search([('module', 'in',
   modules.ids)]` or, if we somehow keep the ids and use `browse`
   instead, would get an error because `_module_data_uninstall` tries to
   access field values of records already removed. Note, although not an
   issue, the removal is redundant for non FK constraints since
   `_model_data_uninstall` already unlinks the record.
2. When a constraint has a name longer than 63 characters (Postgres
   default) we would fail the check for the existence of the constraint
   since the names are truncated.
3. When checking for the presence of a constraint we assumed its type
   would be `u` in `pg_constraint` because for us that means non FK
   (i.e. not `f` type). That's incorrect since there are many more
   types. Here we propose to handle `c,u,x` types.

For bullet 2 we use `tools.make_identifier` that hashes the name and
ensures it fits in the 63 chars limit.

Revert "[IMP] models: warn if constraint key len exceed 63"

The check from commit 823d9e10dc is no
longer needed since the name is ensured to fit length limit.

[IMP] code: improve uninstall tests

Perform extra checks for removal of SQL constraints. Note the test is
commented out in `__init__.py`. It can be uncommented locally for
testing. It's kept commented out to avoid random errors in runbot.

closes odoo/odoo#129084

X-original-commit: af288b7178c25261329dd85a2e64b9dd635cd9e1
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-08-16 16:04:47 +02:00
Om Rabara 52a204f3da [FIX] base: handle database.secret parameter while deletion
An error occurs when the user attempts to delete the 'database.secret' record,
either by following these steps:
- Enable developer mode.
- Go to Settings > Technical > System Parameters.
- Select the 'database.secret' record and attempt to delete it.

Or when the user tries to update the key for the 'database.secret' record using
the following steps:
- Open the 'database.secret' record.
- Update the value of the key field.
-  Save the record.
- The server will stop running and not be accessible.

Error: ValueError: CSRF protection requires a configured database secret

sentry - 4291267997

closes odoo/odoo#131460

X-original-commit: fe694e5b8285ceed99b321c22af534eee25cf282
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
2023-08-10 17:02:15 +02:00
Thibault Delavallée 053290c6e7 [IMP] base, mail: cleanup ir_mail_server tests, improve test logs
RATIONALE

This prepares the move of ICP to mail before replacing them by dynamic alias
domains.

Cleanup tests: try to use loops with input / expected to better understand
the various test cases, add some comments, improve logs when failing to
find the right sent email. Rename tests to have a better test structure
when reading logs.

Remove a test from odoo/odoo@3b6c20805c that adds nothing except testing
the test suite.

OTHER ADDONS

In test_mail: have a specific class for testing servers as other data is
not necessary, and it allows to have a tag for it.

In mass mailing: concatenate test about server finding, several tests can be
done in a single unit test.

Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

closes odoo/odoo#131492

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-08-10 10:39:41 +02:00
niyasraphy be20951c05 [FIX] base: traceback on rpc call if data contain default dict
before this commit, on reading data with default dict data
type is showing error to end user, without returning the
requested data.

for eg, if a read operation is triggered on model sale.order
it wont return the requested data, instead traceback is
shown in response.

in sale.order model the tax_totals field is a computed
field, with data as format default dict which was
causing the issue.

after this commit, without any traceback the requested
data will be returned to the user.

closes odoo/odoo#131123

X-original-commit: cc61b926f4daff26fb577e2f700d013ec15f4b4f
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
2023-08-08 20:07:18 +02:00
Thibault Delavallée aa69554a61 [REF] base, : move and cleanup ir mail server tests
Have all ir.mail.server tests as well as their configuration-related tests
moved into 'test_ir_mail_server' file. That way all tests are contained in the
same file, easing their update.

Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130768
2023-08-03 21:56:57 +02:00
Thibault Delavallée b5949d1672 [REF] test_mail, various: prepare gateway / alias tests
Rename alias domain and aliases used a test data. This allows to make
them easier to read, follow, grep and understand.

Activate multi-company on alias and gateway tests, ensuring it currently
has few impact on tests.

Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130768
2023-08-03 21:56:56 +02:00
Victor Feyens f5de88f3b5 [FIX] sale: failing test with localizations
The test verifying the values of the sale report in a multi-comp & multi-curr
environment relies on the fact that the main company is by default in USD.
Nevertheless, the test fails when a localization is installed (if it changes
the currency of the main company).

This commit makes sure the tests always works, and resurrects the right tool
for that, which was dropped in commit 3752b3166e.

Cf runbot build error 22351

closes odoo/odoo#130201

X-original-commit: a73b5fce4fb02eefc681c55eab8f57cd35105226
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2023-08-01 13:51:32 +02:00
Denis Ledoux 028bf8e0ec [FIX] base: tag test_add_field_valid as post_install
This test can lead to a deadlock,
when trying to drop the constraint on `ir_model_fields`
during the registry loading,
as the test opens a second cursor in addition
to the cursor used for the registry loading,
and drop a constraint on a table which is being read by
the registry loading (`ir_model_fields`), leading to the deadlock.
See screenshot in the PR for illustration.

This has been detected in the runbot nightly builds,
which were pending indefinitely because of that deadlock.
To reproduce, just init a database with `-i base`,
then restart the server with just `--test-tags .test_add_field_valid`.

To avoid to drop the constraint during the registry loading,
and hence to avoid the deadlock,
tag the test as `post_install`.
That way, the test is executed after the registry is loaded,
not during, and the constraint can be drop safely, not leading to a deadlock.

closes odoo/odoo#129660

X-original-commit: 67bc9a9e895245a94aac064e8f5a365ee6052261
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2023-07-26 09:22:19 +02:00
Xavier-Do 1c598d4b00 [FIX] base: avoid being stuck on a test
During the nightly, it looks like some test in
`TestAccountEarlyPaymentDiscount` will reload the registry, restarting
all base tests, including `.test_add_field_valid`

This test cannot be executed on an existing database without providing
a `-i` for a strange reason.

If this is not a real issue, this simple assertion will avoid to get
stuck while running this test on an existing database.

closes odoo/odoo#129491

X-original-commit: 4c6e05cbb43d08c544d9d558f80b67ccb786d482
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2023-07-25 08:19:08 +02:00
Martin Trigaux 935857e01d [FIX] base: do not remove groups added in implied groups
If a user goes from "Access Rights" (base.group_erp_manager) to
"Settings" (base.group_system) he gets an error telling him he did not
belong to Access Rights group.

The reason was that the modification of accesses on the user profile
page was doing
write({'sel_groups_2_4': 4})
which was transled, after the _remove_reified_group call into
write({'groups_id':[(UNLINK, [2, 4]), (LINK, [4]]})
so the user was temporary in a state where he was in the group_system
but no in group_erp_manager.
When the implied groups where computed, an access error was raised as
the administrator was not allowed to modify res.users record (need
group_erp_manager).

Simplify the _remove_reified_group call to only convert the call to
write({'groups_id':[(LINK, [4])]})

closes odoo/odoo#129335

Task-id: 3265053
X-original-commit: 0ed2fa018a7716f0fb3abe587983209403445fc8
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2023-07-24 15:52:05 +02:00
Lucas Perais 3c6235d446 [FIX] base: ir_actions_report: context for not using attachment
Before this commit it was not possible to print a report without
bypassing the retrieval of a previous attachment, or to print
a report without creating one.

After this commit it is possible with the context key "report_pdf_no_attachment"

Part-of: odoo/odoo#128228
2023-07-20 17:01:31 +02:00
Xavier-Do eb708e5cad [IMP] base: avoid cache_invalidation
A clear_cache was added in _update_xmlids in pr 119813.

If it was needed in case of update, it is not useful when creating an
xmlid or when the value is unchanged.

The initial idea was to detect if an update or an insert was done, maybe
using create_date and write_date. Unfortunately the create_date is the
same as the write_date in the same transaction. This is unlikely but in
this case, we could update the cache in place.

The final behavior is to update the cache in all case, and notify other
workers only if a model was updated.

This will help to avoid invalidating the cache too mush during module
loading. Even if the impact on time is small, the increase in queries
was visible.

This solution would even be a slight improvement on previous query count

closes odoo/odoo#129029

Related: odoo/enterprise#44349
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
2023-07-20 14:23:15 +02:00
Benoit Socias 567e5b58d5 [IMP] base, tools, web_editor, *: use original image if size increases
*: web_tour, website

When no transformation is applied on an image, changing the quality
sometimes increases its storage size.

This commit makes sure that the original image remains used if only the
image quality is modified and if this makes its storage size bigger.

Fixes #61619
task-2835144

closes odoo/odoo#103398

Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2023-07-19 13:12:51 +02:00
Xavier-Do 4c9968397b [IMP] base: avoid invalidation on xmlid updates
Splitting the cached revealed a missing cache invaldation.
The cache invalidation added a a query in test_related_fields

The query can be avoid by not updating the xmlid if it is not useful.

closes odoo/odoo#119813

Related: odoo/enterprise#42527
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2023-07-18 11:42:27 +02:00
Xavier-Do 595aa24843 [IMP] registry: multiple ormcache
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.

Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.

The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.

Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.

The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.

We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.

Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.

Note that this poc is inspired from the long term cache but not all
use case where applie yet.

Part-of: odoo/odoo#119813
2023-07-18 11:42:26 +02:00
Benoit Socias 706e696e3d [IMP] base, tools: compute webp image size
This commit introduces the computation of the image size from a webp
binary source without relying on PIL.

This is needed by eCommerce to determine which image to fetch when using
the zoom functionality on the product page.

task-2774352

Part-of: odoo/odoo#85494
2023-07-15 05:10:47 +02:00
Benoit Socias d1292a96a6 [IMP] base,*: support image/webp image format
*: mail, mrp, test_website, web, web_editor

Before this commit '.webp' images could not be used in odoo.

After this commit '.webp' images can be uploaded to odoo.
- can be used in image field
- can be used in HTML field image
- can be used in mails and website
- can be transformed (shape mask, filter effect, crop, rotate, resize,
  adjust quality)

task-2774352

Part-of: odoo/odoo#85494
2023-07-15 05:10:45 +02:00
Denis Ledoux 3d2f76671c [FIX] core: missing escaping in sql constraint name_manual_field
Oversight in revision
8c38baee85

In SQL,
The underscore character ( _ ) represents a single character
to match a pattern from a word or string.

Meaning
`name LIKE 'x_%'`
allows names starting by `x`, and not names starting by `x_` as expected.

Add the escape in the constraint to enforce starting by `x_`
and not just `x`

closes odoo/odoo#127883
2023-07-10 16:59:24 +02:00