Commit Graph
637 Commits
Author SHA1 Message Date
Nasreddin Boulif (bon) 4db8a60619 [FIX] mail,test_mail: message_route; filter emails with wrong domain
Steps to reproduce:

  - Install CRM and Helpdesk modules (for test purposes)
  - Set a custom alias domain (e.g. "mydomain.com")
  - Go to CRM > Configuration > Sales Teams
  - Check that a team has en email Alias (e.g. "info@mydomain.com")
  - Go to Helpdesk > Configuration > Helpdesk Teams
  - Check that a team has en email Alias (e.g. "support@mydomain.com")
  - Email your instance with the following `to` value:
    info@mydomain.com, support@test.com
    (notice second email does not match the DB alias domain)
  - Go to CRM : A task has been created
  - Go to Helpdesk : A ticket has been created

Issue:

  The ticket in Helpdesk should not have been created.

Cause:

  The message_route method does not check the domain of the email
  address before creating the routes.

Solution:

  If `mail.catchall.domain.allowed` system parameter is set, filter to
  only keep the emails address that match the allowed domains (including
  domain set in `mail.catchall.domain` system parameter).

  The value of `mail.catchall.domain.allowed` system parameter should
  be a comma separated list of domains. e.g. `example.com,example.org`

opw-3150972

closes odoo/odoo#128346

X-original-commit: 5b73428104789433fd65431f7631a2e9eb863b2f
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-07-13 10:46:18 +02:00
Alexandre Kühn 7f0e372e34 [IMP] mail: edit message comment in chatter
Before this commit, only logged notes could be edited in chatter.

With this commit, messages sent with "Send message" are also
editable.

Task-3380465

closes odoo/odoo#125939

Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2023-07-06 17:52:01 +02:00
Astik Singh 200ccf1b10 [FIX] mail: prevent from modifying subtypes
Prior this commit:
In mail subtype, any users could have perform modification on it.

After this commit:
Now user have only access to read and only admin can do all the modifications.

Task-3245940

closes odoo/odoo#127539

X-original-commit: 1b99f9b6608793612dda72007a4ca2258373a97f
Signed-off-by: Stéphane Debauche (std) <std@odoo.com>
2023-07-06 13:39:52 +02:00
Yannick Tivisse b1f7e56f79 [IMP] base: Remove private res.partner type
- Improve performances, as the ir.rule restricting private partners
  visibility is also applied on res.users by inheritance, on each
  prefetch.
- Solve the issue of partners set as followers on records (eg: application
  form) and then made private, making them impossible to contact via the
  chatter.
- Solve the multiple access issues when trying to access the bank
  account, or the private address for non HR people like the accountants
  forcing the usage of sudo in the business code.

TaskID: 3101400
2023-07-05 14:21:28 +02:00
Julien Van Roy ca4ba84d98 [FIX] mail: UTF-8 text/xml attachment and omitted charset
When parsing an email containing an xml attachment, the `email` python
module will decode the base64 attachment using the charset or ascii if
the charset is missing.

In some cases, the payload is in UTF-8 but the charset is omitted. This
results in replacement characters for the non ASCII characters.

The solution is to force the charset to UTF-8, since it is a superset of
ASCII that should not be a problem.

NB1: Omitting the charset for text/xml is not recommended. See the RFC
(section 6.4): https://www.ietf.org/rfc/rfc2376.txt

opw-3144519

closes odoo/odoo#126474

X-original-commit: e3a5b46c7b4f5030e165a6d943ec2275aea5d583
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Julien Van Roy (juvr) <juvr@odoo.com>
2023-06-27 10:42:49 +02:00
Xavier Morel 074a91a300 [FIX] test_mail: mutlicompany systray test
odoo/odoo#122354 added this test but didn't handle that other models
might trigger systray activities.

On June 12th, the test started failing because calendar has a "Pricing
Discussion" demo calendar event on the 12th of every month. It
probably would have also failed on the 3rd and 22nd which both have
demo meetings for the admin.

Fix by looking up specifically activities of the test model we're
concerned with.

closes odoo/odoo#124684

X-original-commit: 54dce61ef8fa97e18f0cf53d1f098af55e1210e9
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-06-13 10:13:34 +02:00
dane@odoo.com a954eb3196 [FIX] mass_mailing: Fix when reply_model is Null
There exists edge case on `reply_model`. when value of
`mail_messages.model` in database is empty then `mail_messages.model`
returns False, hence `reply_model` becomes False which raises
an error because _get_id does not accept a falsy values.
(because query: `SELECT id FROM ir_model WHERE model=false` has to
be run and model is char). Because of the fact that this query never
runs and fails opportunity is lost.
task-3248489

closes odoo/odoo#123488

X-original-commit: 6c0d2d7a9d44459f3e09a38bd80ef9b018e8c946
Signed-off-by: Stéphane Debauche (std) <std@odoo.com>
2023-06-05 11:49:47 +02:00
Sébastien Theys a3260cfcd7 [FIX] mail: apply multi-company when feching systray activities
A raw query is not necessary to produce the desired result, found
activities need to be kept only if the corresponding record can be found
with standard search (which includes multi-company check).

Part of task-3266643

closes odoo/odoo#123070

X-original-commit: 9dd7ae942aebe2cfd3e2dcd52e10b3bff5c8e0a9
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2023-05-31 14:23:48 +02:00
Daniel Kosky (dako) b1f1f216d0 [IMP] base_vat: VIES validation on Fiscal Position
At present the user can select an option in the settings to check VAT
numbers against the VIES system. If the VAT number fails this
validation the result is a non-blocking banner message that informs the
user that the VIES validation has failed, but has no further
ramifications (the user can still use an unrecognised VAT number).

This non-blocking functionality is still desirable, however we wish to
determine the validity of certain fiscal positions based on whether the
VIES VAT check is valid.

In order to acheive this, the computed boolean `vies_valid` field is
added, and populated based on the results when comparing the VAT against
the VIES system. It depends on the `vat` and `country_id` of the
partner. If it looks like the VIES check needs to be performed on this
vat and if any company in the db requires a VIES vat check, the check is
performed, and if none do, then check is not performed. The field can be
manually edited, but is also tracked.

Provided we know whether a partner has a valid VIES vat or not, it is
only important sometimes in trying to find the appropriate fiscal
position (because VIES is only confirms validity of a VAT number for
intra-community trade). Because of this, a computed boolean field
called `perform_vies_validation` is added to represent this on the
partner. For example if a partner is from the same country as the
current company, then it doesn't matter that it's VIES valid or not, all
that matters is that there is some string in its vat field for the "VAT
Required" to be satsified, so the `perform_vies_validation` field would
be False. This field is also used to determine whether the `vies_valid`
checkbox should be shown or hidden on the partner form view.

A hook called _get_vat_valid is placed in the method on fiscal position
that retrieves the appropriate fiscal position for a given account move,
and it is overridden by a function in base_vat, which specifies whether
the partner/delivery address matches the 'vat_required' condition when
VIES validity is relevant for the company/partner (see the above
`perform_vies_validation` field).

`sale_stock` and `test_mail` performance tests are updated in order to
account for the additional queries introduced in the _get_vat_required
hook (in fetching the base.europe country ids) and the
_compute_vies_valid respectively.

closes odoo/odoo#116391

Task-id: 3218194
Related: odoo/upgrade#4498
Signed-off-by: William André (wan) <wan@odoo.com>
2023-05-17 14:47:01 +02:00
Thibault Delavallée 2d19d07152 [IMP] test_mail: improve message_format performance test setup
Purpose is to improve the depth of 'message_format' performance tests by being
multirecords-enabled, in addition to being multimessages enabled. We also add
more 2many fields in order to better see their effect (notably link previews
and reactions, recently added).

Task-3322905

Part-of: odoo/odoo#121104
2023-05-16 15:55:16 +02:00
Thibault Delavallée c1a6ef5d0d [IMP] test_mail(_sms): update counters to latest runbot
Update counters to match current runbot state. Several changes (ORM, mail
code organization) lead to some counters being obsolete.

Task-3322905

Part-of: odoo/odoo#121104
2023-05-16 15:55:16 +02:00
Julien Van Roy e553f0b372 [FIX] mail,account_edi: fix creation of invoice upon email reception
When receiving an email on a mailbox with an alias that triggers the
creation of invoices, 4 bugs could occur.

1. If the xml received contains replacement characters (U+FFFD �), and
the charset of the part of the email is "US-ASCII" the encoding of the
string will fail, preventing the rest of the flow to be completed. Be
more resilient, encode the string and ignores these characters if this
case occurs.

NB: sometimes, the charset is omitted for a Content-type: text/xml. This
is valid but not recommended (see:
https://www.ietf.org/rfc/rfc2376.txt). In this case, the default used is
"US-ASCII". This means that any non-ascii char will be lost (they are
replaced by the replacement character: �, see:
https://github.com/python/cpython/blob/3.10/Lib/email/contentmanager.py#L67)
when decoding the attachment.

2. When the xml attachment is created in Odoo, the mimetype is
'text/plain' (rather than 'application/xml'). Thus, the
`_decode_attachment` needs to be more flexible when guessing the type of
the attachment (to know which function to use to read the content of the
attachment and create the invoice).

3. When creating an invoice from an email with an xml attachment, the
xml is attached as the `message_main_attachment_id`. It's only later on
that the content of the xml is read and we possibly find the PDF in
base64 inside. When creating the PDF attachment, it was not set as the
`message_main_attachment_id`, so the PDF was not rendered on the right
part of the invoice form view. Add a clause to replace the
`message_main_attachment_id` in such a case.

4. When the xml attachment represents a credit note, the move_type of
the invoice created by the email alias needs to be changed. Indeed, the
invoice is created before decoding the attachment, so we can only change
the `move_type` later.

opw-3144519
opw-3149649

closes odoo/odoo#121076

X-original-commit: 1e193a92b9c84e75b958985f8067873b90f686e0
Signed-off-by: Laurent Smet <las@odoo.com>
Signed-off-by: Julien Van Roy <juvr@odoo.com>
2023-05-11 08:28:42 +02:00
Pierre-Yves Dufays a0cb558048 [IMP] test_mail: add unfollow record test
Add tests that check the behavior of unfollowing a record from the inbox and
the unfollow link in email.

To support unfollowing a document in the inbox no matter the current company,
we have modified message_unsubscribe in mail_thread to allow internal user to
unsubscribe themself without checking any rights. Indeed, some document have
record rule that prevents reading it if the user is not in the right company
(ex. crm.lead) and then was preventing the user to unfollow the document using
that method. We also add a test checking that internal user can unsubscribe a
record without any read and write rights. And that a portal user can't in the
same circumstance.

Task-3061864

Part-of: odoo/odoo#107978
2023-05-10 13:21:07 +02:00
Pierre-Yves Dufays c8b27bd3a0 [IMP] crm, hr_work_entry_holidays, test_mail{_full}: update query count
In order to display the unfollow link or not in emails or in the inbox, the
system need to query the followers of the document. This is what explains the
added queries.

See odoo/odoo#107978

Task-3061864
2023-05-10 13:21:07 +02:00
Renaud Thiry 39f14ca2c9 [IMP] mail: add auto_comment message type
Currently some auto_reply templates are internal as a means to prevent
notifying users everytime we send a reply to somebody.

This raises the issue that since responses to internal notifications
are themselves internal notifications, often nobody will be notified of
responses to these automated messages.

We fix this by marking these messages as auto_comments, which will
ensure responses to these messages are
considered non-internal (and thus 'discussions', by default).

task-2834304

closes odoo/odoo#94018

Related: odoo/enterprise#35466
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-05-05 09:25:42 +02:00
Renaud Thiry c9f55bb227 [IMP] mail: Show reorder template response
If a tracked value sent a mail template on creation of a record,
the template was displayed before the original message in the chatter.

We fix this by using precommit hooks similar to those already used for
tracking.

We also update the query count for a couple of tests. This is required
because in those tests the test user is not in the cache when we execute
the precommit hook, which we use to fetch a fallback language early in
the precommit.

Task-2834304

Part-of: odoo/odoo#94018
2023-05-05 09:25:42 +02:00
Sébastien Theys 90cb44e1e1 [REF] mail, *: rename mail.channel to discuss.channel
* = bus, calendar, crm_livechat, hr, hr_holidays, im_livechat, mail_bot,
    mass_mailing, privacy_lookup, test_discuss_full, test_mail,
    test_mail_full, website_crm_livechat, website_livechat, base

In preparation of splitting discuss and mail modules.

Part of task-3265211

closes odoo/odoo#118354

Related: odoo/upgrade#4553
Related: odoo/enterprise#39661
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2023-04-21 02:21:53 +02:00
Louis Baudoux 1a965a8045 [IMP] mail: add the ability to set the author of the tracking message
A new `_track_set_author` method is now available to explicitly set the
author of the tracking messages.

Related Enterprise PR: odoo/enterprise#38986

closes odoo/odoo#117073

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-04-18 15:38:49 +02:00
Nasreddin Boulif (bon) c78edec4d5 [FIX] mail_thread: save attachment from mail in same encoding
Steps to reproduce:

  - Configure incoming mail server and set it to create X record
    on incoming mails (X can be any model with a chatter)
  - Create a CSV file and set the encoding to UTF-16
  - Send the CSV file through Gmail to the Odoo instance
  - Go to model X and open the created record
  - In the chatter, click/download the CSV file
  - Open the downloaded file with Geany (or any file editor that can
    show the file encoding)

Issue:

  The file encoding is not the same as the original file (utf-8 instead
  of utf-16).
  Working with Outlook.

Cause:

  The difference between Outlook and Gmail is that Gmail provides the
  charset of the file.

  The content of the mail is retrieved using `email` python lib.
  The lib will try to retrieve the charset of the file and fallback
  on `ASCII` if not available, then return the decode content.

```python
  def get_text_content(msg, errors='replace'):
    content = msg.get_payload(decode=True)
    charset = msg.get_param('charset', 'ASCII')
    return content.decode(charset, errors=errors)
```

  Example:
  content = b'd\x00a\x00,\x00,\x00,\......'
  Outlook:
  charset = 'ASCII'
  return => 'd\x00a\x00,\x00,\x00...'
  Gmail:
  charset = 'UTF-16LE'
  return => 'da,,,,,\n,,,,,\....'

  In the post process of the attachment, the content is encoded in
  'utf-8' (to then encoded in base64) before creating the attachment
  record.

  Content encoded to 'utf-8':
  Outlook: b'd\x00a\x00,\x00,\x00...'
  Gmail:  b'da,,,,,\n,,,,,\n....'

  Therefore, when writing the file on the disk, the encoding is based
  on the binary content.

Solution:

  When parsing the mail, add the encoding charset to the `info` variable.
  Then, when creating the attachment, use the charset in `info` (or
  fallback on 'utf'8' if no charset set) to encode the content.

opw-3089009

closes odoo/odoo#118694

X-original-commit: 2d1b13e68d8bb204ecfe12a9d3db519c4264592c
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Nasreddin Boulif (bon) <bon@odoo.com>
2023-04-17 15:18:54 +02:00
Renaud Thiry 5c146ec219 [FIX] mail: compute resource_ref
Before this, the resource_ref field is computed once in the default_get.
This means if the cache is invalidated, its value is unrecoverable.

Notably, the cache is invalidated when an attachment is generated
(triggering a commit).

So calling _generate_template could invalidate resource_ref,
which we use as a key to retrieve its result.

This results in a traceback for the user when trying to preview
a template where the first record hasn't yet generated its attachment.

As an example, trying to preview `Sales: Send Quotation`
on fresh installs.

task-3162320

X-original-commit: b4bd93c9b61736a7f41d70afe5db977a6e4b35e0
Part-of: odoo/odoo#118710
2023-04-17 11:47:11 +02:00
Renaud Thiry 20fe457377 [FIX] mail: set error_msg in template_preview
If the preview wizard was used to preview a template on a model that
has no record, error_msg would not be set.

This means the field is never set in that case and creating the wizard
results in a cache miss on that field.

The fix is to simply set it, and we add a test to cover that flow of
the wizard.

task-3162320

X-original-commit: 0976ce53e0a97a1f49dc2a262cf66355242fde25
Part-of: odoo/odoo#118710
2023-04-17 11:47:11 +02:00
Pierre-Yves Dufays ed8a101a08 [FIX] test_mass_mailing, {test_}mail: always apply blacklist for mass mailing
Always apply the blacklist in mass_mail composition mode regardless of the
recipient model implementing mail.thread.blacklist or not.

This solves the problem of mail sent to black listed address for model not
inheriting from mail.thread.blacklist.

Technical notes:
- it has been done in mail.compose.message _get_blacklist_record_ids ignoring
the mixin mail.thread.blacklist to avoid model change in stable.
- some tests have one added query because the blacklist is now queried for each
batch mail sends even if the model of the recipient doesn't implement
mail.thread.blacklist.

Task-2834862

closes odoo/odoo#118497

X-original-commit: 263e86114c60650f421354e165364afcd4122461
Signed-off-by: Dufays Pierre-Yves (pydu) <pydu@odoo.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-04-13 17:57:56 +02:00
Martin Trigaux 69f911d994 [IMP] *: enforce usage of Markup in mail
When using message_post, the body format must be explicitly specified.
If html is expected, a Markup object should be used.
If text is given, the content will be escaped.

Before this PR:
message_post was unaware if the content of a message was HTML or
text. This lead to multiple situation where the content was
incorrectly considered as HTML and led to display errors.
In
  self.message_post(body="Hello %s!" % self.name)
if the name contained HTML, it would be evaluated.

In
  self.message_post(body="Contact Raoul <raoul@caramail.be>")
the email would not be displayed as considered as unknown HTML and
discarded by the sanitizer

Now each call must explict the type of content.
Use the escape() helper to properly combine Markup and translations.
It would also be acceptable to use Markup() to wrap a static
translation but escape is better as one can not guarantee the content
of a translation.

closes odoo/odoo#111850

Related: odoo/documentation#3612
Related: odoo/enterprise#36728
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-04-13 16:39:48 +02:00
Mahamadasif Ansari 5ad9686b2f [FIX] mail: composer non-thread _compute_subject
When using the composer on a non-thread model a notification is sent instead
of posting a message, as the model does not support the posting process.
However there is currently a crash due to the default subject computation
which is solved in this commit.

Task-3254379

Part-of: odoo/odoo#117763
2023-04-05 15:11:33 +02:00
Renaud Thiry 442467ceb8 [FIX] mail: handle non-thread model in _message_format
Calling _message_format on a anything but a thread could throw a traceback.

Task-3254379

Part-of: odoo/odoo#117763
2023-04-05 15:11:33 +02:00
mky-odoo 2cc2690c8f [FIX] mail: prevent to call _message_compute_subject outside of a thread model
While we run cron `Data Merge: Find Duplicate Records` an issue raises because
`data_merge.model` has no method `_message_compute_subject`. Indeed it is
possible to notify on non-thread models but that method is a thread-only
method.

In this commit, we only call `_message_compute_subject` when it's available
in model.

sentry-3947126655
Task-3254379

Part-of: odoo/odoo#117763
2023-04-05 15:11:32 +02:00
Thibault Delavallée c9848bbf58 [REF] test_mail(_*): simplify Mail-based test class usage
'TestMailCommon' was useless and is replaced by the 'MailCommon' class defined
directly in Mail addon, easing inheritance and imports.

Task-3263512

Part-of: odoo/odoo#117606
2023-04-04 20:08:37 +02:00
Yolann Sabaux 71ed187223 [FIX] account: display pdf attachment
Steps to reproduce:
- Install l10n_mx modules
- Switch to MX company
- Create invoice and confirm it
- Send it to PAC in test environment (Process Now button)
- Click Send & Print button

Issue:
- XML Preview does not display, only shows the file name in the top right corner of the chatter.

Cause:
In l10n_mx_edi, when posting the invoice, the only attachment available is the xml sent to the government in `_message_set_main_attachment_id`.
Therefore the xml is set as the main attachment.
When clicking on "Send and Print", the pdf is generated but the main attachment is still the xml

Solution:
Redefine the main attachment everytime the the main attachment is an xml

Note:
- octet-stream have also been filtered out

opw-3085934

closes odoo/odoo#116784

X-original-commit: 48e6e81a47f5371f5985d135e56f39f445eb2854
Related: odoo/enterprise#38861
Signed-off-by: Yolann Sabaux (yosa) <yosa@odoo.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-03-28 13:43:57 +02:00
Pierre-Yves Dufays 6cfbcc46a7 [IMP] test_mail: add template preview test about recipients
Add a test that check recipients displayed in the template preview. This test
has been added following the fix that solves the template "partner_to" not
displayed in the preview.

Task-3196081

X-original-commit: 86a492275e56217a5edd57d99a0db7576aa91377
Part-of: odoo/odoo#116431
2023-03-24 10:04:46 +01:00
Walid HANNICHE (waha) 27d4a74724 [FIX] mail: set correct reply_to company
Steps to reproduce:
- select a different company from the main one
- under settings/discuss enable External Email Servers
- set up an alias domain
- create an SO and send it by email
(you can catch the sent email using mailhog)
- reply to that email
(you can use the support-tools[1] and set In-Reply-To: "previous message_id")

Bug:
the reply_to field of incoming message defaults to the first company

Fix:
set the reply_to field to the company asociated to the record
(there's already a fallback to self.env.company  in
"_notify_get_reply_to_formatted_email")

opw-3060214
[1]: https://github.com/odoo/support-tools/tree/master/scripts/mail

closes odoo/odoo#115090

X-original-commit: d86e57404d540762fa51afeabeddd3a0fc45d4d5
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Walid Hanniche (waha) <waha@odoo.com>
2023-03-13 18:45:11 +01:00
Thibault Delavallée c2f8b03e3f [IMP] mail: support layouting in composer mailing mode
RATIONALE

Naming conventions
  * content = core message / subject (+ other fields);
  * email layout = layouting applied to messages when sent to recipients who
    receive notifications by email e.g. access button, ...;
  * comment mode / mailing mode: composer main composition mode: posting on
    record(s) / sending a mailing on records;

Content situation when using a template to post on a composer

  * comment, monorecord mode: choose a template, content is rendered directly
    from template side according to 'lang' field definition -> ok;
  * comment, multirecord mode / mailing mode: choose a template, content is
    the raw content. At post / send, content is rendered and translated if
    matching template content -> ok;

Email layout situation when posting / mailing

  * comment: layout translated based on recipient lang or fallback on template
    'lang' field definition, or current user's lang;
  * mailing: no layout supported;

PURPOSE

Make those the two composer modes less different by supporting layouting in
mailing mode of composer. Consider it a bit experimental.

SPECIFICATIONS

When using the composer in mailing mode, render email layout if given. This
layout is used to encapsulate body.

To simplify rendering and sending, consider all recipients to be 'customer'
as defined in '_notify_get_recipients'. This means all recipients receive the
same basic layouting currently as a first attempt to improve this situation.
Considered lang is the one coming from the template 'lang' field, or fallback
on current user's lang.

Task-3186426 (Mail: Support notification layout in template / email composer)

Part-of: odoo/odoo#106177
2023-03-09 15:54:15 +01:00
Thibault DelavalléeandJulien Banken b4d210b245 [REF] mail: split notification email generation by recipient language
RATIONALE

Naming conventions
  * content = core message / subject (+ other fields);
  * email layout = layouting applied to messages when sent to recipients who
    receive notifications by email e.g. access button, ...;
  * comment mode / mailing mode: composer main composition mode: posting on
    record(s) / sending a mailing on records;

Content situation when using a template to post on a composer

  * comment, monorecord mode: choose a template, content is rendered directly
    from template side according to 'lang' field definition -> ok;
  * comment, multirecord mode / mailing mode: choose a template, content is
    the raw content. At post / send, content is rendered and translated if
    matching template content -> ok;

Email layout situation when posting / mailing

  * comment: layout translated based on template 'lang' field definition
    or fallback on current user's lang;
  * mailing: no layout supported;

PURPOSE

Translate layout into recipient's lang when available, instead of either
current user or template defined lang. As it is contextual better translate
it to the end user lang.

SPECIFICATIONS

Move recipients and email layout rendering into a method returning an iterator.
That way a list of recipient groups (as given by '_notify_get_recipients') can
be split into sub-groups, with each having its own rendering values for
email notifications.

Do a split / recipient lang. It is already fetched when notifying messages
as part of '_get_recipient_data' returned values. We now return recipients
groups per lang and per usage type (user, portal, customer, ...). Rendering
values are computed for each group as several values depend on the lang (
actions, notification buttons, global layout, ...).

Layouts when posting are dependent on recipient lang, and not on current user
anymore. Template lang that forced the content lang is now used only as a
fallback when recipient have no lang.

SUMMARY

When a user in Spanish, posts using a template specifying the lang of the
customer on a record whose customer is in French, with a follower being
in English

  * content will be translated following template choice, aka french;
  * layouting for the french customer will be in french, layouting for the
    english customer is in english, while the core content is always the
    same and in french;

When a user in Spanish, posts some custome content on a record whose customer
is in French, with a follower being in English

  * content is whatever the user wrote;
  * layouting for the french customer will be in french, layouting for the
    english customer is in english, while the core content is always the
    same and in french;

Task-3046371 (Mail: Better Language Support in Composer)
Task-2555155 (Mail: Translate notification action/access buttons)
Task-3186426 (Mail: Support notification layout in template / email composer)

Part-of: odoo/odoo#106177
Co-authored-by: Julien Banken <jbn@odoo.com>
2023-03-09 15:54:15 +01:00
Thibault Delavallée 9887bd0c7b [IMP] mail: correctly propagate language when posting from composer
RATIONALE

Naming conventions
  * content = core message / subject (+ other fields);
  * email layout = layouting applied to messages when sent to recipients who
    receive notifications by email e.g. access button, ...;
  * comment mode / mailing mode: composer main composition mode: posting on
    record(s) / sending a mailing on records;

Content situation when using a template to post on a composer

  * comment, monorecord mode: choose a template, content is rendered directly
    from template side according to 'lang' field definition -> ok;
  * comment, multirecord mode / mailing mode: choose a template, content is
    the raw content. At post / send, content is rendered and translated if
    matching template content -> ok;

Email layout situation when posting / mailing

  * comment, monorecord model, using a template from UX: layout is partly
    translated based on template 'lang' field to match the content translation.
    Part of the layout still uses the current user lang;
  * other comment use cases: layout is translated based on current user lang;
  * mailing mode: layout not supported;

PURPOSE

Correctly propagate lang from template in all situations whenever a template
is used, and translate all layout parts.

SPECIFICATIONS

When using the composer to post messages (either as comment or in batch) the
language coming from the template is not propagated until the notification
process.

A hack has been done to try to guess the language inside the notification
process, based on context keys at odoo/odoo@9e71d228ed. This was done to partly fix
the bug in stable.

Since odoo/odoo@3eb9680602 it is possible to propagate a lang from 'message_post' or
'message_notify' calls until the notification process. We can therefore call
the posting methods using the rendered lang (in both mono and multi record
modes) and remove that hack.

When a lang is propagated using the 'force_email_lang' it is used to choose
the language of the layout (content, buttons). Currently all recipients
receives the same language for layouting. We plan to soon use the recipient
language when possible, then fallback on that forced lang. This will offer
more granularity and a better user experience.

When using scheduled messages (creating message but delaying the sending of
notifications) it is also working as notification parameters are saved when
creating the scheduling record, see odoo/odoo@9b11a9d82b.

EXAMPLE

A user in english uses a template on a lead whose customer is in spanish
with a follower being in german. Template lang is customer's lang:

  * content is translated into spanish (customer's lang);
  * layout is translated into spanish (customer's lang);

Both recipients (customer and follower) receive the content in spanish.

LINKS

Task-3046371 (Mail: Better Language Support in Composer)
Task-2555155 (Mail: Translate notification action/access buttons)

Part-of: odoo/odoo#106177
2023-03-09 15:54:14 +01:00
Thibault Delavallée 110d86f504 [IMP] mail: allow composer to translate from template
RATIONALE

Naming conventions
  * content = core message / subject (+ other fields);
  * email layout = layouting applied to messages when sent to recipients who
    receive notifications by email e.g. access button, ...;
  * comment mode / mailing mode: composer main composition mode: posting on
    record(s) / sending a mailing on records;

Content situation when using a template on the composer

  * comment, monorecord mode: choose a template, content is rendered directly
    from template side according to 'lang' field definition -> ok;
  * comment, multirecord mode / mailing mode: choose a template, content is
    the raw content. At post / send, content is rendered on composer side.
    As composer is not translated -> no translation -> ko;

PURPOSE

Take translations from template when composer content is the same as the
template one to benefits from their translations.

SPECIFICATIONS

When rendering some fields fetching translations can be complicated. Indeed
when using a template the translation is stored on template model while the
rendering is done on composer model. Translations are therefore not fetched
as fields of the composer record itself are not translated. It is a transient
record, not something people translate manually like templates.

As translations are not stored in a table anymore we can't really fetch
translations, except from using directly the template field. In this commit
we therefore do the rendering based on template value instead of composer
value when they are considered as equal and if a translation is asked either
through 'compute_lang' of 'force_lang'. This allows to fetch template
translations instead of composer translations.

If the composer content has been modified compared to the template we keep
the old behavior, which means probably no translations. Let us hope editor
does not mess too much with html content.

EXAMPLE

A user in english uses a template on a lead whose customer is in spanish
with a follower being in german. Template lang is customer's lang:

  * content is translated into spanish (customer's lang);

Concerning layout (to be fixed in next commits):

  * layout is partly translated into spanish (customer's lang) if coming from
    form view, otherwise is in english (current user's lang);

LINKS

Task-3046371 (Mail: Better Language Support in Composer)

Part-of: odoo/odoo#106177
2023-03-09 15:54:14 +01:00
Thibault Delavallée abd93a6374 [IMP] mail: reset composer mixin fields when removing template
Purpose of this commit is to improve behavior or mail.compose.mixin when
removing the template. It now voids the value to avoid having half baked
definition on composers.

Summary of behavior

  * when choosing a template: existing (non void) values override any value
    in the composer mixin;
  * when removing a template: reset values;
  * when going from templateA to templateB: works like choosing a template
    which means only non void values are taken. This may lead to half-baked
    content but trying to remove old template content based on _origin and
    content comparisons would be complicated for few added value;

This makes the 'mail.compose.mixin' behaves like 'mail.compose.message'
wizard which was recently updated at odoo/odoo#107356.

Task-3093257 (Mail: The Composer Update)

Part-of: odoo/odoo#106177
2023-03-09 15:54:13 +01:00
Thibault Delavallée 59a2bb1403 [IMP] mail: improve body / template sync check in composer mixin
RATIONALE

Purpose of this commit is to add helper field knowing if body any model
inheriting from the 'mail.composer.mixin' is synchronized with their
template value.

SPECIFICATIONS

Add a field to check if body of a composer is the same as its template

  * compare both current (raw) values and sanitized values, as sanitization
    process may change some bits of html code;
  * correctly check for void content using the tool;

Obviously this may be broken if editor usage changes the source template value
DOM when saving it in the composer. However we can't do much more than using
the original and sanitized values from template and compare them with the
current composer values.

It is used in this commit to simplify a bit code in '_render_field' override.
It is used to detect any change in body based on template. It highlights
potential changes in QWeb directives which requires usage of editor group.
This method is also cleaned in order to have less calls to super() in nested
if-elif structures. As some additional code will be added soon it is better
to have a simpler code flow.

Finally add some tests about current state of fields synchronization notably
when setting a template with void values, or removing the template.

Task-3046371 (Mail: Better Language Support in Composer)

Part-of: odoo/odoo#106177
2023-03-09 15:54:13 +01:00
Thibault Delavallée 0f6f7c3676 [IMP] mail: make composer mixin take lang from template
Mail composer mixin already synchronizes subject and body based on a mail
template, easing calling rendering methods available through mail render
mixin.

In this commit we add the same mechanism on lang. Lang is taken from the
template and can be used to render content based on language defined on
template, generally based on a customer.

Some query counters are updated, without being sure those are really coming
from this commit. Anyway they will be updated in batch at the end of this
PR, as tracking them on each commit is tedious and not always really
useful.

Task-3046371 (Mail: Better Language Support in Composer)

Part-of: odoo/odoo#106177
2023-03-09 15:54:13 +01:00
Thibault Delavallée 38e78e00c0 [IMP] mail: improve email_layout_xmlid propagation tests
Some additional tests related to odoo/odoo@2b9e288e1f

Task-3046371 (Mail: Better Language Support in Composer)

Part-of: odoo/odoo#106177
2023-03-09 15:54:13 +01:00
Thibault Delavallée c088d5423e [IMP] mail: cleanup _notify_get_recipients code bits
This commit contains mainly code cleaning, docstrings and a small split
for notification tool methods. In this commit we

  * make some notification groups variable explicit;
  * move the filler of groups into its own submethod to ease being called
    from other code (to be used soon);
  * fix some strange overrides or code manipulation;
  * propagate some additional parameters to ease future commits that will
    improve rendering of groups-based notification emails;
  * cleanup, fixup and improve docstrings;

This does not change anything from functional point of view, just preparing
further work.

Task-3046371 (Mail: Better Language Support in Composer)

Part-of: odoo/odoo#106177
2023-03-09 15:54:12 +01:00
Pierre-Yves Dufays 0a527ace82 [IMP] test_mail: add tests to justify sudo when writing attachment
Add message post with attachment test on record of another company than the
user company but for which the user has readonly access on. The goal is to
justify the sudo added in mail_thread:

  - in '_message_post_process_attachments' method to create attachment linked
    to the record (the creation of attachment linked to a record verify the
    right to write to the record, so in readonly we need a sudo);
  - in '_message_set_main_attachment_id' method to link the main attachment to
    the record. Here we write directly on the record for which we only have
    readonly access, so a sudo is needed. Here the attachment is already linked
    to the model and we only "tag" it has main attachment (see odoo/odoo#30659)

Task-3178885 (Mail: fix 'readonly' post support)

X-original-commit: odoo/odoo@8486c7ee05
Part-of: odoo/odoo#114511
2023-03-07 18:03:02 +01:00
Thibault Delavallée 5edb04d491 [IMP] test_mail: add posting tests in multi-company environment with attachments
Purpose of this commit is to add tests to post in record user cannot access
especially in multi company environment.

Main observations

  * ACLs are checked when fetching record_name, then when fetching reply_to
    information, adding a bit of ACLs noise in the process;
  * check_access_rule of mail.message is called when creating the message
    (which is the real protection);
  * posting on a document user cannot see is possible when they have been
    notified of a parent message. Use case is: answer a ping, even on a
    document non readable, should always be doable. However it currently
    crashes notably due to attachments check, who always require the read
    access on the related document even if the message is authorized;

In this commit we also move other tests about multi_company in the right
file (and add a test flag) in order to have all of them in the same place
easing debug and testing.

Task-3213982 (Mail: fix 'multi-company' post support)

X-original-commit: odoo/odoo@5656f4d20d
Part-of: odoo/odoo#114511
2023-03-07 18:03:02 +01:00
Thibault Delavallée 248e0e495b [IMP] test_mail: add composer tests for email layout support and translation
Purpose is to add some tests about email layout support in both comment and
email mode, as well as language support in those two modes. They are added
in a test targeting a complete template usage, testing globally the results
(outgoing notifications or emails, buttons, language, recipients, ...).
Posting in multi-languages environment with notification layout is also
tested. Finally a test about reply-to is moved into its own subtest to
avoid polluting core template tests. Other low level details should already
sufficiently tested. Normally.

Main observations

  * translations are quite broken in mass mode (either batch comment either
    mass mailing). Indeed translations are fetched based on <mail.compose.
    message> body and subject fields, which are probably not translated
    as they are linked to a transient model. Real translations are stored
    at <mail.template> level when users translate their templates;
  * layout translation is partly supported in comment mode, due to a small
    context-based hack. However access buttons are not translated and some
    use case (like using a 'res_domain') are not supported;
  * mass_mail mode does not support layouting at all;
  * when posting manually ('message_post') current users' language determines
    the language of notification email layout e.g. a user using Odoo in French
    will send a french layout to all followers, whatever their language and
    whatever the language of the post (which is not known as user input);

Most of those use cases will be fixed soon.

Also update some query counters while passing by.

Task-3046371 (Mail: Better Language Support in Composer)

Part-of: odoo/odoo#114511
2023-03-07 18:03:01 +01:00
Thibault Delavallée 3147d09b90 [IMP] (test_)mail: improve test tooling
Allow to return <mail.mail> records and found outgoing emails when using
asserts. It eases doing checks in some specific tests e.g. checking
notification layout usage in emails. Indeed this is quite low level and
does not really deserves its own assert tooling methods.

Update tool method generating attachment data to ease name tweak when required
in tests.

Also update query counters, and fix a test not running locally.

Task-3046371 (Mail: Better Language Support in Composer)

X-original-commit: odoo/odoo@b32d937b37
Part-of: odoo/odoo#114511
2023-03-07 18:03:01 +01:00
Thibault Delavallée 2b9e288e1f [IMP] mail: support email_layout_xmlid on template
RATIONALE

Currently email layout choice is done in code and is not controllable through
any user interface. In this commit we prepare improvements in template
management by allowing to choose the email layout directly from a given
mail template. As number of layouts is going to grow better be able to
choose the right one.

SPECIFICATIONS

Add a field on template to propagate email layout choice directly on
template and propagate it to the composer, then post/notify process.

We can now choose directly on a template which notification layout should be
used in conjunction with this template. Composer model already holds a field
'email_layout_xmlid', used in various places in code to give a specific
layout to use when sending notifications emails e.g. SO email layout. This
can now be specified directly from the template itself. It allows to
customize the look and feel of notifications without having to do it
explicitly in a given code flow.

Computation is coming either from template, either reset. When having a
template with a value set, set it on composer. When removing the template
reset it.

Currently no standard template uses it as this task targets mainly a code
cleaning that began with the composer code cleaning. It targets the near
freeze to have the base code updated once, then functional templates will
make use of it (hopefully).

Note that currently notification layout is still not supported at composer
level for mailings. It is simply ignored when generating outgoing mail
records. This will be improved soon.

Global followup of odoo/odoo#107356 / Task-2088884 (Mail: Use editable
computed stored fields in composer).

Prepares code for Task-3046371 (Mail: Better Language Support in Composer)
see odoo/odoo#106177 .

Task-3186426 (Mail: Support notification layout in template / email composer)

closes odoo/odoo#114462

Related: odoo/upgrade#4404
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2023-03-07 02:56:46 +01:00
std-odoo 97ca45e4f3 [IMP] mail, mass_mailing: store the bounce email and allow the user to read it
Purpose
=======
The bounce emails aren't stored in Odoo, which can complicate the
debugging of the email sending.

Now, we store this bounce email, and we allow the user to read it from
the interface, so he can easily find the issue when an email sending
fail.

Specifications
==============
The bounce email is stored on the mail notification for standard emails
sending, and on the mailing traces when using mass mailing.

For some email providers (e.g. Yahoo), the "Final-Recipient" header is
not present. Normally, it allows us to retrieve the original recipient
of the email which bounced and then the partner. So if this header is
not there in a bounce email, we take the first recipient of the parent
<mail.message>.

Change the way that we parse the email body, for the bounce email.
For most email providers, the first mail body is the one that contains
the error and the next one contains the parent email body. So, the
current logic might ignore this body for Outlook and Yahoo.

Task-2116296

closes odoo/odoo#105923

Related: odoo/enterprise#34051
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
2023-03-06 17:47:29 +01:00
Raphael Collet 8659a4e5a3 [FIX] test_mail: hack to make broken test work anyway
The test is based on a record that the user cannot read.  It used to
work by accident, because some side effect leaked data in cache.

Part-of: odoo/odoo#112126
2023-03-05 15:12:56 +01:00
Raphael Collet 44d336128d [FIX] mail: overrides of _search()
This is about the overrides of _search() on models mail.activity and
mail.message, which both make extra queries to implement their specific
access rights.  We combine both queries made in _search() to retrieve
accessible records.  This simply uses the API of the Query object to
retrieve the data that is necessary to restrict access to messages.

Move security check outside of _message_format() for performance.  The
call to check_access_rule() inside _message_format() was redundant in
many cases and generated more SQL queries than necessary.

Also for performance, accessing fields from records should not actually
check permission.  That's a bit freaky, but this reproduces the former
behavior of mail.message.

And finally, make method check_access_rule() on mail.message check
ir.rules, in order to make it consistent with method _search().

Part-of: odoo/odoo#112126
2023-03-05 15:12:56 +01:00
Raphael Collet 7e6cff5479 [IMP] core: search() and _search() no longer have parameter count
The parameter in search() is redundant with method search_count(), and
was making the calls less readable.

The method _search() is aimed at always returning a Query object.  The
method can therefore never return an integer, hence the removal of the
parameter.  This does not actually remove any functionality from the
method; counting result is simply given by using it differently.

Part-of: odoo/odoo#112126
2023-03-05 15:12:54 +01:00
Pierre-Yves Dufays ffd976cc69 [IMP] {test_}mail: make some improvements to the code, no functional changes
Some code cleanup done when moving the main attachment into its own mixin:
- improved docstring
- early returns
- code simplification
No functional changes should occur with this commit.

Task-2648976

Part-of: odoo/odoo#110734
2023-03-02 12:32:11 +01:00
Pierre-Yves Dufays 4991dbc9a8 [REF] mail: adapt test using new main attachment mixin
Task-2648976

Part-of: odoo/odoo#110734
2023-03-02 12:32:11 +01:00