RATIONALE
As multi-company tolerant alias domains will soon replace the usage of
configuration parameters, having them in base then replaced by more advanced
models in mail would be complicated to handle and not useful. Move those
ICP to 'mail' so that all mail configuration is done in that module.
SPECIFICATIONS
Move config parameter used for alias domains configuration in 'mail' module.
Base should be as simple as possible and let mail deal with mail server
complexity.
Move 'mail.{bounce/catchall}.alias' used with 'mail.alias.domain' to make
bounce and catchall emails. Move 'mail.default.from' as it will be integrated
into alias domains in some form.
Note that 'mail.default.from_filter' stays as an ICP in base as it is a
more global default parameter. It is used as default value in 'connect' when
no mail_server is used and no from_filter can be retrieved.
Some tests in 'base' are either fixed, either moved directly into 'mail'.
We now differentiate base behavior (without ICP) from configurable behavior
(with ICP in mail).
Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130750
From filter could be ill-defined, like ' ' or ','. This commit just make
some code more defensive against those values.
Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130750
Split '_get_test_email_addresses' into two methods allowing to generate the
'from' and 'to' when testing SMTP connection. As 'email_to' is always the
same better have a small method for it. Moreover it eases overrides if
some code wants to tune the from / to by overriding only the necessary one.
Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130750
Prepares the move of ICP to mail before replacing them by dynamic alias
domains. Improve test coverage, notably for edge cases. Continue to make
tests more explicit after odoo/odoo#131492. Some tests are also merged to
lessen number of different tests when possible, notably when only a test
parameter differs (like giving an SMTP session or not).
Clean ICP and mail servers setup in test classes allowing to remove some
unnecessary extra initialization. Cleanup a mock in mail.
Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130750
RATIONALE
This prepares the move of ICP to mail before replacing them by dynamic alias
domains.
Cleanup tests: try to use loops with input / expected to better understand
the various test cases, add some comments, improve logs when failing to
find the right sent email. Rename tests to have a better test structure
when reading logs.
Remove a test from odoo/odoo@3b6c20805c that adds nothing except testing
the test suite.
OTHER ADDONS
In test_mail: have a specific class for testing servers as other data is
not necessary, and it allows to have a tag for it.
In mass mailing: concatenate test about server finding, several tests can be
done in a single unit test.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
closesodoo/odoo#131492
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Have all ir.mail.server tests as well as their configuration-related tests
moved into 'test_ir_mail_server' file. That way all tests are contained in the
same file, easing their update.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130768
Rename alias domain and aliases used a test data. This allows to make
them easier to read, follow, grep and understand.
Activate multi-company on alias and gateway tests, ensuring it currently
has few impact on tests.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130768
Purpose
=======
Some SMTP configuration can be used for many domains names. E.G.,
our own SMTP configuration can be used for both "mail.odoo.com" and
"odoo.com". So we want to be able to set a list of allowed domains
name, separated by a comma, instead of a single domain.
Task-3332488
closesodoo/odoo#122234
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Bug
===
When the from_filter of an outgoing mail server contains a domain
(e.g. company.com), and when the system parameter "mail.default.from"
is a full email address, with a different domain name
(e.g. notification@example.com) we concatenate both value which produce
an invalid email (e.g. notification@example.com@company.com).
Instead, we always give the priority to the from_filter, and fallback
on "noreply" for the local part when needed.
Task-3230917
closesodoo/odoo#115890
X-original-commit: bf02481de9932de2e0246ac36fcf59f37a89b511
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Since python/cpython#18544, unittest mock is not able to properly find
the "odoo.tools.config".
When trying to patch the `options` dictionnary, it leads to
`AttributeError: module 'odoo.tools.config' has no attribute 'options'`
In order to have the tests
working in python <= 3.11, we have to import the config module and patch
the options in place.
Part-of: odoo/odoo#112450
Purpose
=======
On our saas, when users configure a mail server, they can't use anymore
the "odoo.com" configuration (which is stored in the odoo-bin argument).
To allow them to continue using the SMTP CLI configuration, we add a
new "smtp_authentication". When this authentication method is chosen,
all the "connection" fields of the mail server are ignored, and the
connection information are taken from the odoo-bin arguments. So they
can choose the from filter of this SMTP configuration, the priority...
Task-3061882
closesodoo/odoo#106297
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
During tests, when matching ir.mail_server has `smtp_encryption` set to
`starttls`, connect() will crash with an AttributeError as follows:
2022-09-22 07:04:27,882 20445 ERROR tests-150-starttls odoo.addons.base.tests.test_ir_mail_server: ERROR: TestIrMailServer.test_mail_server_send_email
Traceback (most recent call last):
File "/home/odoo/src/odoo/15.0/odoo/tools/misc.py", line 804, in deco
return func(*args, **kwargs)
File "/home/odoo/src/odoo/15.0/odoo/addons/base/tests/test_ir_mail_server.py", line 261, in test_mail_server_send_email
IrMailServer.send_email(message, mail_server_id=self.server_domain.id)
File "/home/odoo/src/odoo/15.0/odoo/addons/base/models/ir_mail_server.py", line 575, in send_email
smtp = self.connect(
File "/usr/lib/python3.8/unittest/mock.py", line 1081, in __call__
return self._mock_call(*args, **kwargs)
File "/usr/lib/python3.8/unittest/mock.py", line 1085, in _mock_call
return self._execute_mock_call(*args, **kwargs)
File "/usr/lib/python3.8/unittest/mock.py", line 1146, in _execute_mock_call
result = effect(*args, **kwargs)
File "/home/odoo/src/odoo/15.0/odoo/addons/base/models/ir_mail_server.py", line 313, in connect
connection.starttls(context=ssl_context)
AttributeError: 'TestingSMTPSession' object has no attribute 'starttls'
This commit adds a dummy `starttls()` method to `TestingSMTPSession` to
gracefully handle such case.
closesodoo/odoo#101962
X-original-commit: ec156901dd4db2fc7a2827b1eb78515f6259699e
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Xavier Alt (xal) <xal@odoo.com>
mail_* = mail, test_mail, test_mail_full, test_mass_mailing
PURPOSE
Improve tooling of mail tests. Use SMTP mockup class available in base tests
in mail tests to have a better and simplified mock tool when dealing with
mail tests. Remove old fashioned code.
SPECIFICATIONS
Include ``MockSmtplibCase`` directly into ``MockEmail`` so that we re-use
existing components for mocking outgoing emails. Also make parameters for
gateway coherent between those two classes (use same test values).
Remove 'sim_error' weird parameter of Mail ``mock_mail_gateway`` and use
standard mock 'side_effect' instead in some specific tests, notably on
SMTP.connect() mock.
Add some assert / tools method and improve existing tools, used for creating
or improving test data. This will be used in future commits when adding new
tests.
Add outgoing mail servers data in mail tests, allowing to make tests closer
to real life use cases.
Remove unnecessary calls to ``_init_mail_gateway`` as it is automatically
done in ``MailCommon.setUpClass()``.
Try to consolidate starting test data, notably with countries, phone numbers
or email in mind to be sure test mail are deterministic.
Task-2673913 (TestMail: Cleanup and improve test coverage)
Part-of: odoo/odoo#86393
Purpose
=======
Allow a full email address in the system parameter "mail.default.from".
This is useful if we want to send the emails with a domain different
from the domain used to receive them. E.G. a user on our SaaS can
configure an outgoing mail server for Outlook, use his email address as
the default one (so all emails will be encapsulated into his email
address) but keep our default configuration to receive the emails
(so the catchall domain still remains "mycompany.odoo.com").
Task-2738816
closesodoo/odoo#83325
X-original-commit: 21f685d1f58690e1c9c1c6198461652b108e1e18
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
We have our own html2plaintext, already used in lot of use cases instead of
just a few for the html2txt library.
Notably for emails: most emails going through Odoo stack use our simple
html2plaintext to format the body alternative. When no body alternative
is given to ``build_email`` an alternative is built using the library to
remove. Using our own parser allows to have the same results compared to
using ``MailMail.send()``. Difference lies in spaces and new lines as well
as markdown. Our html2plaintext is a bit simple and does not try to generate
Markdown but generates a simple plaintext version.
This also helps solving some issues with depending on that library.
Task-2702034
closesodoo/odoo#82486
X-original-commit: b3b9627b655cd7cb928925affed6cc8d92661e8d
Related: odoo/enterprise#23364
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose of this commit is to add some tests about body alternative done
in build_email when none is given. It uses a library we are about to
remove and testing it is therefore necessary.
Task-2702034
X-original-commit: 92b102c87bbdb9073f794a414a41460e4146acd2
Part-of: odoo/odoo#82486
When the system broadcasts an email response to document followers,
if the config parameters `mail.force.smtp.from` or
`mail.dynamic.smtp.from` are defined, it will rewrite the `From`
address to avoid spoofing the sender's domain.
**NOTE**: As of 15.0, this is based on the `from_filter` setting on the
corresponding ir.mail_server, rather than the abovementioned config
parameters, but the rest of the discussion stands.
For example, if the `mail.catchall.domain` is set to `example.com` and
an email response comes from:
"John D" <john@doe.com>
it will rewrite it to:
"John D (john@doe.com)" <notifications@example.com>
This will make sure the system never sends outgoing email for an external
domain, as it has no authority for doing so, and that could
break mail filtering/authentication rules (SPF, DMARC, etc.)
During this "encapsulation rewrite step", both the original Sender name
and their email are preserved, and put into the quoted "name" field of
the rewritten address. It seems sensible to preserve as much information
as possible about the original sender.
Unfortunately, the inclusion of the Sender email in the final name makes
it appear to some inbox providers as if the message is trying to
deceptively impersonate another person (as many phishing schemes would).
As of November 2021 GMail at least does this, and will hide the name in
the UI when it happens. It will keep only the rewritten email, which is not
very useful in the case of a notification (even though it's more
technically correct, of course).
This patch removes the original email from the rewritten notification,
keeping only the name, considering that the email is not the most
important part, and it's better to have one of the two than none.
So after the patch, the rewritten address is now:
"John D" <notifications@example.com>
When there is no name in the original address, we keep only the local
part of the email, to avoid the same display issue. The recipient will
have to identify the sender based on the context / past messages.
closesodoo/odoo#81807
X-original-commit: 3c65ec5a8191a392980ceb0a8c584767eae405f1
Signed-off-by: Olivier Dony <odo@odoo.com>
Purpose
=======
When there are multiple databases on a single server, it is necessary
to be able to set the "from_filter" for the implicit SMTP server
on a per-database basis, to allow 'opt-in' to the automatic wrapping
of email notifications.
When set to e.g. `notifications@example.com`, and a default SMTP
server is defined in the server-wide config or CLI, outgoing emails
will be automatically rewritten to come from this email, unless the
From/Return-Path domains match the `mail.catchall.domain` parameter.
Task-2710632
closesodoo/odoo#81277
X-original-commit: 8b468e1f7a3dfeb2bcfa83bff55c4b5adf6b2212
Signed-off-by: Olivier Dony <odo@odoo.com>
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
Thibault Delavallée