Composer tests now use the multi-company enabled model by default, allowing to
test company-dependent behavior. This has no impact on current tests, as there
is no company-dependent fields on composer model, and all tests are anyway
run into the main company (except multi-company specific tests, suffixed
by '_mc' generally).
We therefore also add some multi-company oriented tests to check notably
return-path or environment companies in various scenarios. Go until the SMTP
generation to test mail server choice, smtp_from and filtering, notifications
email.
Followup of odoo/odoo#136318 and odoo/odoo@3ffa1a0611 notably.
Task-36879 (Mail: Support Multi Domains Aliases)
Part-of: odoo/odoo#76734
Also impacts mass_mailing_sms, test_mail_sms,
test_mass_mailing
This PR adds support to receive sms delivery reports.
Before this PR, an SMS was considered 'sent' when successfully
handled by the third party. The user couldn't know if/when an
SMS was actually sent for delivery or delivered to the
recipient's device.
This was similar to the behavior for emails as delivery reports
are not commonly used (and not supported in Odoo).
With this work, the SMS `pending` state is introduced in mail,
mass_mailing, sms and mass_mailing_sms contexts although only fully
used in the latter two modules (+tests of course).
Because of the huge cost related to upgrading very large existing
databases, the following compromises were made:
1. An email and sms notification/trace SENT means DELIVERED.
Those that are sent but NOT DELIVERED are PENDING.
The difference between email and sms traces reinforced with this PR
is that an email sent will be counted as "sent" ~ "delivered"
unless an error is returned for emails while for SMS it can only be
reached if a delivery report is received.
2. The Link between an SMS uuid (shared with trusted parties) and
the tracking records (notifications or traces) is done via an
explicit relationship table (sms_tracker) instead of via a new field.
This however allowed to nicely concentrate the state update logic.
A `process` state is added to represent an intermediate
step in the sending process, such as held at IAP for SMS.
A few adjustments are also included to update for IAP api v3.
Also, adapts and includes new tests.
Task-2560666
Part-of: odoo/odoo#133392
This PR removes filter on ip so that each click should count as one.
previously multiple clicks from the same device only counted as one,
but now because of the ip removal, they are being counted as many
times as you go to the link.
task-3328661
closesodoo/odoo#132812
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
'alias_user_id' field allows to set a user when creating records through the
mail gateway. This is however quite wrong and eases spoofing. The alias owner
is not the creator of any record, nor responsible.
Current possible ways of being owner / responsible of records created through
the mailgateway
* when you send an email to an alias: if you are recognized you are already
set as creating user and logged message author;
* it is possible to use alias_defaults notably to set fields like 'user_id'
if you want to be notified / responsible of records created through this
specific alias;
Those usages are therefore sufficient, no need to have another way to spoof
users. Moreover it is hidden in technical view of aliases, no model allows
to configure it by default. Moreover since odoo/odoo@3edf181 no default
value is given to alias_user_id as it adds more (ACLs / creator) issues than
really helping setting up mail gateway flows.
Task-3453482
Prepares Task-36879 (Mail: Multi-Domain Aliases)
closesodoo/odoo#138213
Related: odoo/upgrade#5259
Related: odoo/enterprise#48692
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Overall cleaning of subscription and exclusion management code from portal.
This code comes mainly from v12 and can now benefit from cleaning and update.
SPECIFICATIONS
In this commit we rename ``mailing.contact.subscription`` model into the
shorter ``mailing.subscription``. This is sufficient to explain the model
purpose. As we plan to add an opt-out model this also allows to have sub
models with suffixes without being too long.
We also rename ``subscription_list_ids`` field on contact model to
``subscription_ids`` as this is shorter and clearer.
Finally the ``mailing_contact_list_rel`` table name that comes from old
implementations (simple m2m table) is renamed to ``mailing_subscription``
to match the model name.
Task-2669037 (Mass Mailing: Refactor js/portal for subscription)
Part-of: odoo/odoo#86084
Use name instead of email, as contact is notably used in sms marketing
application with mainly phone numbers. Better use the name as primary
ordering field. Then use ID to avoid non deterministic behavior.
This requires to fix some tests in 'test_mass_mailing' so that they
use test models instead of existing models. That way they are not
dependent on existing data and existing models definition anymore. An
issue with ordering rose as we modified the ordering of contact model.
Task-2150462 (Mass Mailing: Improve subscription management)
Part-of: odoo/odoo#86084
PURPOSE
Slightly modify various apps to improve the user experience. Changes include
notably labels and views fine tuning, roundings, and small css fixes.
SPECIFICATIONS
- Remove the decorator from the event list view as it's a bit confusing
- Ensure mailing KPIs are now shown with 2 decimal places
- Re-order the mailing stat buttons
- Improve the background / font colors of the cover block
- Improve the labels of:
- Title and confirm button of the /button and /link modals
- Confirm button when archiving a record
Task-3204554
closesodoo/odoo#128108
Related: odoo/enterprise#43937
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Update (some) query counters according to runbot state.
Also make some tests deterministic when involving company name.
Task-36879 (Mail: Support MultiCompany Aliases)
closesodoo/odoo#135288
Related: odoo/enterprise#47345
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Only duplicate emails used to be checked when sending a mass mail.
However it is possible (e.g. using templates) to send a mass mail
to the same person containing different information.
The existing functions to allow models to specify emails
processed in the past by some other means are kept.
A new check is added in the processing that checks the full contents
of the message, subject and attachment ids.
The strings are not hashed as most situations are:
- Sending the exact same mail to everyone
-> Only need to check against one message
-> Same complexity as hashing
- Sending all different emails
-> Checking inequality of str is usually very fast
For attachments, as we cannot compare them easily.
They are ignored for the purpose of equating emails
whenever there are the same number of attachments
in the email values as there are on the composer.
This is because each email should receive its own copy
of the composer attachments. If they have a different
number of attachments, they were generated dynamically
through reports and we assume they are all different.
We can thus remove the 'document based' information
as it is implicitly infered from this check.
-------------------------
Test utils are also updated for two purposes:
1. Add optional body and attachment_name discriminents
assertMailMail assumed all emails could at least be
differentiated by subject. Our test breaks that
assumption, so we use body and attachment to find the
best-fitting email based on the data passed in.
2. Check email_formatted on recipients
When using assertMailMailWEmails, we first find the
email using the non-formatted email of the recipient.
This does not match assertSentMail which checks against
the raw email_to value, which would often be formatted.
Task-2826811
closesodoo/odoo#99541
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Before the commit, the _get_seen_list() function in the mass_mailing module was
not able to correctly identify all the duplicate email addresses in a given mass
mailing. This was because the function chose and used only one way to find an
email address for each record in the mailing list, even though there are many
ways to find an email address for a record.
For example, a crm.lead record might have an email address in its partner_id
field, but it might also have an email address in its email_normalized field.
This can vary from record to record.
To fix this issue, the _get_seen_list() function was updated to only look at the
email address to which emails have already been sent, rather than trying to
fetch it from the record itself. This ensures that all duplicate emails are
correctly identified and that no duplicate emails are sent in the mass mailing.
Task-3234378
closesodoo/odoo#135081
X-original-commit: 66f9aa25af049aa3faf0f76475a4a2b63b5d0903
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
With input 'name email@domain.com' (missing chevrons allowing to clearly spot
the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
whole input is considered as being the email.
To improve the heuristic we can add a fallback by recalling 'getadresses'
on the input with spaces replaced by commas when it found only an email and
no name. The new email will be split into sub pairs allowing to find the real
email and various name parts, allowing to make a new name / email pair.
Emails should not contain spaces thus this is coherent with email formation.
This fallback actually comes from a specific code done in '_parse_partner_name'
of Partner model. Supporting it directly at tools level make the behavior
coherent for all models.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@18c71edf59
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
As of rfc5322 section 3.4.1 local-part is case-sensitive. However most main
providers do consider the local-part as case insensitive. With the introduction
of smtp-utf8 within odoo, this assumption is certain to fall short for
international emails. We now consider that
* if local part is ascii: normalize still 'lower' ;
* else: use as it, SMTP-UF8 is made for non-ascii local parts;
Concerning domain part of the address, as of v14 international domain (IDNA)
are handled fine. The domain is always lowercase, lowering it is fine as it
is probably an error. With the introduction of IDNA, there is an encoding
that allow non-ascii characters to be encoded to ascii ones, using 'idna.encode'.
Also remove usage of 'email_re' in mailing email check. It is too restrictive
compared to real formatting we support (or try to). Valid outgoing emails
were directly canceled, notably when containing unicode.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@3ce5fb3072
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS: MAIL COMPOSER IN MAILING
When using the composer with a mailing, it currently skips recipients whose
email is a multi-email due to the strict usage of 'email_normalize'.
We can improve multi-email support by effectively checking for the first
email found, using the "less strict" mode of normalize. It means more emails
are detected as valid, and therefore sent.
Due to lower support of multi-emails when sending emails, this even allows
to send multiple emails as all emails are mailed.
SPECIFICATIONS: DEFAULT RECIPIENTS
Mailings are generally done using default recipients, aka using a model method
that returns the people to mail: customers ('partner_id'), customer emails
('email_from'), specific implementation, ...
This is implementation using '_message_get_default_recipients' that returns
'partner_ids', 'email_to' and 'email_cc' that are then used in the mail
composer to generate final recipients.
In this commit we better handle the content of email fields to avoid issues
with multi-emails. For that purpose we correctly split the content of those
fields. We now have several 'email_to' for records having multi-emails instead
of a single badly-formatted 'email_to'.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@4a0d87d44f
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When having multi-emails input in an email field, 'email_normalized' field is
currently 'False', as they expect the field to contain a single email. This
has several drawbacks
* searching partners or fetching information based on emails does not work as
most tool methods use 'email_normalized' which is False (see e.g.
'_message_partner_info_from_emails', '_mail_find_partner_from_emails'
or 'find_or_create');
* blacklist is not available as it is based on 'email_normalized';
* mass_mailing wrongly considers those emails as invalid and cancel their
mail and related trace, as it tries to skip sending emails to invalid
emails;
Be more defensive and use first found email in case of multi-emails field.
Other emails are ignored. It is already an improvement that does not break
flows in stable and allow more emails to be sent.
before
-> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
-> email_normalized: False
after
-> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
-> email_normalized: raoul1@raoul.fr
A side effect is that it helps finding back some partners, as indicated in
tests where less phantom partners are created. It also helps suggested
partners / emails flow in discuss.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@90218186c5
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When building the final 'email_to' of outgoing emails using 'formataddr' we
have issues if email contains multi emails or formatted email. Main fix of
this commit is to extract emails and rebuild the 'email_to' list based on
found emails.
E.g. partner Raoul - email: "Raoul" <raoul@raoul.fr>
-> before: to: "Raoul" <"Raoul" <raoul@raoul.fr>> (double format)
-> after: to: "Raoul" <raoul@raoul.fr>
E.g. partner Raoul - email: raoul1@raoul.fr, raoul2@raoul.fr
-> before: to: "Raoul" <raoul1@raoul.fr, raoul2@raoul.fr>
single email with multiple emails, depends on server fault tolerance)
-> after: to: "Raoul" <raoul1@raoul.fr>, "Raoul" <raoul2@raoul.fr>
multi emails
Fix that computation by using all normalized emails found in 'email' fields
and rebuilding a formatted email based on name + those emails. We do not
use `email_formatted` as it is not really multi-enabled. We prefer a local
defensive approach to be as tolerant as possible with respect to user inputs.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1c4b704149
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
RATIONALE
Add tests related to not standard usage of email field. Two main use cases
are tested here
* formatted emails: `"Full Name" <email@domain.com>` stored into the 'email'
field;
* multi emails: `email1@domain.com, email2@domain.com` stored into a single
'email' field;
Additional tests: tests with unicode / ascii / case / wrong formatting are also
added to check the support in normalize and format methods.
IMPLICATION
Email field is generally managed as "containing a valid email". This means
it is sometimes used as it in 'formataddr' as well as to perform searches or
identification checks. Example of issue: partner 'Raoul' has a formatted email
like "Raoul" <raoul@raoul.fr>. Using 'formataddr' in email_from leads to
from: "Raoul" <"Raoul" <raoul@raoul.fr>>
-> which is incorrect (but often dynamically corrected by email servers);
Email field holding multi-emails are not normalized, as current normalize
is done only if the field holds a single email. It means
* no easy finding based on 'email_normalized', e.g. various tools like
'_mail_find_partner_from_emails' or 'find_or_create' do not find partners
based on this email;
* no exclusion list management;
* issue with formatting, like
to: "Raoul" <raoul@raoul.fr,raoul.other@raoul.fr>
-> which is incorrect (but often dynamically corrected by email servers);
USAGE: OUTGOING EMAILS
Those use cases currently generate faulty outgoing emails. This is valid for
recipients ('email_cc', 'email_to') as well as author ('email_from').
For formatted emails: `email_to` is formatted again based on name and email
which leads to sending emails to `"Full Name" <"Other"<email@domain.com>>`.
Note that multi emails without formatting may work as it leads to email_to
`"Full name" <email1@domain.com,email2@domain.com>`. Some outgoing email
servers correctly send multiple emails. It depends on their fault
tolerance.
USAGE: FIND BASED ON EMAIL (NORMALIZED)
When searching for partners (e.g. using '_mail_find_partner_from_emails' or
'find_or_create') normalized version of input is used.
In case of multi emails sanitize is 'False', as normalization expects a single
email in the field. Therefore no partner is found. In processes that do a
"search or create" (e.g. using a template on a record) this leads to creating
a new partner (or several partners in case of multi emails) each time.
USAGE: OTHER FLOWS
Other flows are build on top of '_mail_find_partner_from_emails' / 'create'
of outgoing emails and are impacted by formatted email / multi email usage.
Those include notably
* mass_mailing: '_message_get_default_recipients' should be defensive to
give correct values when creating mailing emails;
* mass_mailing: faulty emails is based on normalize and multi-emails are
considered as faulty and ignored;
* after post hook: '_message_post_after_hook' tries to link messages without
author (but email_from) with newly-created partners, when partners are
created from chatter. It is therefore impacted by those corner cases;
* marketing_automation: built on top of mass_mailing and suffers from the
same issues;
USAGE: UNICODE
Unicode in emails should be supported. 'formataddr' and IrMailServer notably
received fixes to support unicode. Some check performed on email addresses
fail when unicode is involved, which leads to some emails not being sent
while they could.
SPECIFICATIONS
Add tests related to those corner cases. Also add tests for computation of
`email_formatted` field of Partner model. It currently generates wrong email
values for the same corner cases (multi emails, formatted emails).
Tests are also added for the computation of `email_normalized` field used
notably for blacklists. It is not computed currently when being in multi
email mode which prevents from any blacklist mechanism as well as make
email finding harder. `_mail_find_partner_from_emails` tool method is also
tested with multi email as it uses the same heuristic as normalized email
field.
Tests are also added for mass mailing, when having to mail documents that
have a partner with formatted emails / multi-emails, or that have an email
field with formatted emails / multi-emails.
Also restore a test removed at odoo/odoo@afcb734908 while it should have been
updated to state that email addresses containing non-ascii characters are
supported.
Add some tests for tools methods used in various email processing flows.
Unicode tests are also added.
In future commits we will try to make email usage a bit more defensive to
try to lessen issues with that kind of use cases.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@fc8442f133
Part-of: odoo/odoo#134934
Prepares the move of ICP to mail before replacing them by dynamic alias
domains. Improve test coverage, notably for edge cases. Continue to make
tests more explicit after odoo/odoo#131492. Some tests are also merged to
lessen number of different tests when possible, notably when only a test
parameter differs (like giving an SMTP session or not).
Clean ICP and mail servers setup in test classes allowing to remove some
unnecessary extra initialization. Cleanup a mock in mail.
Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130750
RATIONALE
This prepares the move of ICP to mail before replacing them by dynamic alias
domains.
Cleanup tests: try to use loops with input / expected to better understand
the various test cases, add some comments, improve logs when failing to
find the right sent email. Rename tests to have a better test structure
when reading logs.
Remove a test from odoo/odoo@3b6c20805c that adds nothing except testing
the test suite.
OTHER ADDONS
In test_mail: have a specific class for testing servers as other data is
not necessary, and it allows to have a tag for it.
In mass mailing: concatenate test about server finding, several tests can be
done in a single unit test.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-3453347 (Mail: Move Mail ICP from Base to Mail)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
closesodoo/odoo#131492
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Rename alias domain and aliases used a test data. This allows to make
them easier to read, follow, grep and understand.
Activate multi-company on alias and gateway tests, ensuring it currently
has few impact on tests.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#130768
Always apply the blacklist in mass_mail composition mode regardless of the
recipient model implementing mail.thread.blacklist or not.
This solves the problem of mail sent to black listed address for model not
inheriting from mail.thread.blacklist.
Technical notes:
- it has been done in mail.compose.message _get_blacklist_record_ids ignoring
the mixin mail.thread.blacklist to avoid model change in stable.
- some tests have one added query because the blacklist is now queried for each
batch mail sends even if the model of the recipient doesn't implement
mail.thread.blacklist.
Task-2834862
closesodoo/odoo#118497
X-original-commit: 263e86114c60650f421354e165364afcd4122461
Signed-off-by: Dufays Pierre-Yves (pydu) <pydu@odoo.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
'TestSMSCommon' was useless and is replaced by the 'SMSCommon' class defined
directly in SMS addon, easing inheritance and imports. See community PR for
more details.
Task-3263512
Part-of: odoo/odoo#117606
With this PR, the `digest_data` template is changed, so many test cases fail.
This commit adapts the test cases by changing the `data-field` from `div` to
`table`.
task-2717426
Part-of: odoo/odoo#89549
Purpose
=======
The bounce emails aren't stored in Odoo, which can complicate the
debugging of the email sending.
Now, we store this bounce email, and we allow the user to read it from
the interface, so he can easily find the issue when an email sending
fail.
Specifications
==============
The bounce email is stored on the mail notification for standard emails
sending, and on the mailing traces when using mass mailing.
For some email providers (e.g. Yahoo), the "Final-Recipient" header is
not present. Normally, it allows us to retrieve the original recipient
of the email which bounced and then the partner. So if this header is
not there in a bounce email, we take the first recipient of the parent
<mail.message>.
Change the way that we parse the email body, for the bounce email.
For most email providers, the first mail body is the one that contains
the error and the next one contains the parent email body. So, the
current logic might ignore this body for Outlook and Yahoo.
Task-2116296
closesodoo/odoo#105923
Related: odoo/enterprise#34051
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Bug
===
The unlink of the <mail.mail> in the CRON is problematic because we
accumulate a lot of records, and the CRON timeout.
In particular, when we sent a mailing, we receive the "opened" event
(blank image in the email), and so we need to update the mailing trace.
But, if we unlink the mail at the same time, it locked the mailing trace
table and we couldn't write the new value.
The reason for that is that before, the unlink took more queries, but
it was done one record at a time, so we could commit the change and
release the lock between each unlink.
Task-3179157
See odoo/odoo/pull/73271
closesodoo/odoo#112703
X-original-commit: 57ae1b9b8b61f5f4719a8a81e9d0d21fab58cfda
Related: odoo/enterprise#37069
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Stéphane Debauche (std) <std@odoo.com>
Bug
===
When we remove some mail messages, we invalidate the cache of the
related documents. But in the same loop we call _invalidate_documents
which invalidate the cache, and so at the next iteration we will need
to make a new SQL query to know if the message is a "thread message".
So because of the prefetch ids, and because we invalidate in the loop,
if we unlink 1000 message, we will make 1000 SQL queries to fetch the
fields values of the 1000 messages.
Note that the unlink method will invalidate the entire cache anyway,
so unlike the write / create methods, we shouldn't need to invalidate
manually the related documents.
Task-3171093
closesodoo/odoo#112494
X-original-commit: f8958e9bbd4c7c37f614b20f7ed4d4825d438362
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Signed-off-by: Stéphane Debauche (std) <std@odoo.com>
Update query counters now that fields are editable computed fields. Some
counters are higher as the usage of composer in code is now closer to the
usage in view, with fields being correctly computed. Some other counters
are a bit lessened, but it globally stays the same.
Task-2088884 (Mail: Use editable computed stored fields in composer)
Part-of: odoo/odoo#107356
Update counters now that all changes in this PR are validated.
Main observations
* batch mode is improved: few tests effectively run on a real batch of
records but in those use cases there are more gains due to better batch
management of values rendering and recipients management;
* posting using a view does not call the composer anymore in all situations
allowing to gain a lot of queries by not creating a composer and calling
a dummy onchange on it;
* various small gains in various tests, notably linked to usage of low-level
reference fetch and various small code tweaks;
Task-2710804 (Mail: Clean MailThread API)
Part-of: odoo/odoo#99482
Update counters according to latest runbot counters. It allows to better spot
side effects of upcoming changes.
Task-2710804 (Mail: Clean MailThread Posting API)
Part-of: odoo/odoo#99482
Add possibility to add a message when adding or removing a blacklist entry
for both mail and phone_validation (used for SMS). This replaces the
``action_remove_with_reason`` method.
When updating active flag this is added as a tracking note to avoid having
several messages. Indeed message is concatenated with the tracking itself
instead of adding message for tracking + a message for the log itself. When
a new record is created, a note is logged.
Task-2710804 (Mail: Clean MailThread API)
Prepares Task-2150462 (Mass Mailing: Unsubscribe flow refactoring)
Part-of: odoo/odoo#106568
Update to current runbot state, in order to better spot changes potentially
introduced with this PR.
Task-2710804 (Mail: Clean MailThread Posting API)
closesodoo/odoo#106182
Related: odoo/enterprise#34197
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose is to try to have code easier to read and to update by having a
common way of sorting / writing code. Reorder some fields definitions and
dictionaries, update docstrings, ... in mail applications or when invoking
mail thread API.
Additional stuff worth noting here
* add some tagged in tests helping debugging / choosing tests to execute;
* in a test about mail generation with server action: correctly check
body_html field, not body which comes from the mail.message inheritance
(currently filled due to a side effect but actual field to check is the
html one);
* add same check on input for ``_render_template_qweb`` as done on other
rendering methods (even if rendering on [False] should be supported);
* move code translation update from specific tests into main test class
(code update due to new translations, followup of odoo/odoo@f8c2b02abe);
Some test linting done here
* reorder mail_render tests, remove some duplicated tests;
* reorder mail_template tests, remove some duplicated tests;
Task-2710804 (Mail: Clean MailThread Posting API)
closesodoo/odoo#106025
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Prevent archiving in-use mail servers by displaying an error message that
lists where it is still used, allowing to easily identify what need to be
updated before being able to archive the mail server.
Additionally,
- prevent the use of archived server as a fall-safe
- when duplicating a mailing with an archived mail server, replace mail
server by the default one
Detailed explanation:
1. A check has been added that raise an exception when trying to connect to the
smtp server or send an email when the server is archived.
With that solution,
- testing the connection of an archived server displays an error telling that
an archived server cannot be used.
- if a mail is still sent with an archived server, mail are in error :
"Connection failed (outgoing mail server problem)"
This fail-safe ensures that no mail will be sent through an archived mail server
and that the user will get some feedback about it.
The same fail-safe for the incoming mail server has been added.
Notes:
- the connection will outlive the archiving of a mail server still allowing
to send email through the archived server until the connection is closed. But
connection are not kept for long so this shouldn't be a problem.
- it cannot be tested because the connect method return immediately in test
mode.
2. When a mail server is archived, an user error is raised if it is in-use.
The implementation relies on each module to override the method
"_active_usages_compute" in "ir_mail_server" to complete the list with
user-friendly message describing the active elements that could send mail
through the mail server. This has been implemented for:
- l10n_it_edi: server used to send e-invoice
- mail: optional server configured for template
- mass_mailing:
-- default mail server
-- active server configured for mailing
Mail server are referenced in other elements but are not active anymore, it is
just for temporary or history purpose. Those references doesn’t prevent the
archiving of the mail server:
- mail_message
- wizard survey_invite and compose_message
- res_config_settings
Task-2821516
closesodoo/odoo#91240
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Have more reliable tests.
Better spot side effects coming from sub addons.
Lessen non deterministic counters due to local db.
SPECIFICATIONS
Make crm, event and mail performance tests post install.
Update query counters with
* local values (install module only with enterprise activated);
* community / enterprise runbots (if value is different);
* some notes on non deterministic issue if known;
Task-2925606
closesodoo/odoo#96446
Related: odoo/enterprise#29726
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
In this commit we introduce a new module ``test_mail_sms`` that holds the
tests for sms application, like ``test_mail`` does for mail. Currently all
sms tests are inside ``test_mail_full``. After this commit code is moved
from ``test_mail_full`` into that module. Full testing module should be used
mainly to test integrations with a lot of submodules and for performance
tests, not testing details of SMS implementation.
We also add ``mass_mailing_sms`` as dependency of ``test_mass_mailing`` so
that both mailing types are tested in the same module. It eases maintenance
and writing of tests. Mass mailing SMS tests from `test_mail_full`` are
therefore moved in ``test_mass_mailing``.
This commit also allows some cleaning in classes used in tests, to have
classes in ``test_mail_sms`` and ``test_mail_full`` that contain everything
necessary to test mail features.
Task-2890111 (Test Mail/Mass Mailing: SMS tests reorganization and move)
Part-of: odoo/odoo#96223
Add a model and a test allowing to test the seen list using raw SQL based
on partner_id field. Test indicates a not stored partner_id field currently
crashes beyond redemption.
Task-2852943
X-original-commit: a15bda3bed339a0ac422a3c82ec9cc295a11a785
Part-of: odoo/odoo#94532
Purpose
=======
Improve the performance of mass mailing by removing the <mail.mail>, in
batch, all at once. The unlink operation is very expensive for the ORM,
in terms of SQL queries.
A new field `to_delete` is required to know which <mail.mail> we need
to delete. The reason is that the `failure_type` is stored on the
<mail.notification>, and it's possible to have <mail.mail> without
<mail.notification>.
Task-2587345
Part-of: odoo/odoo#73271
Only the group_salesman can see the opportunities statbuttons
Fix a few not-smart mass_mailing tests:
- running the mass mailing queue processes the messages as the user
running the queue, and will try to send the demo messages, depending
on demo data (when re-running the tests) the test user may not have
the accesses required; sending just the one test email avoids that
issue
- the leads kpi assumes the user has access to leads, that ain't
necessarily the case
For the second issue, when sending statistics if the mailing has a
user `_prepare_statistics_email_values` should be called with that
user as "current" (cf `_action_send_statistics`), so we can just work
off of the current env, though it might be a good idea to eventually
check that.
closesodoo/odoo#89774
X-original-commit: 39af865800c6752b60171f16d6056ceb3c5a1636
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When using a template with inline placeholders in the body, the Test
button will replace the placeholders, but they will remain in the final
body when the mass mailing is sent for real.
This commit fixes this issue by using the correct rendering engine for
the Test button too.
OPW-2819032
OPW-2828461
closesodoo/odoo#89346
X-original-commit: b0dcd2fa35d949dd722f1bc70401ceeb7f5d8231
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Paul Morelle <pmo@odoo.com>
PURPOSE
Improve tooling of mail tests. Lessen custom code, try to use same tools
in all tests.
SPECIFICATIONS
Use MailCommon as a base class for performance test. It now uses the same
boilerplate as other tests, leading to easier to reproduce and understand
testing setup.
This notably now adds outgoing mail servers which means adding one query
to some tests involving emails.
Task-2673913 (TestMail: Cleanup and improve test coverage)
Part-of: odoo/odoo#86393
mail_* = mail, test_mail, test_mail_full, test_mass_mailing
PURPOSE
Improve tooling of mail tests. Use SMTP mockup class available in base tests
in mail tests to have a better and simplified mock tool when dealing with
mail tests. Remove old fashioned code.
SPECIFICATIONS
Include ``MockSmtplibCase`` directly into ``MockEmail`` so that we re-use
existing components for mocking outgoing emails. Also make parameters for
gateway coherent between those two classes (use same test values).
Remove 'sim_error' weird parameter of Mail ``mock_mail_gateway`` and use
standard mock 'side_effect' instead in some specific tests, notably on
SMTP.connect() mock.
Add some assert / tools method and improve existing tools, used for creating
or improving test data. This will be used in future commits when adding new
tests.
Add outgoing mail servers data in mail tests, allowing to make tests closer
to real life use cases.
Remove unnecessary calls to ``_init_mail_gateway`` as it is automatically
done in ``MailCommon.setUpClass()``.
Try to consolidate starting test data, notably with countries, phone numbers
or email in mind to be sure test mail are deterministic.
Task-2673913 (TestMail: Cleanup and improve test coverage)
Part-of: odoo/odoo#86393
Followup to #82105: turns out we kind-of forgot that records could be
updated with new attachment and the exact same issue could occur.
So with the same reasoning as the previous PR, re-attach attachments
to the current object when updating it.
closesodoo/odoo#83083
X-original-commit: 398070ec1b6ed7d6a8e7c0ab75d45b8a9ad0e0d0
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When the system broadcasts an email response to document followers,
if the config parameters `mail.force.smtp.from` or
`mail.dynamic.smtp.from` are defined, it will rewrite the `From`
address to avoid spoofing the sender's domain.
**NOTE**: As of 15.0, this is based on the `from_filter` setting on the
corresponding ir.mail_server, rather than the abovementioned config
parameters, but the rest of the discussion stands.
For example, if the `mail.catchall.domain` is set to `example.com` and
an email response comes from:
"John D" <john@doe.com>
it will rewrite it to:
"John D (john@doe.com)" <notifications@example.com>
This will make sure the system never sends outgoing email for an external
domain, as it has no authority for doing so, and that could
break mail filtering/authentication rules (SPF, DMARC, etc.)
During this "encapsulation rewrite step", both the original Sender name
and their email are preserved, and put into the quoted "name" field of
the rewritten address. It seems sensible to preserve as much information
as possible about the original sender.
Unfortunately, the inclusion of the Sender email in the final name makes
it appear to some inbox providers as if the message is trying to
deceptively impersonate another person (as many phishing schemes would).
As of November 2021 GMail at least does this, and will hide the name in
the UI when it happens. It will keep only the rewritten email, which is not
very useful in the case of a notification (even though it's more
technically correct, of course).
This patch removes the original email from the rewritten notification,
keeping only the name, considering that the email is not the most
important part, and it's better to have one of the two than none.
So after the patch, the rewritten address is now:
"John D" <notifications@example.com>
When there is no name in the original address, we keep only the local
part of the email, to avoid the same display issue. The recipient will
have to identify the sender based on the context / past messages.
closesodoo/odoo#81807
X-original-commit: 3c65ec5a8191a392980ceb0a8c584767eae405f1
Signed-off-by: Olivier Dony <odo@odoo.com>
When mailings have no responsible, KPIs emails are broken as there is no
from and to on the mail. This leads to mails being invalid and set in
exception.
Moreover currently author and from / to of those D+1 emails are not coherent
as author is current user (generally odoobot as this is generated through
a cron) while from and to are based on mailing responsible.
In this commit we make statistics emails more coherent
* when a responsible is set on the mailing: author, from and to are linked
to the responsible;
* when there is no responsible, current user is set as it may be sent by
regular people;
* reply-to is set to company email as it is often the case with 'marketing'
emails;
Task-2686586 (Repair mailing statistics email)
X-original-commit: 95a21ca16f560cf4341c547f6bc909d3261acd1b
Part-of: odoo/odoo#79877
When there is an issue updating mailing state (concurrent access, or some other
error that may happen), mailing statistics emails may be sent in loop. As we
do not think timing is so important, we now use the email queue to send
statistics emails.
Task-2686586 (Repair mailing statistics)
X-original-commit: f78ac45c2eafc3cf6b213e1f1608e76e243bd8d2
Part-of: odoo/odoo#79877
Recently some "de-t-rawify" [1] was done through Odoo. A change in digest
layout broke mailing statistics. Indeed mailing statistics are build on digest
main layout to re-use it. In mailing case a ``kpi_name`` value was missing when
rendering statistics columns, leading to a crash. This is now fixed in both
mailing and sms marketing.
In this commit we also add tests for statistics emails (D+1 KPIs) rendering
for both mail and SMS marketing mailings. Email details are also tested
in order to better highlight future changes and tweaks in email sending
linked to D+1 KPIs emails.
Task-2641394 (Digest emails sending improvement)
Task-2582128 (Digest onboarding and usage improvement)
Task-2686586 (Repair mailing statistics email)
[1] odoo/odoo@c56e8c9f5a
X-original-commit: c9905ceb3172d8e80e3aff4ae9bc925dcb6b6519
Part-of: odoo/odoo#79877
Purpose of query counter tests is to try to match real life use cases. In
mail those generally involve a correctly configured mail gateway. This is
why we set those parameters in performance tests, leading to a small increase
in some counters.
Task-2661036 (Performance tests data cleanup)
Prepares Task-36879 (MultiCompany Aliases)
Part-of: odoo/odoo#77845
Thibault Delavallée