When a mail is in failure state, a red envelope appears next to the message
in a thread but it was difficult to send the mail again.
This tasks will allow users to send mail again easily, or mark notification
as cancelled if the user want to ignore this failure.
A notification will appear in sender systray while mail are in failure.
Task: #46158
PR: #24628
Purpose is to lessen query count by avoiding to browse and/or fetch data when
not necessary. This commit does two main things
* first one is to avoid checking for existing statistics in mass_mailing
where only checking existing of a mass_mailing on a mail to send is
sufficient. It allows to avoid fetching data from statistics table;
* second one is to rewrite part of the post_process code called after sending
emails. Using list comprehension and mailing_id existence on mail_mail
allows to save several queries on standard post flows;
Most tests benefit from this code update. On overall community runbot about
3K queries are saved with those small optimizations.
Send methods of mail.mail model are set private and simplified. Those
methods have been written when a lot of overrides were necessary to
generate notification emails. Since then the flow has changed but
methods have not been updated completely. This commit therefore simplified
a bit the call chain and parameters.
This commit is purely technical and does not hold any functional change.
Purpose is to clean code in order to prepare future changes in mail
application.
Commit 0d3d0e9563 added a key in context when sending emails to know
the target model name. It was used at that time to forge for footer of
notification emails.
Since that time notification emails have evolved a lot. The code based on
that context key has been removed. Context key has been transformed into a
context switch. However it is not used anymore and can be removed. It allows
to clean a bit the code and save a context switch.
This commit is purely technical and does not hold any functional change.
Purpose is to clean code in order to prepare future changes in mail
application.
This commit allows to gain some queries in mail sending process, meaning
removing context switch is interesting. One query is gained when using
the send process. Complex performance test involving several message post
gain 3 queries each.
Purpose
* move notify in message_post instead of always doing it at message
creation
Specifications
* message create should only create a message
* message post should be in charge of calling the notification process
Now that we're closer to switching to P3 for good, these helpers have
outlived their usefulness, and mostly add noise.
All remaining dict.iter*() or dict.view*() must be converted to the
normal keys(), values() or items() calls.
Whenever the result is likely to be used for more than the scope of a
loop, or when the dict needs to be modified during iteration, the calls
must be wrapped in a ``list()``, to protect the new P3 semantics.
Those cases are very exceptional.
Also removed some dead code or improved the API to remove unnecessary
conversions.
No formal decision there, and the stdlib API seems like a PITA as it's
based around native strings, not bytes and not text.
* try to use cross-model API instead of random encode/decode
* unicode all the mail_template things
* formataddr *requires* a text tuple now
In P2, formataddr((False, False)) "worked" (generated "False
<False>"), in P3 it will fail and blow up.
* attachments are quite definitely bytes
* remove try_coerce_ascii because it's garbage
* cross-version metaclass spec
* more formally deprecate browse_record and browse_null since they
were using metaclasses anyway
* update docstrings referencing the latter
In Python 3:
* various builtins and dict methods were changed to return
view/iterable objects rather than lists
* and the separate Python 2 view/iterable builtins and methods were
removed altogether
This is problematic when using these items as list (which the happens
repeatedly in Odoo), but more viciously when iterating *multiple times*
over them (which also happens, which I've messed up multiple times while
writing this, and which is a pain to debug even when you've just created
the issue).
Convert all code using these to semantics-matching cross-version
helper functions to get the LCD behaviour between P2 and P3, and
forbid the builtins via lint.
issue #8530
With this commit, the mimetype will be sent with the attachment.
That will allow some Mail Client (e.g. thunderbird) to 'understand' the
purpose of the attachemnt.
E.g. when you sent a text/calendar mimetype, thunderbird propose you to add
automatically the event into your calendar.
The mass-mailing App suffered from severe issues due to its inability
to detect and handle duplicates "at the email level", and the absence
of any global blacklist system, leading to lack of user trust.
Mailing-lists typically include multiple records with the same email,
and it is critical to avoid sending them the same email several times.
A related problem is the unsubscription of an email that is present
in other records (duplicates). Opting out the first email should
automatically blacklist it for other records as well.
Ideally we should have a global blacklist table in order to
share the unsubscription requests globally across models (Leads,
Partners, Mailing-list contacts). It would also allow importing
it from other blacklist systems. (TODO for master)
This commit introduces a partial solution, made of several
small changes:
- In mail.mail: double-check that an outgoing email has the
correct status (`outgoing`) before sending it. This allows
adding emails in the queue and cancelling them before they
actually get sent.
- In crm.lead: force predictable recipients for mass-mailing,
by always using the email of the lead rather than the email
of the linked partner when there is one. This simplifies the
computation of the blacklist and seen list. Other areas in
the codebase already assume as much.
- In mass.mailing:
+ Before sending out a mailing-list batch, compute the
blacklist (all opted-out emails) and the seen_list
(emails who previously received this mail) to make
sure we only ever target valid recipients.
+ While delivering the mass-mailing, any email targeting
an address that is in the blacklist or "seen list" is
canceled before being sent. The corresponding statistics
entry is considered "not delivered".
Also updates the "seen list" continuously.
+ When a mass-mail belongs to a campaign with the
"unique AB/B testing" flag, the "seen list" is
common to the whole campaign, as an extra safety.
+ Auto-delete mass-mailing test messages sent with the
test wizard, to avoid polluting the mail_mail table
Note: this fix uses a simple regex for efficiently extracting
the blacklist in pure SQL from different models, and doing so,
assumes that each record only holds a single email
(no comma-separated adresses). This should be sufficient for
most cases. The regex:
([^ ,;<@]+@[^> ,;]+)
Commit 00bd7274c4 introduced speed improvements when
sending emails through batching and reusing an SMTP connection for a
given batch.
However exception management was slightly changed during that patch.
Previously to this patch all errors in the sending process were considered
as MailDeliveryException, except for MemoryError and psycopg errors.
However now mail_mail.send() may raise another kind of exception in the
SMTP connect process. Moreover a raise_exception parameter allows to
choose to either raise an exception, or just set emails in exception
state without raising an error. However since the patch it is not taken
into account correctly.
This leads to several issues, notably
* auth_signup waits for a MailDeliveryException when sending a password
request email. A badly configured SMTP server leads to a connection
error that is not caught. This prevents creating users, which could
be annoying.
* chatter does not handle exceptions as notification emails are sent
using raise_exception being False. Having an issue in the connect
process crashes the chatter.
This commit fixes the exception management so that it behaves like
before the patch.
Make sure the _send() method propagates the provided smtp_session
to benefit from the speedup obtained by avoiding multiple SMTP
sessions.
Cfr 00bd7274c4
Work around performance issue when sending large mass-mailing:
1. Processing a large email queue would crash with MemoryError.
-> The queue agent will now browsing the emails 1 by 1 during
delivery to avoid filling the memory.
-> The agent will not select more than 10k emails per delivery.
The delivery rate can be increased by waking it more often.
2. Reopening a connection to the SMTP server every time is slow.
-> ir.mail_server.send_email() now accepts an extra `smtp_session`
optional parameter. When passed, that session will be used and will
not be disconnected after the delivery. The queue processing agent
prepares batches of emails to be sent to the same outgoing mail
server, and uses a persistent smtp_session to avoid the multiple
reconnection.
To account for MTA that would refuse to process a large number
of emails in the same session (that should be rare), a system
parameter can be used to set the batch size (default: 1000).
3. As a bonus, the ir.mail_server.connect() method is now
responsible for resolving the actual mail server parameters,
rather than having half of the logic in the send_email() method.
Warning: this commit drops support for "maildir:/" format in
outgoing mail servers hostnames. This feature is deprecated,
unused, untested, and was meant for debugging only.
Remove unrestricted "read" access. To make code internally using `ir.model`
work, add a private method `_get` on `ir.model` to retrieve the record
corresponding to a model name, without access rights issue.
Change signature of method `get_authorized_fields` to make it use a model name
instead of a model id. This removes the necessity of a search on `ir.model`.
When creating a mail.mail without giving a mail_message_id, a new mail.message
is created. It does then through _notify that notifies people of the message
creation. This leads to mail being sent two times, one when sending the
mail.mail, one when sending the notification.
This commits fixes that behavior. When creating directly a mail.mail without
giving a mail.message, call to _notify is bypassed. This way either an email
is created, either a message that has to be notified is created, but not both.
Currently it is impossible in Discuss to know whether an email has been sent
to a customer and whether it failed or bounced. A notified partner has an
entry in the needaction m2m table. In this commit we decorate this table to
add fields about the email notification: is an email sent, did it failed, did
it bounce.
This information is kept only for customers. Internal users does not use this
information. Moreover their notification is deleted once the message is read
in the Chatter. This avoids having a notification table that grows quickly.
Chatter now holds a new icon for email details. It allows to know on a thread
status of emails sent to customers.
Previously Return-Path were composed using bounce_alias-<mail_id>-<model>-
<thread_id> . However this may lead to return adresses not supported by
the incoming mail server. We now form the bounce alias using bounce_alis+
<mail_id>-<model>-<thread_id>. Using a + indicates that the right part of
the address is for information and used to manage the bounce. Only the
left part is used for the email routing, allowing the return email to
effectively land in the alias mailbox.
Rev. 56cec604e6 modifies
the auto-commit behavior of the mail queue scheduler,
to allow using it within tests.
The condition was wrong and always True as soon as
the registry was loaded, permanently disabling
auto-commit, both during tests and outside tests.
This broke the duplicate emails protection.
message_get_email_values is declared as receiving cr, uid, id. However this method
is an override of a multi method. A decorator has been added to tell that id is
actually a list of ids and avoid list encapsulation issues with the result. This
way the method parameters are not changed.
[CLEAN] mail: removed some unused imports, by the way
Notification emails have been redesigned. They notably include buttons
allowing to perform some action directly from the email.
The notification creation and sending has been partially rewritten
and improved. The purpose is to lessen the number of rendering to perform
when sending emails to recipients. Recipients are first categorized into
groups. Basic groups are partners and users. The notification template
is then rendered twice, one for followers and one for not-followers. In most
cases there will be few rendering to perform. Through inheritance it
is possible to further categorize users. For example HR users / officers
that have approve / refuse buttons in their email.
A custom data structure is used to store data about buttons and actions.
URLs, follow / unfollow are added in the structure and used in the
template to render the email for a given group.
New routes are added in mail. Those allow to perform some action, like
going to a form in create mode, following / unfollowing, executing a method,
sending a signal for a workflow. Those routes are for users only and rely
on classic access rights.
A generic route for viewing records is added. It replaces the old redirect
action. According to some specific action given by the already-existing
get_access_action, the record will be visible for everybody (forum, blog)
or restricted (going on the Inbox / login / form view, according to access
rights).
The next commit will add the various inherits necessary to add the actions
in the main addons.
The fields were a bit messy. They have been reorganized so that field declarations
are grouped into content / from / to / gateway.
Nothing changed, only organization and some tooltips.
XavierDo