The query was computing the count for all mailing lists, not just the
one being requested. This prevents optimizing the SELECT with indexes.
In addition, using a subselect for the blacklist forced a full table
scan on the contacts - switching to a LEFT JOIN avoids that.
With a blacklist of a few hundred thousand entries, and a table of
mailing list contacts in the same order of magnitude, this brings the
SELECT from 3+ seconds down to 3 milliseconds - with the same result.
Bonus: no need to perform the regexp extraction of emails to test for
NULL values -> substring(x) is NULL iff x is NULL.
If a recipient has a formatted email address like 'A <a@b.com>', the returned
opt_out list must contain normalized email ('a@b.com') in order to be comparable
during the get_mail_values of the mail.compose.message.
Otherwize, It won't match.
Task ID : 1909314
Closes PR #28667
Since the opt out per mailing list, the intermediate model was used
to list all the recipient of a mailing list. The side effect was that
importing new contacts into mailing list was not possible anymore.
Also, creation of a new contact into a mailing list was made indirectly
by creating a new many2many record between contaxct and list, which was
not really user friendly.
This commit restores the mailing contact model as the view of the mailing
list recipients. In other terms, the contact import is restored.
An opt-out computed field as been added to be able to filter on opted out
contact inside the mailing list recipients view.
This opt_out field SHOULD NOT be used in a form view as it need the active
mailing list id in the context in order to be computable and searchable.
This field has no sense without this active mailing list id context.
Task ID 1891228
Linked with blacklist and opt out per mailing task (ID 33224)
Closes PR #27632
Don't forward context from mass mailing to partners view.
When grouping mass mailing by state, passing context makes
Odoo trying to group partners by state, and then get a traceback.
closesodoo/odoo#27775
Fix to commit: a1d6064dcc
Which erroneously put a mass_mailing method in mail,
as mail.mail.statistics is defined in mass_mailing.
Since it does not depend on the records, it been extracted from the
loop.
opw 1890461
closesodoo/odoo#27431
Purpose is to fix the comparison between mass mailing record email field
and blacklist entries by sanitizing email to ensure comparison is done
on same basis as what is stored in blacklist table. Also fix the action
merge where the extraction of email is needed before comparing to the blacklist.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
This commit adapts the business code to changes introduced by
the parent commit in order to keep the same behaviour as before.
All readonly=False fields will have to be checked afterwards to confirm
that the business case requires write access to the source field.
In A/B testing mode, a given recipient will only ever receive a single
email from a given mass-mailing campaign, no matter how many mailings
were sent to them.
This is useful for A/B-testing various mailings in order to check their
results. But is very difficult to understand for users who simply want
to organize their mailings into campaigns.
Additionally, the A/B testing option is hidden as a technical feature, so
it is hard to discover in order to troubleshoot unexpected number of
emails sent by a given mailing.
It will be safer to keep it off by default.
(keeping the parameter explicit, to better show that it is on purpose)
This partially reverts dee5c31264
Purpose of this commit is to improve user experience in mass mailing by
redirecting users on filtered views when clicking on sent / delivered /
read / opened / clicked stat buttons. Currently it redirects to the
statistics view which is not the most useful one.
This commit is linked to task ID 1885121 and closes PR #27003.
- When checking if a recipient has already received a mass mailing, the
record model for the recipient was not checked.
Which leads to false positives.
Purpose of this commit is to improve settings by improving labels and
section titles. Purpose is to ease use of settings by users and make
it easier to understand.
This commit is linked to task ID 47221 and PR #26483.
The default value for failure_type was the 'NONE', instead of being
simply False/NULL. This represents a significant waste of tablespace
with no added value.
Switching to true NULL value for representing the absence of error
also save some precious time during database upgrades on databases with
large numbers of emails.
While fixing the values for failure_type, also fixed a few typos and
untranslated strings.
See original feature at 133eeb1bbf
A mass mailing can be considered both as bounced and opened. This can
occur with some email providers, such as Gmail. Indeed, Gmail sends
out-of-office replies to the `Return-Path` address, which is also used
for bounced emails. Therefore, it is not possible to determine
accurately if a message sent to to the `Return-Path` is a genuine
bounced email, or an out-of-office reply.
However, if the recipient opens the email, it is possible to determine
that the bounce was counted incorrectly. In this case, we empty the
`bounced` field. Similarly, we only write on the `bounced` filed if the
`opened` field is empty. This could occur if the recipient opens the
email before the out-of-office reply is received.
As a bonus, we also make sure to avoid updating the status if no route
was found. No route means that the email was bounced, so no reason to
set it as opened.
opw-1869957
In order to avoid sending mail indefinitely to a wrong email address,
Take the mail statistics for the recipient of the last 3 month :
if more than 5 mails bounced (with interval of more than 1 week)
the email is blacklisted.
Purpose
=======
- Apply the blacklist implementation to Improve mailing subscription to be more
compliant with the European GDPR law. Keep a list of people who does not want
to receive promotional emails (or mass mailing in general) anymore.
- Allows the recipient to update himself his mailing preferences
Specifications
===========
This commit is regrouping some main changes on mass mailing.
Apply Blacklist for following models through blacklist.mixin :
- crm.lead
- res.partner
- mail.mass_mailing.contact
- mail.channel.partner
- Opt_out per mailing list instead of per mailing contact.
- Replace opt_out by blacklist in crm.lead + res.partner models
- Added 'ignored' state for mass_mailing. Ignored = blacklisted, opted-out
Ignored email are not included into final statistics to avoid confusion.
- Unsubscribe(d) pages migrated from website_mass_mailing to mass_mailing module
as thoses pages should work without having the website module installed
Detailed implementation
===================
Mass-mailing :
- A blacklisted email is notified by the ban icon next to the email field.
(at the left of the email field for display purpose)
- Renaming the '_get_blacklist' method that was actually
searching opt_out list into 'get_opt_out_list'
- Opt out per mailing list : Add opt_out + related fields (for display and
ergonomy reasons) on the relational model
- Display relational model tree view instead of mass_mailing.contact tree
view when clicking on mass_mailing_list in kanban view
-> In order to be align between contact_nbr displayed in kanban tile and
the content of the tree view
-> mass_mailing contact is accesible via the user icon in this tree view
- Remove custom filters 'filter_contact_subscription' and
'filter_contact_unsubscription' as opt_out is not on mass_mailing.contact
model anymore
- Add 'is_public' to mailing list
The name of this mailing list can be seen (or not) by recipient in
the unsubscription page
If the mailing lists used in the mass mailing are not public,
the user in only informed that he has been unsubscribed.
If the mailing lists are public, the user is informed that he has been
unsubscribed from the mailing lists
and he has the choice to modify his subscription to all the public
mailing list he is or was subscribed to.
- Unsubscription Page :
- mass_mailing_contact :
Opt_out per mailing list, done by email and not by id, as multiple
contact can have the same email
The recipient can add/remove himself to/from the blacklist
The recipient can send a feedback about why he unsubscribed
- crm.lead + res.partner :
Once the recipient unsubscribe, he is automatically blacklisted
The recipient can 'Come back' and remove himself from the blacklist
if he changes his mind
- Show blacklist button parameter added in config :
The idea is to enable/disable the fact that the recipient can add
himself to the blacklist by showing or not the 'blacklist me' button
Only applies for mass_mailing.contact. The recipient, once
blacklisted, can always, no matter the value of this parameter,
'come back' and unblacklist himself
crm.lead + res.partner :
- Replace opt_out by blacklist in crm.lead + res.partner models.
As when a res.partner or crm.lead unsubcribe, we assume that the recipient
does not want to receive mass mailing anymore, at all, even if we adds him
to a mailing list afterwards. He is then blacklisted to avoid this.
With this behaviour, if a new lead is created with the same email address,
he won't be able to receive mail in mass_mode.
But he will still be able to receive '1 to 1' direct email.
Task ID 33224
Closes#25966
Purpose
=======
Improve mailing subscription to be more compliant with the European GDPR law.
Keep a list of people who does not want to receive promotional emails
(or mass mailing in general) anymore.
Specifications
===========
This commit is regrouping some main changes on mass mailing.
- Added blacklist : Avoid sending mass mailing to blacklisted recipient
(blacklisted = email address that doens't want to receive mass mailing anymore)
Detailed implementation
===================
Mail :
- Add Blacklist mechanism in mail module (NOT in mass-mailing) :
as we can send mass-mail without the mass-mailing module
- Unicity in email -> To avoid error in import, override the create and return
the existing record if any, else, create the record normally.
- Blacklisting is done by email address and is cross model.
Will apply to model that inherit the blacklist.mixin.
- field 'is_blacklisted' -> computed : check if email is in blacklist
+ search method to be able to filter on is_blacklisted
- When a email address is blacklisted, it will never get mass mailings anymore.
Even if the email address is added to another mailing lists
- Avoid sending notification to blacklisted recipients when sending email
in mass mail mode. If the recipient is blacklisted, we should not even send a
notification in the recipient's chatter for an email that he won't even
receive.
- When a email address is blacklisted, it can still get 'normal' mailings.
- The blacklist shoud be accessible in
Mass Mailing / Configuration / Blacklist
and Settings / Technical / Email / Blacklist
-> Renaming Settings / Technical / White / Black List config menu item
into Channel Moderation to avoid confusion with Mass Mail Blacklist
- Add indexes to the blacklist table (on email) to make it fast for access for
the different use cases
- _primary_email : attribute that must be overriden to specify which field must
be used as email in the blacklist mechanism.
- Filtering the blacklisted recipient in mail composer :
done in mail._get mail value()
In case of real mass mailing, we need the statistics to be computed in order
to know how many recipients were ignored in the mail.
So we cannot avoid sending mail but instead flag the mail as canceled.
Task ID 33224
These two columns are used to compute the mailing statistics, to be
displayed in the kanban dashboards. Adding them makes the dashboards
a lot faster on large databases (e.g. a few milliseconds instead of
several seconds)
The kanban view displays mailing statistics for all mailings that are
sending or done. To compute these statistics, it uses the `total` as the
expected number of emails to send. This `total` used to be computed as
the number of records matching the mailing filter.
This is both slow (filters are seldom written in an optimized way), and
incorrect because the recipients were frozen when the email was sent.
The number of matching records may have changed since then.
To fix both problems, introduce a new `expected` computed field that
simply returns the number of emails that were put in queue at the time
of sending, and thus represent the real number of expected emails.
This field is computed along with the rest of the statistics.
The framework supports a special `group_expand` attribute on fields,
to provide a helper method that expands the list of columns in a
read_group (typically meant to display empty columns in kanban views)
See rev. 580e78ad37
More efficient, and less code.
When a mail is in failure state, a red envelope appears next to the message
in a thread but it was difficult to send the mail again.
This tasks will allow users to send mail again easily, or mark notification
as cancelled if the user want to ignore this failure.
A notification will appear in sender systray while mail are in failure.
Task: #46158
PR: #24628
If the `web.base.url` contains a trailing `/`, the replacement of the
`/unsubscribe_from_list` link won't work since the string to replace
will be `my_url//unsubscribe_from_list` instead of
`my_url/unsubscribe_from_list`.
Fixes#24731
opw-1848572
Olivier Dony