In this commit we remove overrides of create / write on subscription model to
make 'unsubscription_date' an editable computed field instead.
While being there, tracking on blacklist model is ordered, as a side dish
to prepare other mailing improvements.
Prepares Task-2150462 (Mass Mailing: Improve subscription management)
closesodoo/odoo#122106
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Enforce strict types for returned values for
* create
* write
* unlink
* default_get
to make those methods more consistent and reliable.
Also make sure they can be called with empty self/values,
i.e. that they follow the same behavior as the base methods
defined in the main orm Model.
closesodoo/odoo#116809
Related: odoo/enterprise#38880
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
The parameter in search() is redundant with method search_count(), and
was making the calls less readable.
The method _search() is aimed at always returning a Query object. The
method can therefore never return an integer, hence the removal of the
parameter. This does not actually remove any functionality from the
method; counting result is simply given by using it differently.
Part-of: odoo/odoo#112126
RATIONALE
Purpose of this commit is to be explicit in subtype chosen when invoking the
message composer / calling message_post. As default value may not always be
clear, better be explicit in case the composer default value changes.
SPECIFICATIONS
Add explicit references to subtype when it is not obvious what will be the
final subtype, notably when using helpers (post_with_view or template which
uses the composer that is not crystal clear in its subtype management).
In this commit we also add support of XMLID-based subtype when invoking the
composer. A ``default_subtype_xmlid`` context key is transformed into a
``default_subtype_id``, to be used notably in JS where we cannot easily
use a ``ref``-like statement. Post API now also supports 'subytpe_xmlid'
argument allowing to give the xml id and ease calling the methods.
Use ``_xmlid_to_res_id`` to get directly the ID of subtypes in order to
avoid useless queries from ``ref`` that does an exists.
Also remove useless values given to post API, notably author_id that is by
default the current users' partner.
Task-2710804 (Mail: Clean MailThread Posting API)
Part-of: odoo/odoo#99482
Add possibility to add a message when adding or removing a blacklist entry
for both mail and phone_validation (used for SMS). This replaces the
``action_remove_with_reason`` method.
When updating active flag this is added as a tracking note to avoid having
several messages. Indeed message is concatenated with the tracking itself
instead of adding message for tracking + a message for the log itself. When
a new record is created, a note is logged.
Task-2710804 (Mail: Clean MailThread API)
Prepares Task-2150462 (Mass Mailing: Unsubscribe flow refactoring)
Part-of: odoo/odoo#106568
The possible index names have been renamed "btree", "btree_not_null"
(instead of "not null") and "trigram" (instead of "gin").
Task 2742526
Part-of: odoo/odoo#83274
Three supported types:
- btree (default for index=True)
- btree not null (when >90% of the data are null)
- gin trigram search (for char fields)
Review of indexes on all objects.
closesodoo/odoo#83015
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
This commit fixes all issues detected by the new pylint
gettext-variable test.
It converts some calls to the new syntax
_("Foo %s", bar)
to progressively migrate the code to the new syntax.
A few calls were not technically incorrect but still detected by the
linter.
_("Foo" +
"Bar")
has been converted to
_("Foo"
"Bar")
as it has the same effect and make sure the argument is of type
asteroid.Const instead of BinOp).
closesodoo/odoo#53683
Related: odoo/enterprise#11467
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Using a few regex like
\((_\(.*%s.*)(\) % )([\w\[\]][\w .\[\]\(\)'"]*)\)
($1, $3))
Old syntax is still compatible but starts the migration to the new
syntax that catches error.
Purpose of this commit is to call toggle_active or archive / unarchive methods
instead of manually writing on active field. Indeed this allows to trigger
business code related to archive / unarchive which is normally located
in toggle_archive (called by action_archive and action_unarchive).
Task ID 2170708
Community PR odoo/odoo#46563
Previously a blacklisted phone number or email address could only be
removed from their respective blacklist by going to the corresponding
blacklist view [in mass_mailing_(sms)] and manually
archiving/deactiviting the entry. All users could see that an email was
blacklisted via an fa-ban icon added next to their corresponding field
in the modules: crm, mass_mailing, mass_mailing_sms, and contacts (via
extension).
This commit adds additional fa-ban icons next to blacklisted phone
numbers and makes all instances of these icons clickable to remove them
from their corresponding blacklist using a wizard. There are several
limitations to this implementation including:
1. If both a mobile and phone number field appear within a crm lead
instance, then only the mobile field will indicate if it is blacklisted
(due to current implementation of PhoneMixin which only checks 1 phone
value against the blacklist and "sms" which returns mobile numbers first).
I.e. if a phone field value is blacklisted, but a value is typed into the
mobile field, then the phone field will never indicate that it is blacklisted.
2. If someone clicks on a unblacklisting icon after changing the
corresponding field value without clicking off the field input, then the
latest typed in value will be the one submitted for unblacklisting,
which may not actually be blacklisted (due to "is_blacklisted" flag
being a computed field and it not having a chance to re-compute to hide
the button).
3. If someone changes values in the blacklist, then someone who already
has a form open will not see icon disappear until something triggers a
re-compute of the "phone_sanitized_blacklisted" flag (this was already the
case with the icon, but now a user may try to unblacklist a value that is
already unblacklisted.)
4. Since the icon needs to be visible by everyone to indicate whether or
not a phone/email is blacklisted, users without corresponding unblacklist
permissions will be able to click on the icon. When they click on it, they
will be informed they do not have permission to unblacklist. A wizard was
determined to be the best way to implement this unblacklisting for the
following reasons:
- A "Unblacklisting Reason" is needed and will not be stored as a field.
- A field widget is unable to do this due to security settings +
inability to refresh view after unblacklisting with a
"Unblacklisting Reason" without wiping unsaved changes.
Additionally, to better align with GDPR, the corresponding form view for
the phone/email blacklist has been updated to use the same wizard to
track "Reason for unblacklisting". Unfortunately there is no
straightforward way to prevent direct "Archive" action, so users are
still able to bypass unblacklisting without being asked for a "reason
for unblacklisting".
Supports task: 2117635
Upgrade PR: odoo/upgrade#939
COM PR: odoo/odoo#45315
PURPOSE
Perform some code cleaning in mail module
SPECIFICATIONS
mail.address.mixin is used only for mail.thread.blacklist mixin. Let us
merge those two and simplify class organization.
LINKS
Task ID 2222033
closesodoo/odoo#48115
Related: odoo/upgrade#967
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This branch is the combination of several optimizations in the ORM:
* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;
* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;
* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);
* make method `modified` take advantage of inverse fields to inverse
dependencies;
* filter records by evaluating a domain on records in Python;
* a computed field with `readonly=False` behaves like a normal field
with an onchange method;
* computed fields are computed in superuser mode by default.
Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.
closesodoo/odoo#35659
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
PURPOSE
Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.
SPECIFICATIONS
The number of message bounce is incremented each time the email bounce
on a specific email address. To have a correct information, we should
reset to zero this counter if we receive a mail from this address.
Indeed, if we receive an email from an email address, this email is
active and the message bounce number (if > 0) is not relevant anymore.
Only models inheriting form blacklist are impacted by this improvement
in order to limit a bit side effect.
Also, add a check in autoblacklist rule. No need to check the stats
if message_bounce is < 5.
This commit introduces
* ``_routing_reset_bounce`` routing method managing the bounce reset;
* ``_message_reset_bounce`` model method that resets bounce counter
in blacklist enabled models;
LINKS
Related to task ID : 1893155
Linked to PR #33340
PURPOSE
Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.
SPECIFICATIONS
Purpose
* move bounce information detection in message parsing. It allows to have
this information available in various steps of routing instead of having
to manually re-compute them;
* handle bounce in specific methods allowing easy override;
* improve bounce management, notably when detecting a bounce not linked
to the bounce alias configuration;
* better integration with blacklist mechanism;
Specifications
* compute bounce information in ``_message_parse_extract_bounce``.It parses
bounce information and returns a dictionary allowing to update parsed email
values;
* remove override in mass_mailign that basically does what mail already
does;
* manage bounce in ``_routing_handle_bounce``;
* when detecting a bounce, correctly call the bounce management method on
all models inheriting from blacklist;
* correctly update bounce counter;
* bounced mailing traces and automatic blacklist in mass mailing should
be done in ``_routing_handle_bounce``;
* add some tests;
LINKS
Related to task 1893155
Linked to PR #33340
Multi is the default api for methods, it is not necessary to explicitly
decorate methods with it, adds clutter and most people use it because
they see that the rest of the code uses it.
Done with `find . -type f -name '*.py' | xargs sed -i '/@api.multi/d'`
* mail.blacklist.mixin now clearly make inheritance on mail.thread. Indeed
we consider models using the blacklist mechanism as being used in mailing
features, meaning they will anyway inherit form mail.thread. In standard
Odoo it is the case for the 3 models using it (Lead/Opportunity, Contact
and Mailing Contact);
* define message_bounce directly in mail.blacklist.mixin. Bounce counter
is indeed linked to the blacklist and mailing mechanism;
* rename mail.blacklist.mixin to mail.thread.blacklist to ensure coherency
with mail.thread and mail.thread.cc (another mixin build on mailL.thread);
Inherit declarations in various addons are updated accordingly.
Related to task ID 1911679
Linked to PR #29483
Purpose is to clean the use of tracking parameters on fields. Parameters are
merged and is now tracking=<int> or tracking=True.
This commit is linked to task ID 1903814 and PR #28430.
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
Apply the mail.address.mixin on blacklist mixin. So that every model
inheriting the blacklist.mixin will also inherit the address.mixin.
That simplify also the comparaison to - and the searches on - the
blacklist.
Task ID 1896677
The objective of not creating the record if the email was invalid
was to avoid crash on import. But if the number of record created does not
match the number of record that would have been created (see in flush()
in models.py), this cannot work.
Also, it's necessary to tell the user that his import file has an error
and where. This is why a UserError must be returned.
Task ID 1902139
Closes PR #28011
Purpose of this commit is to make search coherent with sanitize of emails
done when storing blacklist entries. We parse search terms in order to
find something related to email field and sanitize it to avoid issues
linked to case insensitivity or formatted addresses.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
Purpose of this commit is to correctly extract email address from the email
field of models inheriting from the blacklist mixin. Indeed email field
could contain a formatted address like "Raoul Grosbedon <RAOUL@example.com>".
Related blacklist entry would be raoul@example.com. We have to extract
the email from the email field and lowerize it in the SQL query or the
computation in order to have a fully working blacklist mixin. This fixes
some bits left at commit 2ff9b379ef.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
This commits fixes the use of email field in blacklist mixin as it was based
on 'email' field instead of the one coming from the _primary_email class
attribute since commit 2ff9b379ef.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
Now that blacklist records are set as lower case be sure all checks in
blacklist are done accordingly by using the sanitizer before comparing
email adresses.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
Currently blacklist is configured to have unique emails. Notably when creating
new entries it checks for existing records with same email to avoid crashing
about duplicate entries.
However it does not check if given values use unique emails. When importing
a big list of emails you are not sure emails are unique. This commit ensures
values given to create are correct to avoid crashing when importing a lot
of data.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
Purpose of this commit is to standardize content of blacklist records and
ensure only real email addresses are stored.
We introduce a sanitize method to extract email address from string (like
Raoul Grosbedon <raoul@example.com> -> raoul@example.com) and return a
lower case version of it.
Purpose is to gradually get rid of lower() that appear a lot in the code
and speedup computation by being sure everything is always lower case.
Moreover indexing on blacklist model will be more performant is using
directly email field content instead of having to lowerize it.
This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.
As building a custom filter on 'Blacklist is false' is creating a filter '!= True'
instead of '= False', the assumption that operator must always be '=' is wrong.
This commit inverts the value and operator if the operator is '!='.
Fix for Task ID 33224
PR #27334
Several quick fixes:
- repair malformed SQL queries and injection problems
- proper token comparisons (consteq)
- better translatable terms (less context-lessplaceholders)
- basic testing of case-(in)sensitivity of blacklist
- safety check for non-implemented cases in search methods for computed
field
More testing and review is needed for case-sensitivity and email
extraction from generic "addressing fields".
In unsubscribe page, if the user try to come back
(and be removed from the blacklist)
but his email have been deleted from the blacklist,
it was leading to an error.
Instead, and in order to keep a trace of what the user asked,
if the email is not found in the blacklist, the record will be
created but archived directly.
Also, it aligns the return of both methods _add/_remove
Purpose
=======
Improve mailing subscription to be more compliant with the European GDPR law.
Keep a list of people who does not want to receive promotional emails
(or mass mailing in general) anymore.
Specifications
===========
This commit is regrouping some main changes on mass mailing.
- Added blacklist : Avoid sending mass mailing to blacklisted recipient
(blacklisted = email address that doens't want to receive mass mailing anymore)
Detailed implementation
===================
Mail :
- Add Blacklist mechanism in mail module (NOT in mass-mailing) :
as we can send mass-mail without the mass-mailing module
- Unicity in email -> To avoid error in import, override the create and return
the existing record if any, else, create the record normally.
- Blacklisting is done by email address and is cross model.
Will apply to model that inherit the blacklist.mixin.
- field 'is_blacklisted' -> computed : check if email is in blacklist
+ search method to be able to filter on is_blacklisted
- When a email address is blacklisted, it will never get mass mailings anymore.
Even if the email address is added to another mailing lists
- Avoid sending notification to blacklisted recipients when sending email
in mass mail mode. If the recipient is blacklisted, we should not even send a
notification in the recipient's chatter for an email that he won't even
receive.
- When a email address is blacklisted, it can still get 'normal' mailings.
- The blacklist shoud be accessible in
Mass Mailing / Configuration / Blacklist
and Settings / Technical / Email / Blacklist
-> Renaming Settings / Technical / White / Black List config menu item
into Channel Moderation to avoid confusion with Mass Mail Blacklist
- Add indexes to the blacklist table (on email) to make it fast for access for
the different use cases
- _primary_email : attribute that must be overriden to specify which field must
be used as email in the blacklist mechanism.
- Filtering the blacklisted recipient in mail composer :
done in mail._get mail value()
In case of real mass mailing, we need the statistics to be computed in order
to know how many recipients were ignored in the mail.
So we cannot avoid sending mail but instead flag the mail as canceled.
Task ID 33224
Thibault Delavallée