Prevent archiving in-use mail servers by displaying an error message that
lists where it is still used, allowing to easily identify what need to be
updated before being able to archive the mail server.
Additionally,
- prevent the use of archived server as a fall-safe
- when duplicating a mailing with an archived mail server, replace mail
server by the default one
Detailed explanation:
1. A check has been added that raise an exception when trying to connect to the
smtp server or send an email when the server is archived.
With that solution,
- testing the connection of an archived server displays an error telling that
an archived server cannot be used.
- if a mail is still sent with an archived server, mail are in error :
"Connection failed (outgoing mail server problem)"
This fail-safe ensures that no mail will be sent through an archived mail server
and that the user will get some feedback about it.
The same fail-safe for the incoming mail server has been added.
Notes:
- the connection will outlive the archiving of a mail server still allowing
to send email through the archived server until the connection is closed. But
connection are not kept for long so this shouldn't be a problem.
- it cannot be tested because the connect method return immediately in test
mode.
2. When a mail server is archived, an user error is raised if it is in-use.
The implementation relies on each module to override the method
"_active_usages_compute" in "ir_mail_server" to complete the list with
user-friendly message describing the active elements that could send mail
through the mail server. This has been implemented for:
- l10n_it_edi: server used to send e-invoice
- mail: optional server configured for template
- mass_mailing:
-- default mail server
-- active server configured for mailing
Mail server are referenced in other elements but are not active anymore, it is
just for temporary or history purpose. Those references doesn’t prevent the
archiving of the mail server:
- mail_message
- wizard survey_invite and compose_message
- res_config_settings
Task-2821516
closesodoo/odoo#91240
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Have more reliable tests.
Better spot side effects coming from sub addons.
Lessen non deterministic counters due to local db.
SPECIFICATIONS
Make crm, event and mail performance tests post install.
Update query counters with
* local values (install module only with enterprise activated);
* community / enterprise runbots (if value is different);
* some notes on non deterministic issue if known;
Task-2925606
closesodoo/odoo#96446
Related: odoo/enterprise#29726
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
In this commit we introduce a new module ``test_mail_sms`` that holds the
tests for sms application, like ``test_mail`` does for mail. Currently all
sms tests are inside ``test_mail_full``. After this commit code is moved
from ``test_mail_full`` into that module. Full testing module should be used
mainly to test integrations with a lot of submodules and for performance
tests, not testing details of SMS implementation.
We also add ``mass_mailing_sms`` as dependency of ``test_mass_mailing`` so
that both mailing types are tested in the same module. It eases maintenance
and writing of tests. Mass mailing SMS tests from `test_mail_full`` are
therefore moved in ``test_mass_mailing``.
This commit also allows some cleaning in classes used in tests, to have
classes in ``test_mail_sms`` and ``test_mail_full`` that contain everything
necessary to test mail features.
Task-2890111 (Test Mail/Mass Mailing: SMS tests reorganization and move)
Part-of: odoo/odoo#96223
Add a model and a test allowing to test the seen list using raw SQL based
on partner_id field. Test indicates a not stored partner_id field currently
crashes beyond redemption.
Task-2852943
X-original-commit: a15bda3bed339a0ac422a3c82ec9cc295a11a785
Part-of: odoo/odoo#94532
Purpose
=======
Improve the performance of mass mailing by removing the <mail.mail>, in
batch, all at once. The unlink operation is very expensive for the ORM,
in terms of SQL queries.
A new field `to_delete` is required to know which <mail.mail> we need
to delete. The reason is that the `failure_type` is stored on the
<mail.notification>, and it's possible to have <mail.mail> without
<mail.notification>.
Task-2587345
Part-of: odoo/odoo#73271
Only the group_salesman can see the opportunities statbuttons
Fix a few not-smart mass_mailing tests:
- running the mass mailing queue processes the messages as the user
running the queue, and will try to send the demo messages, depending
on demo data (when re-running the tests) the test user may not have
the accesses required; sending just the one test email avoids that
issue
- the leads kpi assumes the user has access to leads, that ain't
necessarily the case
For the second issue, when sending statistics if the mailing has a
user `_prepare_statistics_email_values` should be called with that
user as "current" (cf `_action_send_statistics`), so we can just work
off of the current env, though it might be a good idea to eventually
check that.
closesodoo/odoo#89774
X-original-commit: 39af865800c6752b60171f16d6056ceb3c5a1636
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When using a template with inline placeholders in the body, the Test
button will replace the placeholders, but they will remain in the final
body when the mass mailing is sent for real.
This commit fixes this issue by using the correct rendering engine for
the Test button too.
OPW-2819032
OPW-2828461
closesodoo/odoo#89346
X-original-commit: b0dcd2fa35d949dd722f1bc70401ceeb7f5d8231
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Paul Morelle <pmo@odoo.com>
PURPOSE
Improve tooling of mail tests. Lessen custom code, try to use same tools
in all tests.
SPECIFICATIONS
Use MailCommon as a base class for performance test. It now uses the same
boilerplate as other tests, leading to easier to reproduce and understand
testing setup.
This notably now adds outgoing mail servers which means adding one query
to some tests involving emails.
Task-2673913 (TestMail: Cleanup and improve test coverage)
Part-of: odoo/odoo#86393
mail_* = mail, test_mail, test_mail_full, test_mass_mailing
PURPOSE
Improve tooling of mail tests. Use SMTP mockup class available in base tests
in mail tests to have a better and simplified mock tool when dealing with
mail tests. Remove old fashioned code.
SPECIFICATIONS
Include ``MockSmtplibCase`` directly into ``MockEmail`` so that we re-use
existing components for mocking outgoing emails. Also make parameters for
gateway coherent between those two classes (use same test values).
Remove 'sim_error' weird parameter of Mail ``mock_mail_gateway`` and use
standard mock 'side_effect' instead in some specific tests, notably on
SMTP.connect() mock.
Add some assert / tools method and improve existing tools, used for creating
or improving test data. This will be used in future commits when adding new
tests.
Add outgoing mail servers data in mail tests, allowing to make tests closer
to real life use cases.
Remove unnecessary calls to ``_init_mail_gateway`` as it is automatically
done in ``MailCommon.setUpClass()``.
Try to consolidate starting test data, notably with countries, phone numbers
or email in mind to be sure test mail are deterministic.
Task-2673913 (TestMail: Cleanup and improve test coverage)
Part-of: odoo/odoo#86393
Followup to #82105: turns out we kind-of forgot that records could be
updated with new attachment and the exact same issue could occur.
So with the same reasoning as the previous PR, re-attach attachments
to the current object when updating it.
closesodoo/odoo#83083
X-original-commit: 398070ec1b6ed7d6a8e7c0ab75d45b8a9ad0e0d0
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When the system broadcasts an email response to document followers,
if the config parameters `mail.force.smtp.from` or
`mail.dynamic.smtp.from` are defined, it will rewrite the `From`
address to avoid spoofing the sender's domain.
**NOTE**: As of 15.0, this is based on the `from_filter` setting on the
corresponding ir.mail_server, rather than the abovementioned config
parameters, but the rest of the discussion stands.
For example, if the `mail.catchall.domain` is set to `example.com` and
an email response comes from:
"John D" <john@doe.com>
it will rewrite it to:
"John D (john@doe.com)" <notifications@example.com>
This will make sure the system never sends outgoing email for an external
domain, as it has no authority for doing so, and that could
break mail filtering/authentication rules (SPF, DMARC, etc.)
During this "encapsulation rewrite step", both the original Sender name
and their email are preserved, and put into the quoted "name" field of
the rewritten address. It seems sensible to preserve as much information
as possible about the original sender.
Unfortunately, the inclusion of the Sender email in the final name makes
it appear to some inbox providers as if the message is trying to
deceptively impersonate another person (as many phishing schemes would).
As of November 2021 GMail at least does this, and will hide the name in
the UI when it happens. It will keep only the rewritten email, which is not
very useful in the case of a notification (even though it's more
technically correct, of course).
This patch removes the original email from the rewritten notification,
keeping only the name, considering that the email is not the most
important part, and it's better to have one of the two than none.
So after the patch, the rewritten address is now:
"John D" <notifications@example.com>
When there is no name in the original address, we keep only the local
part of the email, to avoid the same display issue. The recipient will
have to identify the sender based on the context / past messages.
closesodoo/odoo#81807
X-original-commit: 3c65ec5a8191a392980ceb0a8c584767eae405f1
Signed-off-by: Olivier Dony <odo@odoo.com>
When mailings have no responsible, KPIs emails are broken as there is no
from and to on the mail. This leads to mails being invalid and set in
exception.
Moreover currently author and from / to of those D+1 emails are not coherent
as author is current user (generally odoobot as this is generated through
a cron) while from and to are based on mailing responsible.
In this commit we make statistics emails more coherent
* when a responsible is set on the mailing: author, from and to are linked
to the responsible;
* when there is no responsible, current user is set as it may be sent by
regular people;
* reply-to is set to company email as it is often the case with 'marketing'
emails;
Task-2686586 (Repair mailing statistics email)
X-original-commit: 95a21ca16f560cf4341c547f6bc909d3261acd1b
Part-of: odoo/odoo#79877
When there is an issue updating mailing state (concurrent access, or some other
error that may happen), mailing statistics emails may be sent in loop. As we
do not think timing is so important, we now use the email queue to send
statistics emails.
Task-2686586 (Repair mailing statistics)
X-original-commit: f78ac45c2eafc3cf6b213e1f1608e76e243bd8d2
Part-of: odoo/odoo#79877
Recently some "de-t-rawify" [1] was done through Odoo. A change in digest
layout broke mailing statistics. Indeed mailing statistics are build on digest
main layout to re-use it. In mailing case a ``kpi_name`` value was missing when
rendering statistics columns, leading to a crash. This is now fixed in both
mailing and sms marketing.
In this commit we also add tests for statistics emails (D+1 KPIs) rendering
for both mail and SMS marketing mailings. Email details are also tested
in order to better highlight future changes and tweaks in email sending
linked to D+1 KPIs emails.
Task-2641394 (Digest emails sending improvement)
Task-2582128 (Digest onboarding and usage improvement)
Task-2686586 (Repair mailing statistics email)
[1] odoo/odoo@c56e8c9f5a
X-original-commit: c9905ceb3172d8e80e3aff4ae9bc925dcb6b6519
Part-of: odoo/odoo#79877
Purpose of query counter tests is to try to match real life use cases. In
mail those generally involve a correctly configured mail gateway. This is
why we set those parameters in performance tests, leading to a small increase
in some counters.
Task-2661036 (Performance tests data cleanup)
Prepares Task-36879 (MultiCompany Aliases)
Part-of: odoo/odoo#77845
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
PURPOSE
Improve modeling and performances of mass mailing by manually updating trace
status instead of using complex computed fields and lessening fields usage.
Remove some fields and keep only relevant metrics to simplify and clean trace
model.
SPECIFICATIONS
In this commit we clean the way trace state is managed. Currently it is a
computed field based on several datetime fields. However this generates a
lot of noise in the table as well as unnecessary computation
* there are several columns (one for each state) storing datetime at
which status was reached. Generally only 2 or 3 contain relevant
information;
* state could be set in code directly to avoid a computed field based on
many triggers;
* recomputing it each time a date changes is not necessarily necessary;
* state value can always be updated manually as this is main done through
some automated server update (mailgateway, link clicks, ...);
As trace states and its triggers should not be updated manually it is better
to synchronize it in code flow. When there is an exception or update done
through sending or gateway status is updated as well accordingly. Various
datetime fields are also updated at the same time. In order to align with
notification model mail and sms trace status are updated to a classic field.
Only last status update is now kept as there is no need to store the entire
history of status change.
We keep only a datetime for relevant metrics: open, reply and click. Other
datetime bring no real value. Knowing when a trace was in error or bounced
is not necessary. Indeed exception generally indicates a server issue (at
sending), cancel indicates a data issue (at sending) and bounce depends on
customer email server.
Status update is removed as using write_date is sufficient. Once created
traces are updated only when an external event occurs (opened, replied, ...).
It allows to simplify trace model.
We also rename ignored field into canceled to match naming use through mail
and sms.
QUERY COUNTERS
This change has some positive update on query counters when sending mailings
as traces have less unnecessary status update compared to priori this change.
LINKS
Task ID-2377974
Community PR odoo/odoo#61467
Enterprise PR odoo/enterprise#14633
Upgrade PR odoo/upgrade#1907
RATIONALE
Currently there are differences between mail and sms error management
especially when sending them in batch (mass mode). Moreover cancel
(ignored) and error (exception) states meaning is not clear. Finally
some failure types management between mail and sms can be cleaned.
SPECIFICATIONS
Meaning of ignored / error we want to enforce now is
* error: there was something wrong at sending and user has an action to
perform, i.e. server failed -> check its logs;
* canceled: invalid recipients due to contact information or mailing
configuration (blacklist, opt out, void or invalid email or phone number).
In indicates issues linked to records themselves;
In this task we also add failure information granularity on mailing traces
linked to email like what is done currently on SMS. This can be related to
mailing (blacklist, optout, duplicates) or related to recipient (no recipient,
incorrectly formatted).
We also correctly distinguish optout from blacklist when sending SMS.
SPECIFIC USE CASES
* recipient without email / number: mail / sms is set as canceled, trace is
ignored;
* recipient with invalid email (no @) / number (formatting impossible):
mail / sms is set as canceled, trace is ignored;
-> we now distinguish when possible a void email from a wrong email using
a newly-added selection key (mail_email_missing);
* recipient with email / number blacklisted: mail / sms is canceled, trace
is ignored;
* recipient with email / number that optouted from mailing: mail / sms is
canceled, trace is ignored;
* recipient with email that bounces: mail is sent and will be set as bounce
when receiving bounce in gateway; trace follow same path;
* recipient with number that bounces: not supported as currently no support
of bounce through IAP;
* mail server error, IAP error: mail / sms is set as exception / error,
trace is set in exception;
This means we introduce new failure types on mailing.trace model to reflect
those failure types
* ``mail_missing``: missing email (different from wrong value);
* ``mail_bl``: blacklisted;
* ``mail_optout``: optouted;
* ``mail_dup``: duplicated email skipped during mass email send;
We introduce ``sms_optout`` on SMS and trace models as it was merged with
blacklist previously. Now both errors are distinguished.
LINKS
Task ID-2377974
Community PR odoo/odoo#61467
Enterprise PR odoo/enterprise#14633
Upgrade PR odoo/upgrade#1907
Since a few time, test_mail holds tests linked to specific test models.
Generic testing of mail features are located in mail. Followup of fix done
in 12 and forward ported until odoo/odoo@b7af68590e
Also rename test_mass_mailing_server.py unit test file to test_mailing_server
to match current naming of files.
Task ID-2377974
Community PR odoo/odoo#61467
Update query counters according to last runbot state. This helps spotting
query counters change linked to this PR.
Task ID-2377974
Community PR odoo/odoo#61467
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
Purpose
=======
The purpose of this new module is to manage the mailing lists. Now they
are no more <mail.channel> (will email_send set to True) but they have
their own model.
Specifications
==============
The mailing list are basically a public discussion that users can have
by email. They can respond to email, send new messages, etc... All the
members of the mailing list will receive the message by email.
Users can moderate the emails of the mailing list in the same way as
they did with the "email" mail channel.
- *accept*, will accept the emails and send it to the members of the
mailing list
- *discard*, will drop the email without warning the author
- *reject*, will drop the email and send a notification email to the
author
- *allow*, will accept the email and all other pending emails of the
same author and create a whitelist for him
- *ban* will drop the email and all other pending emails of the same
author and create a blacklist for him
Now a portal view is available (in /groups) and replace the old module
"Website Mail Channel" that was removed in the previous commit. In this
view, users can subscribe / unsubscribe to the mailing lists and some
links to this portal view are added in the footer of the emails of the
mailing list.
As before with mail channels, you can send guidelines to the members,
notify the moderators whose an action is required (in a CRON), send back
an email to the author of an email to say "Your message is waiting
moderation"...
Links
=====
Task-2510267
See odoo/odoo/pull/71599
See odoo/enterprise/pull/19296
See odoo/upgrade/pull/2600
Traces are always linked to documents as mailings are sent on
documents. Those fields should be required to ensure database
coherency.
Task ID-2525759
closesodoo/odoo#71786
Related: odoo/upgrade#2513
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The optimization 2ccf0bd0dc does not take into account that to
"unblacklist" a user, you have the archive the mail.blacklist record so
only mail.blacklist active records need to be taken into account.
Added test without the fix fails with:
AssertionError: False is not true : MailTrace: email
test.record.02@test.example.com (recipient res.partner(), state:
sent, record: mailing.test.blacklist(166,)): found 0 records (1 expected)
opw-2536304
closesodoo/odoo#71277
X-original-commit: 965c67cb16d2a4ac10d35a01da3af311867a47e6
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
PURPOSE
Remove hardcoded list of models on which mailing is possible. Indeed it is
not modular and not really smart with enterprise code not being reachable
in community.
SPECIFICATIONS
Replace hardcoded list of models on which mailing (both mail or sms) is
possible by a computed searchable field on ``ir.model`` based on a class
attribute.
It allows to cleanly define models having mass mailing capabilities and add
this attribute in bridge modules (when existing) or directly on base model
definition to avoid bridge modules.
In this commit we introduce a basic ``_mailing_enabled`` class attribute
activating mailing on model.
Mailing models may also have a ``_mailing_get_default_domain`` method allowing
to define a custom default domain when sending a marketing mailing on records
on this class.
Mailing models can now define a ``_mailing_get_opt_out_list(_sms)`` method
allowing to define custom behavior to fetch opt-outed records. Instead of
defining a model-based behavior on Mailing itself, it now calls the model
defined one. We still have two methods, one for mailing and one for SMS
opt out computation as it relies on different underlying models and fields.
LINKS
Task ID-2431217
COM PR odoo/odoo#67322
ENT PR odoo/enterprise#16876
UPG PR odoo/upgrade#2236
In `prepend_html_content`, if body `html_body` and `html_content` are
`Markup` objects, the processing of `html_content` through `re.sub`
will strip away the `Markup` flag, leading to the later concatenation
of the body and content first escaping the content (to safely convert
it to a `Markup`) before performing the concatenation proper.
As a result, the `preview` of a `mailing.mailing` would be
double-escaped, and the markup hiding it would instead be printed as
part of the body.
Assume that the type of `html_content` is str-compatible, and
immediately rewrap the value after processing it through
`re.sub`. This should do nothing if `html_content` was an actual
`str`, and will re-flag it as a `Markup` if it was one.
Also change the finding of the insertion point to use `re.search`, I
don't understand why this uses `finditer`, it just makes the code more
complicated.
closesodoo/odoo#71090
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Add a unicity constraint (on the url, UTM campaign, medium and source).
As we must care about null values, we can not implement it in SQL
(possible in pure SQL, but we will need to create an index on the
columns, which is not possible in Odoo if we want to customize the
error message).
Clean the code, before the "search or create" behavior was implemented
in the create, causing strange behavior (creating a link tracker with
the same values as an existing one didn't raise an error, but open
the form view of the existing link tracker).
Task-2090344
closesodoo/odoo#48845
Related: odoo/enterprise#9608
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Propagate reply_to radio keys (update and new) to mass mailing in order to have
a coherent naming (was thread and email). This naming is also coherent with
gateway naming (message_update and message_new).
LINKS
Task ID-2117639
COM PR odoo/odoo#40931
ENT PR odoo/enterprise#17941
UPG PR odoo/upgrade#2419
Purpose of this commit is to add tests related to answers through mail gateway
when having reply-to mode of mailing set to either updating existing threads
(aka replying on mailed records), either creating new threads (aka using the
reply-to alias configuration).
LINKS
Task ID-2117639
COM PR odoo/odoo#40931
In this commit we try to simplify tests organization by
* merging mailing_internals and mass_mailing tests, as they globally handle
mailing-related tests (blacklist, opt-out, gateway update through replies
and clicks);
* move wizard (mailing.test) tests in their own file as they target a
specific model;
* add some loggers to reduce noise;
* add some users wrappers to test using real users;
LINKS
Task ID-2117639
COM PR odoo/odoo#40931
PURPOSE
Purpose is to have more tests when sending mail mailings, notably about
canceled or failed mails or sms as well as jinja and links rendering.
SPECIFICATIONS
In this commit we improve Email Marketing mailing tests. We notably
* add tests for void and invalid email and numbers. It allows to check they
correctly update their trace status;
* add tests for unsubscribe and view links embedded in mass mailing emails;
* add tests to simulate a click on links sent through mass mailing and
ensure click statistics are effectively updated;
* add tests to simulate bounce emails coming back to the mail gateway;
Default content of mailings used in test is updated to ensure jinja is
correctly rendered, including links and some corner cases. This will also
helps ensuring behavior is kept when converting to QWeb.
Various docstrings are added to helpers and custom asserts.
LINKS
Task ID-2508643
Followup of odoo/odoo#68874 (improve mail tests)
Prepares Task ID-27033 (support QWeb in templates)
Prepares Task ID-2377974 (clean trace and status management in mass mailing)
COM PR odoo/odoo#69461
ENT PR odoo/enterprise#17780
X-original-commit: 7b940ad46bde30f9bf16837f7bff5a70cd88c640
When using assertMailMailWEmails custom assert we now check that a sent
mail.mail matches a sent email. It allows to have a complete check from
mailing traces to sent emails.
Task ID-2500615
COM PR odoo/odoo#68874
ENT PR odoo/enterprise#17558
X-original-commit: 59da8627e0d732b12fb98faa6e66e43b8c75dc9c
The goal of this change is to simplify the code managing `create_date`
and `write_date` in methods `create()` and `write()`, and also to remove
weird behaviors caused by the way those fields were updated.
Assume we update a simple field on a record. This adds pending updates
for the field and `write_date`. However, the value of `write_date` is
not known yet: it will be updated as `NOW() AT TIME ZONE 'UTC'` in SQL.
So `write_date` is actually given a dummy value in pending updates, and
it is invalidated from cache, until its value is flushed to the database
and fetched again.
Now assume we access another field on the record, and that field is not
in cache. The prefetching mechanism will read all column fields,
including `write_date`, and flush them first.
# this adds pending updates foo: 42, write_uid: 1, write_date: False
record.foo = 42
# assume 'bar' is not in cache; this prefetches all column fields,
# which flushes the pending updates above before reading them back
result = record.bar
We can avoid flushing pending updates if the values read from database
do not overwrite existing values in cache. If you assume that the value
of a pending update is in cache (in the example, `foo: 42`), you don't
need to flush the corresponding field. Indeed, the value of `foo` will
remain 42 in cache, whatever its value in the database. This assumption
(pending updates are in cache) is true for all fields *except* for
`write_date`: it is invalidated from cache, and given a dummy value in
pending updates. This branch actually makes this assumption true for
all fields. The avoidance of flushing pending updates will be done in
another commit.
In order to directly assign `write_date` its value, we use a cache for
the value `NOW() AT TIME ZONE 'UTC'` from the database. This costs at
most one query per transaction, and potentially saves a few queries.
Co-authored-by: Victor Feyens <vfe@odoo.com>
Adapt some query counts to their optimal value, in order to measure the
effect of the following commits on queries.
In module test_performance, some query counts were actually not correct:
the initial flush() done by the context manager assertQueryCount() may
prefetch some data to the cache, and that prefetching is not accounted
for in the query count. This is very true when assertQueryCount() is
preceded by a cache invalidation. We have to move the invalidation
inside the context manager, so that the prefetching is now counted.
This test ensures that when using the test sending tool of mass mailing (sms)
a wrong jinja content is detected if we have any record available to evaluate
it.
PR odoo/odoo#55696
Task ID-2312442
X-original-commit: 183a7677616ac36ed06109e22288aacb185ac097
Make the method `_search` return a `Query` object, and make that object
generate a subquery when used on the right-hand side of a condition.
closesodoo/odoo#52403
Related: odoo/enterprise#10945
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
PURPOSE
Improve the Email Marketing tour and onboarding to create a "wow effect" when
users test the app.
SPECIFICATIONS
Create a tour for mass_mailing to let user visit Odoo Email Marketing through
a simple mailing creation flow.
This require the following functional changes
* by default, if user chooses to contact mailing lists and there is a single
mailing list available, add it as default value for chosen list;
* create a master mailing list data containing the admin user so that he can
easily follow the tour and test the mailing application;
LINKS
Task ID 2194876
Community PR odoo/odoo#48259
Upgrade PR odoo/upgrade#983
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Modules holding tests and helpers
* link_tracker: mainly mock, asserts and tools for link tracker tests
(MockLinkTracker);
* mail: mainly gateway mock and base for mail tests
* MockEmail -> mocks for mail gateway;
* MailCase -> tools and asserts for mail tests;
* MailCommon -> base for mail functional tests);
* sms: mainly SMS gateway mock and base for sms tests
* MockSMS -> mocks for SMS gateway;
* SMS Case -> tools and asserts for mail / SMS tests;
* SMSCommon -> update of MailCommon with SMS capabilities);
* mass_mailing: mainly asserts and tools for mass mailing tests
* MassMailCase -> update of MailCase for mass mailing tools and asserts;
* MassMailCommon -> update of MailCommon with mass mailing);
* mass_mailing_sms: mainly asserts and tools for mass SMS tests
* MockMassSMS -> update of MockSMS for mass SMS tools and asserts;
* MassSMSCommon -> update of MassMailCommon with SMS capabilities);
Modules for tests
* test_mail: module for mail app tests (TestMailCommon);
* test_mass_mailing: module for mass mailing app tests (TestMassMailCommon);
* test_mail_full: tests integrating all discuss features, currently mainly
mail and SMS (TestMailFullCommon);
Enterprise: update test_mail_enterprise and test_marketing_automation
Task ID 2247037
Community PR odoo/odoo#50384
Enterprise PR odoo/enterprise#10266
Upgrade PR odoo/upgrade#1122
USE CASE
Send a mailing to some recipients. Set reply-to of this mailing to an alias
creating records on a model inheriting from utm.mixin. UTM informations on
mailing are not propagated to the newly-created records.
SPECIFICATIONS
When an user answers to a mail of a mailing with an alias creating records
on an UTM enabled model, propagate UTM from the mailing to the new record.
In order to set UTM informations during record creation, "message_new" has
been overridden in mass_mailing. That way it is automatically available on
all models inheriting from utm.mixin.
A test simulating this scenario has been added. It checks if UTMs info are
correctlwy set on the created record by comparing them to the mailing's UTM
info.
Task Id 2210334
closesodoo/odoo#48602
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Consider an email going through the mail gateway. Its headers References
and In-Reply-To are used in mail gateway to find if it is an answer to an
existing discussion thread.
Notably in mass mailing it is used to set mailing traces (called statistics
before 13.0) as opened and replied. However currently only the References
header is used.
In this commit we now use References or In-Reply-To like what is done in
mail gateway.
== 13.2 FORWARD PORT SPECIFIC ==
Code in tests has been updated to use new mockups tools and variables available
in test_mail
LINKS
Task ID 2257717
PR #51445
Forward-port-of: #51319
Forward-port-of: #51247
X-original-commit: 9921e38692e7dc4547afc34acc395da340eca410
URPOSE
Prepare code cleaning and optimization in mail, mass_mailing and SMS by
cleaning models for readability and code complexity and footprint reduction.
SPECIFICATIONS
Add tool methods and asserts for link tracker tests.
Add tool methods and assets for mass mailing tests, notably about mailing
traces and their link with emails.
Merge existing and duplicated tests and rewrite them a bit.
Rename and improve test_mass_mailing models.
LINKS
Prepares task ID 2238597 (notification and trace models cleaning)
Task ID 1906925 (mass mailing tests cleaning)
PR #50169
Related: odoo/enterprise#10180
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Several counter fields are performed inefficiently. They use compute methods
using length of 2many fields instead of making use of more efficient ORM
methods like read_group.
SPECIFICATIONS
No functional change, only technical performance improvements.
Purpose is to improve lead_count performance. Some people claim to gain a lot
of queries: from 90 to 30 SQL queries to read 39 partners. Hard to reproduce.
LINKS
Task ID 2196182
PR odoo/odoo#44662
Co-Authored-By: Fabien Pinckaers <fp@odoo.com>
Co-Authored-By: Victor Feyens <vfe@odoo.com>
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
PURPOSE
Perform some code cleaning in mail module
SPECIFICATIONS
mail.address.mixin is used only for mail.thread.blacklist mixin. Let us
merge those two and simplify class organization.
LINKS
Task ID 2222033
closesodoo/odoo#48115
Related: odoo/upgrade#967
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When the `select` returns nothing more than ids that are already known, there is
no need to make it at all.
This removes one query from `_read` every time it has to fetch only fields that
are stored in a different table (o2m, ...), which happens all the time when
reading a stored field first (triggering prefetch) and then reading a o2m.
The query that is now removed was used to check access rules, but the trick is
to use `check_access_rule` to verify the rules in python instead.
Part of task-2061122
closesodoo/odoo#36263
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Pierre-Yves Dufays