PURPOSE
Clean posting process and improve mail.message definition and comprehension.
SPECIFICATIONS
In order to be more explicit subtype parameter is renamed to subtype_xmlid.
It therefore clearly indicates it should be a valid subtype Xml ID. Support
of ill formatted Xml IDs is removed because there is no reason to try to
add some random prefix. Give something that exists or go to hell, punk !
LINKS
Task ID 2071556
PR #38692
From now on mail.mail is considered as a technical model. Indeed people should
not really manually craft mails by hand. Instead various functional flows
should either send mails, either craft mails based on some user input.
We therefore make mail restricted to admin users. Flows creating mail.mail
are updated to use sudo, and ensure it was done in a context that makes
sense to delegate this power to the user.
Task ID 1853147
PR #32243
Purpose of this commit is to give a way to access to company email and catchall
with formatting. Those will be used in various automated emails. Indeed
currently several templates use either ``company_id.partner_id.email``,
``company_id.email``, or even don't provide fallback values.
With this commit from a company record people will be able to use
* a correctly formatted catchall: ``"My Company Name"
<catchall_alias@catchall_domain>``
* an email_formatted field like partner email_formatted that is either its
partner-related email_formatted value, or formatted catchall if its partner
is not correctly configured;
Various calls to mail creation are updated accordingly.
Task ID 1853147
PR #32243
This is a performance fix. It avoids the cost of XML/HTML translations
for fields that may not be necessary when prefetching fields on records.
This patch marks such fields as non-prefetchable by default.
The code that changes the attribute `translate` on HTML fields (from
`translate=True` to `translate=html_translate`) has been adapted to
allow the setup of textual fields to mark the field as non-prefetchable.
Jairo Llopis made a comparative benchmark: evaluating `name_get` on
`event.event` records. The method needs the field `name` and without
the patch, the prefetching mechanism reads the translated HTML field
`description` as well. This patch speeds up the benchmark from 5800ms
to 800ms (see https://github.com/odoo/odoo/pull/37967#issuecomment-538364011).
closesodoo/odoo#40771
X-original-commit: e5ee5e5b65f85d66c8d59594ead9eece172c4282
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Pedro Baeza <pedro.baeza@tecnativa.com>
Co-authored-by: Jairo Llopis <jairo.llopis@tecnativa.com>
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Now that all tests using old ``BaseFunctionalTest`` and ``MockEmails``
classes are updated we can safely remove them.
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test message post
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test message management
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test mail message
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test mail gateway
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test mail channel
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test_message_composer, test_message_track and
test_mail_activity.
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test_mail_followers, test_mail_mail,
test_mail_template.
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in apps tests or force them to
write custom quick and dirty tools and asserts.
SPECIFICATIONS
Update tests to new tools / asserts / helpers / classes defined in mail
and test_mail. Notably
* ``BaseFunctionalTest`` class is replaced by the new ``TestMailCommon``
pimped one;
* ``assertNotifications`` is replaced by ``assertSinglePostNotifications``
(shortcut for a simple message_post) or ``assertPostNotifications``
(complete asserts involving several messages and notifications);
* correctly invoke ``mock_mail_gateway`` when mocking email sending;
* replace manual check of sent emails or ``assertEmails`` by
``assertSentEmails``;
* remove custom mocks / checks and replace them by now standard mocks
and assertions (if any);
In this commit we update: test_discuss, test_invite, test_ir_actions and
test_odoobot.
LINKS
Task ID 2068986
PR #38070
PURPOSE
Currently tools and asserts for mail tests are located inside test_mail
module. It makes difficult to re-use them in application tests or force them
to write custom quick and dirty tools and asserts. Purpose of this merge
is therefore to move tools classes and mocks to mail directly and use them
in various sub modules.
SPECIFICATIONS
In this commit we add ``TestMailCommon`` that will be the class for all test_*
modules linked to mail features. It is build upon tools made available in
mail tests (mocks, tools, asserts) and give the functional basis for mail
tests.
This commit also introduces ``TestMailMultiCompanyCommon`` that is a small
multi company testing class build on ``TestMailCommon``.
``TestRecipients`` class is kept as it creates some testing partners, used
notably for notifications and recipients.
Future commits will gradually make tests use this class instead of the
``BaseFunctionalTest`` that will be set to retirement.
LINKS
Task ID 2068986
PR #38070
The tracked field is now a relation to the corresponding ir.model.field.
This prevents potential privacy issues should the field be deleted or renamed.
closesodoo/odoo#39232
Taskid: 2088634
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
A patched method was not unpatched. Which triggered issues in a staging
master branch trying to untie a bit mail tests.
closesodoo/odoo#40099
X-original-commit: 5dbf4c56765c3661eb2dca24e90bc0b420918e9a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When loading a page on an existing starting database, registry is not
fully loaded causing potential error when trying to access model
existing in database (views, menitem, ...) since model added in last
loaded module does not exist in registry.
Thus, executing browser js test may lead to errors when executed during
an update on a database with other modules installed.
HTTPCase should be executed post_install to ensure that registry is
fully loaded to avoid this problem.
Since HTTPCase are slower than other test, it is also a good idea to
execute them at the end, in order to prioritize fast fail.
With this commit, a warning is isued if such a test class is tagged to
run at install time.
While at it, remove deprecated at_install and post_install helpers and
remove the deprecated phantom_js alias.
closesodoo/odoo#39462
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
New performances test looks non deterministic, sometimes break with
28 queries instead of 25. This commit temporary pump up query count
to avoid staging fails, further investigation is needed.
closesodoo/odoo#39132
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Currently there are some high-level tests that trigger composer creation,
call or send_mail action. However it is always a good idea to have some
performance tests only for some parts of mail application.
In this commit we also fix some naming and remove unnecessary variables
as a side dish.
closesodoo/odoo#38475
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Somehow followup of dbb6cfdbf8 . Purpose is
to use formataddr from tools instead of manually handcrafting the full
email address format. Indeed the tools one correctly encapsulates and
format name and email.
Task ID 2068986
Before this commit, when assigning a user onto a record,
through the field user_id, the email was sent
in the current user's language
After this commit, it is sent in the partner's language
OPW 2060898
closesodoo/odoo#38142
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
PURPOSE
In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.
SPECIFICATIONS
Reorganize tests, notably
* put composer with template tests in with composer tests;
* put post with template tests with post tests;
* put tracking multi company test with tracking tests;
LINKS
Task 1958697
PURPOSE
In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.
SPECIFICATIONS
Split cc tests to either discuss or mail gateway tests. It allows to remove an
unnecessary file.
LINKS
Task 1958697
PURPOSE
In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.
SPECIFICATIONS
Reorganize tests, notably
* put message_post related test in test_message_post;
* have only composer related test in test_message_compose(r);
* put alias tests in test_mail_gateway;
* move some discuss tests to post tests as they are linked to notification
details, not really Discuss features.
LINKS
Task 1958697
PURPOSE
In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.
SPECIFICATIONS
* put channel moderation tests in test_mail_channel;
* rename resend to message management;
* rename multi company test to redirect tests:
LINKS
Task 1958697
PURPOSE
In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.
SPECIFICATIONS
Quickly clean common class and lessen data creation by default. Offer some
tool methods, notably to create portal users used in some tests but not
all, or to create mail templates.
LINKS
Task 1958697
In resend wizard tests
In some cases tuple comparison of states seems to fail. Indeed order seems
to change in some cases depending on installation order. With this commit
we correctly check partner / notification status one by one, avoiding
to rely on order / set length that is not guaranteed.
In mail gateway tests
Add a specific order on name to be clearer about what we mean when fetching
users as a fallback, until better heuristic. Using name is sufficient as
we have no need to rely on complex partner display_name.
Task 2060220 (testing SMS integration)
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules: `email.(charset|header|mime|utils)`.
The new `email.message.EmailMessage` API exposes a much easier interface
to create multiparted emails [1], is capable of doing all the necessary
headers value conversion (RFCs [2045], [2047], [2049]) [2] and get/set
different flavor of payload (text/bytes) in a straightforward way [3].
All headers are structured in a way to support python native types, i.e.
the `Date` header supports `datetime.datetime` objects and automatically
performs the required formatting. The same goes for multi-valued headers
like the `To` header, one can directly set a python list of values, it
will be automatically be formatted according to the RFCs.
The dedicated encoding and decoding functions are no more needed thus
has been removed has part of the refactor. FTR, [RFC2231] is an update
of [RFC2047] and based on tests we've just conducted, both GMail and
Thunderbird now support RFC2231 encoding just fine in all headers:
attachments names, From, etc.
[1]: http://docs.python.org/3/library/email.message.html
[2]: http://docs.python.org/3/library/email.headerregistry.html
[3]: http://docs.python.org/3/library/email.contentmanager.html
[2045]: https://tools.ietf.org/html/rfc2045
[2047]: https://tools.ietf.org/html/rfc2047
[2049]: https://tools.ietf.org/html/rfc2049
[2231]: https://tools.ietf.org/html/rfc2231closesodoo/odoo#35929
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules. Among them the `email.utils.formataddr`
function, this function takes a `(name, email)` pair and returns a
string value suitable for From, To and Cc headers.
The stdlib function is capable of handling several character encoding
and two binary-to-ascii encoding. Odoo only uses `utf-8` which is
compatible with the base64 b2a encoding. The re-implementation of that
function has been simplified to only support base64-ed utf-8 and ascii.
[PEP 594]: https://www.python.org/dev/peps/pep-0594/\#email-lagacy-api
[PEP 594][1] is going to depreciate legacy `email.message.Message` API in
PY3.8 in favor `email.message.EmailMessage`.
All `email.message_from_x` constructor functions take an optional
`policy` argument. The policy can either be `policy.compat32` to
generate an `email.message.Message` instance, either be `policy.SMTP`
to generate an `email.message.EmailMessage` instance.
The [current][2] default policy is `policy.compat32` but will change in
a future version of python. It is recommended to always specify the
policy.
[1]: https://www.python.org/dev/peps/pep-0594/#email-legacy-api
[2] https://docs.python.org/3.7/library/email.parser.html#email.parser.BytesParser
PURPOSE
Currently mailgateway runner user is used to create most documents through
incoming emails. Purpose of this commit is to try to link document creation
and update to users, using the email to make the matching.
SPECIFICATIONS
New sudo implementation allow to bypass access rights while keeping effective
user as creator. Use this mechanism to create documents still using the
mailgateway runner user while linking creator to matching user.
Heuristic to use to match email_from and users
* if route is linked to an alias: check its owner record and check in its
followers;
* find users having this emails;
* if route is linked to an alias: use alias_user_id field;
* fallback on mailgateway user;
Other side effect is that creator and author of incoming email linked to the
new record will now better match the document creator when linked to an
existing user.
LINKS
Task 1919267
PR #30597
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This branch is the combination of several optimizations in the ORM:
* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;
* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;
* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);
* make method `modified` take advantage of inverse fields to inverse
dependencies;
* filter records by evaluating a domain on records in Python;
* a computed field with `readonly=False` behaves like a normal field
with an onchange method;
* computed fields are computed in superuser mode by default.
Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.
closesodoo/odoo#35659
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
Follow up of 61de1c263a
- Remove the dangerous **kw from the route in favor of specifying useful args.
- Use the existing method to link attachments to message instead of duplicating
the logic in an incomplete manner.
- Fix an issue where the actual email would not be sent if the message was empty
but an attachment was defined.
- Ensure the send attempt is done immediately instead of after the cr commit.
The goal is to show potential error messages correctly to the user with the
crash manager instead of a generic error page.
This also increases the send timeout and prevents the error in the first
place in most situations.
closesodoo/odoo#35263
Signed-off-by: Christophe Simonis <chs@odoo.com>
PURPOSE
Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.
SPECIFICATIONS
Purpose
* move bounce information detection in message parsing. It allows to have
this information available in various steps of routing instead of having
to manually re-compute them;
* handle bounce in specific methods allowing easy override;
* improve bounce management, notably when detecting a bounce not linked
to the bounce alias configuration;
* better integration with blacklist mechanism;
Specifications
* compute bounce information in ``_message_parse_extract_bounce``.It parses
bounce information and returns a dictionary allowing to update parsed email
values;
* remove override in mass_mailign that basically does what mail already
does;
* manage bounce in ``_routing_handle_bounce``;
* when detecting a bounce, correctly call the bounce management method on
all models inheriting from blacklist;
* correctly update bounce counter;
* bounced mailing traces and automatic blacklist in mass mailing should
be done in ``_routing_handle_bounce``;
* add some tests;
LINKS
Related to task 1893155
Linked to PR #33340
PURPOSE
Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.
SPECIFICATIONS
Parsing email values: make message_parse effectively prepare all required
values for later processing
* update message_parse to add some more data directly in parsed dictionary
and avoid having to parse value again in later processing and computation.
* notably recipients computation is now done directly in parsing. It
includes email check and reconstruction using tools methods. That way
routing just has to handle those values and not compute part of them;
* from and cc are correctly managed with parsed and cleaned (sanitized)
versions in dictionary;
* remove all code decoding message header and replace them by fetching
the parsed value instead;
* filter parsed values before calling message_post to ensure right values
are given to it instead of having to manage them in message_post. Indeed
caller has to give the right input;
* make parsing sub-methods (extract payload, post process payload) have
a dictionary as input and output to always manage a dictionary of values
through the parsing methods and update their namespace;
Routing: remove deprecated code parts
* remove use of email_references regex. Indeed emails are routed using their
messageId and not the regex anymore since several versions [1];
* clean some unnecessary variables;
* simplify variable type input, like message_parse accepts only valid
email.message instances and to avoid several decode / encode. Tests are
updated accordingly;
Update docstrings of those methods.
LINKS
Related to task 1893155
Linked to PR #33340
[1] See df037e2959 and 0028c42136
Thibault Delavallée