Commit Graph
273 Commits
Author SHA1 Message Date
Kevin Baptiste 2c6cd81f30 [IMP] mail: track fields based on ir_model_fields
The tracked field is now a relation to the corresponding ir.model.field.
This prevents potential privacy issues should the field be deleted or renamed.

closes odoo/odoo#39232

Taskid: 2088634
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2019-11-19 07:50:34 +00:00
Thibault Delavallée 9ad3564ac9 [FIX] test_mail: please bring back our send method !
A patched method was not unpatched. Which triggered issues in a staging
master branch trying to untie a bit mail tests.

closes odoo/odoo#40099

X-original-commit: 5dbf4c56765c3661eb2dca24e90bc0b420918e9a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2019-11-12 08:39:24 +00:00
Yannick Tivisse bee0991528 [IMP] test_mail: Adapt tests to work with/without demo data 2019-11-05 16:18:10 +01:00
Christophe Monniez 8d5da6e4be [IMP] tests: log a warning when HttpCase test in at_install
When loading a page on an existing starting database, registry is not
fully loaded causing potential error when trying to access model
existing in database (views, menitem, ...) since model added in last
loaded module does not exist in registry.

Thus, executing browser js test may lead to errors when executed during
an update on a database with other modules installed.

HTTPCase should be executed post_install to ensure that registry is
fully loaded to avoid this problem.

Since HTTPCase are slower than other test, it is also a good idea to
execute them at the end, in order to prioritize fast fail.

With this commit, a warning is isued if such a test class is tagged to
run at install time.

While at it, remove deprecated at_install and post_install helpers and
remove the deprecated phantom_js alias.

closes odoo/odoo#39462

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2019-10-30 12:35:23 +00:00
Xavier-Do 0c67122201 [FIX] test_mail: pump up non deterministic query count
New performances test looks non deterministic, sometimes break with
28 queries instead of 25. This commit temporary pump up query count
to avoid staging fails, further investigation is needed.

closes odoo/odoo#39132

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2019-10-21 15:09:41 +00:00
Thibault Delavallée 0d2f046851 [IMP] test_mail: add performance test for composer creation and send_mail
Currently there are some high-level tests that trigger composer creation,
call or send_mail action. However it is always a good idea to have some
performance tests only for some parts of mail application.

In this commit we also fix some naming and remove unnecessary variables
as a side dish.

closes odoo/odoo#38475

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2019-10-18 15:28:15 +00:00
Thibault Delavallée 2535a52177 [IMP] mail, portal: use formataddr to format email
Somehow followup of dbb6cfdbf8 . Purpose is
to use formataddr from tools instead of manually handcrafting the full
email address format. Indeed the tools one correctly encapsulates and
format name and email.

Task ID 2068986
2019-10-18 14:17:45 +00:00
Christophe Simonis d74b451805 [MERGE] forward port branch 13.0 up to f4105eb9c7 2019-10-09 02:08:17 +02:00
Raphael Collet f064437f06 [FIX] test_mail: query counts
closes odoo/odoo#38034

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-10-07 13:43:21 +00:00
fw-bot 2b830e5d41 [FIX] mail: translate assignation msg in recipient lang
Before this commit, when assigning a user onto a record,
through the field user_id, the email was sent
in the current user's language

After this commit, it is sent in the partner's language

OPW 2060898

closes odoo/odoo#38142

Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
2019-10-08 07:52:13 +00:00
Raphael Collet 338997d7a1 [FIX] test_mail: adapt query counts
closes odoo/odoo#37577

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-30 15:43:27 +00:00
Raphael Collet 439ebff0f5 [FIX] test_mail: wrong way to check many2many values 2019-09-30 15:43:27 +00:00
Martin Trigaux 35a9150d79 [FIX] *: add missing model description
Some models did not specified a _description
2019-09-24 07:33:09 +00:00
Sébastien Theys 35ccc8bb06 [IMP] test_mail: update query count to reflect improvements
Set threshold to actual values since these have not been updated after recent
improvements.

PR: #36654
Part of task-2070602
2019-09-12 08:45:49 +00:00
Thibault Delavallée 62bd4c07f4 [MOV] test_mail: move composer and post tests from template tests
PURPOSE

In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.

SPECIFICATIONS

Reorganize tests, notably

  * put composer with template tests in with composer tests;
  * put post with template tests with post tests;
  * put tracking multi company test with tracking tests;

LINKS

Task 1958697
2019-09-06 13:37:56 +00:00
Thibault Delavallée 6ccaae1b2f [MOV] test_mail: split cc tests to either discuss or mailgateway tests
PURPOSE

In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.

SPECIFICATIONS

Split cc tests to either discuss or mail gateway tests. It allows to remove an
unnecessary file.

LINKS

Task 1958697
2019-09-06 13:37:56 +00:00
Thibault Delavallée 8a8da21d2e [MOV] test_mail: split post and composer tests
PURPOSE

In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.

SPECIFICATIONS

Reorganize tests, notably

  * put message_post related test in test_message_post;
  * have only composer related test in test_message_compose(r);
  * put alias tests in test_mail_gateway;
  * move some discuss tests to post tests as they are linked to notification
    details, not really Discuss features.

LINKS

Task 1958697
2019-09-06 13:37:56 +00:00
Thibault Delavallée 803d87c46d [MOV] test_mail: rename some test files
PURPOSE

In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.

SPECIFICATIONS

  * put channel moderation tests in test_mail_channel;
  * rename resend to message management;
  * rename multi company test to redirect tests:

LINKS

Task 1958697
2019-09-06 13:37:56 +00:00
Thibault Delavallée 46241ce8a9 [REF] test_mail: lessen data created by default
PURPOSE

In order to rewrite some tests, remove low-level tests and add coverage
first step is to reorganize a bit test_mail content. Several tests cases
are spread among several files and finding back some feature coverage
tests is not easy.

SPECIFICATIONS

Quickly clean common class and lessen data creation by default. Offer some
tool methods, notably to create portal users used in some tests but not
all, or to create mail templates.

LINKS

Task 1958697
2019-09-06 13:27:59 +00:00
Thibault Delavallée 3d034dafc6 [FIX] test_mail: fix some ordering issues in tests
In resend wizard tests

In some cases tuple comparison of states seems to fail. Indeed order seems
to change in some cases depending on installation order. With this commit
we correctly check partner / notification status one by one, avoiding
to rely on order / set length that is not guaranteed.

In mail gateway tests

Add a specific order on name to be clearer about what we mean when fetching
users as a fallback, until better heuristic. Using name is sufficient as
we have no need to rely on complex partner display_name.

Task 2060220 (testing SMS integration)
2019-09-04 08:34:24 +00:00
Julien Castiaux 18299d7e50 [REF] base,mail,tools: use new python email API
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules: `email.(charset|header|mime|utils)`.

The new `email.message.EmailMessage` API exposes a much easier interface
to create multiparted emails [1], is capable of doing all the necessary
headers value conversion (RFCs [2045], [2047], [2049]) [2] and get/set
different flavor of payload (text/bytes) in a straightforward way [3].

All headers are structured in a way to support python native types, i.e.
the `Date` header supports `datetime.datetime` objects and automatically
performs the required formatting. The same goes for multi-valued headers
like the `To` header, one can directly set a python list of values, it
will be automatically be formatted according to the RFCs.

The dedicated encoding and decoding functions are no more needed thus
has been removed has part of the refactor. FTR, [RFC2231] is an update
of [RFC2047] and based on tests we've just conducted, both GMail and
Thunderbird now support RFC2231 encoding just fine in all headers:
attachments names, From, etc.

[1]: http://docs.python.org/3/library/email.message.html
[2]: http://docs.python.org/3/library/email.headerregistry.html
[3]: http://docs.python.org/3/library/email.contentmanager.html
[2045]: https://tools.ietf.org/html/rfc2045
[2047]: https://tools.ietf.org/html/rfc2047
[2049]: https://tools.ietf.org/html/rfc2049
[2231]: https://tools.ietf.org/html/rfc2231

closes odoo/odoo#35929

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-19 14:24:31 +00:00
Julien Castiaux dbb6cfdbf8 [REF] mail: remove deprecated email formataddr
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules. Among them the `email.utils.formataddr`
function, this function takes a `(name, email)` pair and returns a
string value suitable for From, To and Cc headers.

The stdlib function is capable of handling several character encoding
and two binary-to-ascii encoding. Odoo only uses `utf-8` which is
compatible with the base64 b2a encoding. The re-implementation of that
function has been simplified to only support base64-ed utf-8 and ascii.

[PEP 594]: https://www.python.org/dev/peps/pep-0594/\#email-lagacy-api
2019-09-17 11:32:54 +00:00
Julien Castiaux 1b057dd76b [REF] mail,test_mail: prefer EmailMessage over deprecated Message
[PEP 594][1] is going to depreciate legacy `email.message.Message` API in
PY3.8 in favor `email.message.EmailMessage`.

All `email.message_from_x` constructor functions take an optional
`policy` argument. The policy can either be `policy.compat32` to
generate an `email.message.Message` instance, either be `policy.SMTP`
to generate an `email.message.EmailMessage` instance.

The [current][2] default policy is `policy.compat32` but will change in
a future version of python. It is recommended to always specify the
policy.

[1]: https://www.python.org/dev/peps/pep-0594/#email-legacy-api
[2] https://docs.python.org/3.7/library/email.parser.html#email.parser.BytesParser
2019-09-17 11:20:13 +00:00
Julien Castiaux 4f03a5f136 [FIX] *: remove old deprecated modules/functions
PEP-594 is deprecating a bunch of modules. As part of the cleanup, we
are also dealing with long deprecated modules, functions and aliases.

* `assert_` -> `assertTrue`
* `assertEquals` -> `assertEqual`
* `assertNotEquals` -> `assertNotEqual`
* `assertAlmostEquals` -> `assertAlmostEqual`
* `assertRaisesRegexp` -> `assertRaisesRegex`
* `assertRegexpMatches` -> `assertRegex`
* `base64.encodestring` -> `base64.encodebytes`
* `base64.decodestring` -> `base64.decodebytes`
* `inspect.getargspec` -> `inspect.signature`
* `inspect.formatargspec` -> `inspect.signature`
* `logging.warn` -> `logging.warning`

closes odoo/odoo#36863

Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-17 11:36:42 +00:00
Christophe Simonis 6c948c36bb [MERGE] forward port branch saas-12.5 up to 197474e5b9 2019-09-13 18:07:59 +02:00
Shreya Modi af80c68ae5 [IMP] mail: improve create_uid heuristics in mail gateway
PURPOSE

Currently mailgateway runner user is used to create most documents through
incoming emails. Purpose of this commit is to try to link document creation
and update to users, using the email to make the matching.

SPECIFICATIONS

New sudo implementation allow to bypass access rights while keeping effective
user as creator. Use this mechanism to create documents still using the
mailgateway runner user while linking creator to matching user.

Heuristic to use to match email_from and users

* if route is linked to an alias: check its owner record and check in its
followers;
* find users having this emails;
* if route is linked to an alias: use alias_user_id field;
* fallback on mailgateway user;

Other side effect is that creator and author of incoming email linked to the
new record will now better match the document creator when linked to an
existing user.

LINKS

Task 1919267
PR #30597

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2019-08-21 13:28:03 +00:00
Raphael Collet 9920f20e4c [IMP] models: ORM speedup
This branch is the combination of several optimizations in the ORM:

* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;

* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;

* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);

* make method `modified` take advantage of inverse fields to inverse
dependencies;

* filter records by evaluating a domain on records in Python;

* a computed field with `readonly=False` behaves like a normal field
with an onchange method;

* computed fields are computed in superuser mode by default.

Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.

closes odoo/odoo#35659

Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
2019-08-20 12:43:59 +00:00
Christophe Simonis 140ee6b8f0 [MERGE] forward port branch saas-12.4 up to 98a55917a6 2019-08-14 16:48:10 +02:00
Christophe Simonis b792ba7d9a [MERGE] forward port branch saas-12.3 up to 752b2553c6 2019-08-08 11:52:13 +02:00
Christophe Simonis 7d04dc1c7c [MERGE] forward port branch saas-12.2 up to 9e74730a26 2019-08-06 18:48:24 +02:00
Christophe Simonis 0fadd586fa [MERGE] forward port branch 12.0 up to ff9ddfdacc 2019-08-06 15:57:57 +02:00
Christophe Simonis ff9ddfdacc [MERGE] forward port branch saas-11.3 up to 97d81f1220 2019-08-06 12:13:19 +02:00
Christophe Simonis 94aa43722e [FIX] test_mail: bump expected query count
324268a976 has the side effect of
generating an extra query.
2019-08-02 14:38:06 +02:00
Sébastien Theys 3620cb68a0 [FIX] mail, portal, account: clean up portal attachment
Follow up of 61de1c263a

- Remove the dangerous **kw from the route in favor of specifying useful args.

- Use the existing method to link attachments to message instead of duplicating
the logic in an incomplete manner.

- Fix an issue where the actual email would not be sent if the message was empty
but an attachment was defined.

- Ensure the send attempt is done immediately instead of after the cr commit.
The goal is to show potential error messages correctly to the user with the
crash manager instead of a generic error page.

This also increases the send timeout and prevents the error in the first
place in most situations.

closes odoo/odoo#35263

Signed-off-by: Christophe Simonis <chs@odoo.com>
2019-08-12 16:27:00 +00:00
David Beguin f4524f03c3 [REF] mail: move and improve bounce management in mail gateway
PURPOSE

Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.

SPECIFICATIONS

Purpose
  * move bounce information detection in message parsing. It allows to have
    this information available in various steps of routing instead of having
    to manually re-compute them;
  * handle bounce in specific methods allowing easy override;
  * improve bounce management, notably when detecting a bounce not linked
    to the bounce alias configuration;
  * better integration with blacklist mechanism;

Specifications
  * compute bounce information in ``_message_parse_extract_bounce``.It parses
    bounce information and returns a dictionary allowing to update parsed email
    values;
  * remove override in mass_mailign that basically does what mail already
    does;
  * manage bounce in ``_routing_handle_bounce``;
  * when detecting a bounce, correctly call the bounce management method on
    all models inheriting from blacklist;
  * correctly update bounce counter;
  * bounced mailing traces and automatic blacklist in mass mailing should
    be done in ``_routing_handle_bounce``;
  * add some tests;

LINKS

Related to task 1893155
Linked to PR #33340
2019-08-09 14:30:55 +00:00
Thibault Delavallée 7b79045fab [REF] mail: perform some code cleaning in mail gateway about parsing email values
PURPOSE

Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.

SPECIFICATIONS

Parsing email values: make message_parse effectively prepare all required
values for later processing

  * update message_parse to add some more data directly in parsed dictionary
    and avoid having to parse value again in later processing and computation.

    * notably recipients computation is now done directly in parsing. It
      includes email check and reconstruction using tools methods. That way
      routing just has to handle those values and not compute part of them;
    * from and cc are correctly managed with parsed and cleaned (sanitized)
      versions in dictionary;
    * remove all code decoding message header and replace them by fetching
      the parsed value instead;

  * filter parsed values before calling message_post to ensure right values
    are given to it instead of having to manage them in message_post. Indeed
    caller has to give the right input;
  * make parsing sub-methods (extract payload, post process payload) have
    a dictionary as input and output to always manage a dictionary of values
    through the parsing methods and update their namespace;

Routing: remove deprecated code parts

  * remove use of email_references regex. Indeed emails are routed using their
    messageId and not the regex anymore since several versions [1];
  * clean some unnecessary variables;
  * simplify variable type input, like message_parse accepts only valid
    email.message instances and to avoid several decode / encode. Tests are
    updated accordingly;

Update docstrings of those methods.

LINKS

Related to task 1893155
Linked to PR #33340

[1] See df037e2959 and 0028c42136
2019-08-09 14:30:55 +00:00
Xavier-Do bb37ab2119 [REF] mail, test_mail, *: rename needaction_partner_ids to notified_partner_ids
,*:crm, website_blog

The mail.message needaction_partner_ids meaning changed over time,
and doesn't actually reflect the list of partner in needaction.
More than that the message_format needaction_partner_ids used by js
began to diverged from mail.message field since 19fb650863

To avoid any confusion, renaming it to notified_partner_ids will help
to avoid confusion between python and js meaning, and reflect the actual
content of this field : a list of partner that where notified once on
this message. This field is mainly usefull for testing.

Task-ID 402597
2019-08-05 11:50:20 +00:00
Nikunj Ladava 0c37b9fb39 [IMP] mail, test_mail: new 'History' mailbox in Discuss
This commit adds a new mailbox in the Discuss app called 'History'.
Messages that have been marked as read are moved to 'History'.

Notifications are required to link history messages to users. So
notifications are now deleted after they are more tha 6 months old.
This means History mailbox keeps less than 6 months old messages.

Note that this feature only works when user notifications are handled
in Odoo. This can be set in the user preferences, under the
"Notification Management" section.

Task-ID 402597
2019-08-05 11:50:19 +00:00
Raphael Collet c65325b5c4 [FIX] mail: setting template to False in composer clears attachments
closes odoo/odoo#34104

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-06-13 15:27:18 +00:00
Christophe Simonis 618ea1ab67 [FIX] test_mail: bump query count
For unknown reason (I didn't manage to find the culprit), the previous
forward-port generate more queries than before.
2019-06-17 18:27:28 +02:00
Martin Trigaux 52936ec260 [I18N] *: export saas-12.3 source terms
closes odoo/odoo#34068

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2019-06-12 12:39:50 +00:00
Christophe Simonis 4f18c7c44b [FIX] test_mail: correct test
*/ Adapt test to new version
*/ do not use removed method `do_in_onchange`
*/ remove duplicated test
2019-06-06 13:33:16 +02:00
Christophe Simonis 71a50a2214 [MERGE] forward port branch saas-12.3 up to 409679866b 2019-06-06 11:54:35 +02:00
Christophe Simonis c8d7ad9a6b [MERGE] forward port branch saas-12.2 up to 995aa1685e 2019-06-03 15:49:31 +02:00
Christophe Simonis cfe0523714 [MERGE] forward port branch 12.0 up to 8f21148e1a 2019-05-31 14:37:38 +02:00
Thanh Dodeur c212cfe899 [REF] *: removes datas_fname from ir.attachment
This commit removes the field `datas_fname` from `ir.attachment` as
it was unnecessary and most of the time the duplicate of `name` or
`url`.

Task #1909865

closes odoo/odoo#32976

Signed-off-by: Martin Geubelle (mge) <mge@openerp.com>
2019-06-05 09:12:13 +00:00
Yannick Tivisse 1e77f2fde8 [FIX] mail: Fix /mail/view/ controller when not logged in
closes odoo/odoo#33756

Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2019-05-31 10:04:22 +00:00
Xavier-Do de7c119e66 [IMP] mail: reduce message_post query count and clean code
Purpose of this commit is to improve performance of post and notify through
various code improvements.

Containing
  * making of _notify_customize_recipients a mail.thread method;
  * _message_auto_subscribe_followers optimized;
  * renaming and optimisation of _notify_partners;
  * message_post_attachements optization;
  * remove unused mail_post_autofollow_partner_ids context key;
  * minimize message create queries;
  * only check record related access right for thread messages not in pending
    moderation;
  * get attachements from values instead of db;
  * storing lang in ctx from the beginning;
  * don't track on write on attachments;

Related to task 1943901
Linked to PR #32404
2019-05-29 13:34:32 +00:00
Xavier-Do 581a74143e [REF] mail: simplify notification code
Purpose of this commit is to clean notification process: calls, methods
API, method name, variable propagation.

Contains notably

  * simplify API of methods used to group recipients when sending notification
    emails;
  * improve and rename methods used in email notification process;
  * move some methods on model itself as non mail thread records could be
    mass-mailed and _notify_email_headers could be called on other records;

Related to task 1943901
Linked to PR #32404
2019-05-29 13:34:32 +00:00
Xavier-Do 7a6269719f [MOV] mail: move notification methods from mail.message / res.partner to mail.thread
Purpose of this commit is to move notification methods to mail.thread. Indeed
currently they are split among several models: thread, message, partner.
It makes code difficult to understand. Base record has to be propagated in
order to run methods on it. It is therefore simpler to make those methods
on mail.thread and correctly have model methods.

All notification methods are moved into a single section in the mail.thread
file. This implies some more code move inside mail.thread but allow to have
methods classified by their purpose.

Related to task 1943901
Linked to PR #32404
2019-05-29 13:34:32 +00:00