Commit Graph
41 Commits
Author SHA1 Message Date
Xavier ALT 99d4bc5e04 [FIX] mail: correctly parse body as html for pure-html email
Some external tools send email as pure html (no multipart) and when
parsing such email we ends up having the raw HTML as body (text)

This commit ensure we correctly parse and sanitize the body as HTML
for such emails.

closes odoo/odoo#132127

Task-id: 3451889
X-original-commit: a4763d214bfc7b09510226e552f749f9cd95870e
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Xavier Alt (xal) <xal@odoo.com>
2023-08-16 21:12:52 +02:00
Thibault Delavallée b5949d1672 [REF] test_mail, various: prepare gateway / alias tests
Rename alias domain and aliases used a test data. This allows to make
them easier to read, follow, grep and understand.

Activate multi-company on alias and gateway tests, ensuring it currently
has few impact on tests.

Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)

Part-of: odoo/odoo#130768
2023-08-03 21:56:56 +02:00
Julien Van Roy ca4ba84d98 [FIX] mail: UTF-8 text/xml attachment and omitted charset
When parsing an email containing an xml attachment, the `email` python
module will decode the base64 attachment using the charset or ascii if
the charset is missing.

In some cases, the payload is in UTF-8 but the charset is omitted. This
results in replacement characters for the non ASCII characters.

The solution is to force the charset to UTF-8, since it is a superset of
ASCII that should not be a problem.

NB1: Omitting the charset for text/xml is not recommended. See the RFC
(section 6.4): https://www.ietf.org/rfc/rfc2376.txt

opw-3144519

closes odoo/odoo#126474

X-original-commit: e3a5b46c7b4f5030e165a6d943ec2275aea5d583
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Julien Van Roy (juvr) <juvr@odoo.com>
2023-06-27 10:42:49 +02:00
Julien Van Roy e553f0b372 [FIX] mail,account_edi: fix creation of invoice upon email reception
When receiving an email on a mailbox with an alias that triggers the
creation of invoices, 4 bugs could occur.

1. If the xml received contains replacement characters (U+FFFD �), and
the charset of the part of the email is "US-ASCII" the encoding of the
string will fail, preventing the rest of the flow to be completed. Be
more resilient, encode the string and ignores these characters if this
case occurs.

NB: sometimes, the charset is omitted for a Content-type: text/xml. This
is valid but not recommended (see:
https://www.ietf.org/rfc/rfc2376.txt). In this case, the default used is
"US-ASCII". This means that any non-ascii char will be lost (they are
replaced by the replacement character: �, see:
https://github.com/python/cpython/blob/3.10/Lib/email/contentmanager.py#L67)
when decoding the attachment.

2. When the xml attachment is created in Odoo, the mimetype is
'text/plain' (rather than 'application/xml'). Thus, the
`_decode_attachment` needs to be more flexible when guessing the type of
the attachment (to know which function to use to read the content of the
attachment and create the invoice).

3. When creating an invoice from an email with an xml attachment, the
xml is attached as the `message_main_attachment_id`. It's only later on
that the content of the xml is read and we possibly find the PDF in
base64 inside. When creating the PDF attachment, it was not set as the
`message_main_attachment_id`, so the PDF was not rendered on the right
part of the invoice form view. Add a clause to replace the
`message_main_attachment_id` in such a case.

4. When the xml attachment represents a credit note, the move_type of
the invoice created by the email alias needs to be changed. Indeed, the
invoice is created before decoding the attachment, so we can only change
the `move_type` later.

opw-3144519
opw-3149649

closes odoo/odoo#121076

X-original-commit: 1e193a92b9c84e75b958985f8067873b90f686e0
Signed-off-by: Laurent Smet <las@odoo.com>
Signed-off-by: Julien Van Roy <juvr@odoo.com>
2023-05-11 08:28:42 +02:00
Nasreddin Boulif (bon) c78edec4d5 [FIX] mail_thread: save attachment from mail in same encoding
Steps to reproduce:

  - Configure incoming mail server and set it to create X record
    on incoming mails (X can be any model with a chatter)
  - Create a CSV file and set the encoding to UTF-16
  - Send the CSV file through Gmail to the Odoo instance
  - Go to model X and open the created record
  - In the chatter, click/download the CSV file
  - Open the downloaded file with Geany (or any file editor that can
    show the file encoding)

Issue:

  The file encoding is not the same as the original file (utf-8 instead
  of utf-16).
  Working with Outlook.

Cause:

  The difference between Outlook and Gmail is that Gmail provides the
  charset of the file.

  The content of the mail is retrieved using `email` python lib.
  The lib will try to retrieve the charset of the file and fallback
  on `ASCII` if not available, then return the decode content.

```python
  def get_text_content(msg, errors='replace'):
    content = msg.get_payload(decode=True)
    charset = msg.get_param('charset', 'ASCII')
    return content.decode(charset, errors=errors)
```

  Example:
  content = b'd\x00a\x00,\x00,\x00,\......'
  Outlook:
  charset = 'ASCII'
  return => 'd\x00a\x00,\x00,\x00...'
  Gmail:
  charset = 'UTF-16LE'
  return => 'da,,,,,\n,,,,,\....'

  In the post process of the attachment, the content is encoded in
  'utf-8' (to then encoded in base64) before creating the attachment
  record.

  Content encoded to 'utf-8':
  Outlook: b'd\x00a\x00,\x00,\x00...'
  Gmail:  b'da,,,,,\n,,,,,\n....'

  Therefore, when writing the file on the disk, the encoding is based
  on the binary content.

Solution:

  When parsing the mail, add the encoding charset to the `info` variable.
  Then, when creating the attachment, use the charset in `info` (or
  fallback on 'utf'8' if no charset set) to encode the content.

opw-3089009

closes odoo/odoo#118694

X-original-commit: 2d1b13e68d8bb204ecfe12a9d3db519c4264592c
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Nasreddin Boulif (bon) <bon@odoo.com>
2023-04-17 15:18:54 +02:00
Julien CastiauxandBaptiste Vergote 37ca7770a6 [FIX] mail: bad CTE/QP decoding of rfc822-headers
Based on RFC3462, a Content-Type text/rfc822-headers
exists and provide a mechanism to label and return
only the RFC 822 headers of a failed message (bounce)

These are only the headers and not the full message.

The Content-Type-Encoding should be either 7-bit(US-ASCII)
or Quoted-Printable (QP) as in the section 2 of the RFC.

Spawn the error:
After getting reported by a customer, I had to reproduce
by spamming wrong outlook addresses and add logging
in a sh database on the message_process of mail_thread.py
and logged the `message` variable.

After few retry, I got one of the part that was defined
as followed:
Content-Description: Undelivered Message Headers
Content-Type: text/rfc822-headers
Content-Transfer-Encoding: quoted-printable

The `get_payload()`function used was only assuming
that there is a full email on that part and that
it could only be encoded as an email, which was not
the case in this situation (quoted-printable:
76 characters per line, character `=` used
as the end of line character).

opw-3064589
task-3131561

closes odoo/odoo#110901

X-original-commit: f86f4b671696178f8fa42b81d8753d2591578431
Signed-off-by: Julien Castiaux <juc@odoo.com>
Co-authored-by: Baptiste Vergote <bve@odoo.com>
2023-01-24 23:21:57 +01:00
Thibault Delavallée 28b4ba4049 [IMP] mail, various: allow to link multiple reports to templates
RATIONALE

Improve usage of composer in comment or email mode: support batch-posting in
comment, support more configuration from templates, improve global model.

SPECIFICATIONS

Update report_template field on template model to be a many2many field instead
of a many2one. It allows to attach multiple dynamic reports to a given template
instead of being limited to a single one.

Name should now come from the report itself, which should be considered as
complete by itself. Template cannot override report naming anymore.

Task-2868153 (Mail: Allow multi reports in mail templates)

Part-of: odoo/odoo#99482
2023-01-17 20:58:40 +01:00
bve-odooandJulien Castiaux 48109c15d3 [FIX] mail: parse Content-Type binary/octet-stream
Some mail clients set a wrong content-type for PDF attachments, they
set `binary/octet-stream` which doesn't exist[^1]. Odoo would crash
with a traceback in case it tried to extract the content of such
malformed attachment because Python doesn't know how to decode such
content-types.

Looking at the actual content of the emails we received, we would had
expected the content-type to be `application/octet-stream` instead. It
is fair to assume that the mail client software is buggy or poorly
configured and that it assumes that `application/octet-stream` and
`binary/octet-stream` represent the same thing.

Following Postel's Law[^2], it is better to still handle those malformed
emails, assuming the content-type is `application/octet-stream`.

opw-3030113
opw-2716507

[^1]: https://www.rfc-editor.org/rfc/rfc2046#section-3
[^2]: https://en.wikipedia.org/wiki/Robustness_principle

closes odoo/odoo#107317

X-original-commit: e6169325b968711a0a28444818b34aa1932a6bae
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Co-authored-by: Julien Castiaux <juc@odoo.com>
2022-12-06 14:34:47 +01:00
Thibault Delavallée 94208cb8b4 [IMP] test_mail: improve coverage of post helpers
Purpose of this commit is to add some test about MailThread helpers built on
top of ``message_post`` / ``composer`` (log, post with view, ...). Some tests
about batch are also added.

Some performance tests are also added for those helpers.

Counters are updated. Note that they did not change, it is just an update
based on current master counters.

Task-2710804 (MailThread Api Cleaning)

closes odoo/odoo#100184

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2022-09-14 23:38:05 +02:00
mafo-odoo 6e9e7d32cc [FIX] mail, test_mail: decode_message_header and email_split_tuples format modification
Steps to reproduce:
1.) Create a custom email domain and incoming email server on a database, set the
Actions to Perform on Incoming Mails to Create a new record: Helpdesk Ticket.
2.) Set an email alias for a Helpdesk team, set the assignment method to
balanced/random, assign some users to the team.
3.) Set another email address to forward emails to the alias for the Helpdesk team.
4.) Emails received directly by the email alias will create tickets and assign
properly, emails that are forwarded to the email alias will fall back on assignment
defaults.

Explanation:
When we get the "Delivered-To" field for the message dictionnary we use
decode_message_header and the message.get_all() function, this function
returns a list with two addresses but it is transformed back into a string
in decode_message_header with a space as separator. This create an issue
when we use email_split_and_format on this string as it uses
email.utils.getaddresses that expects a list of headers field or a text
where addresses are separated with a comma instead of a string with the
header fields separated by " ". Because of that getaddresses fails to get
the right addresses and the recipients field of the message dictionnary is
missing the right address. Hence when we check if the alias is in this
values it does not find it and use the default fall back.

Solution:
To solve the issue we set the separator as a comma in decode_message_header.

opw-2917543

closes odoo/odoo#98761

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2022-08-24 19:54:02 +02:00
std-odoo 9c1cdd330e [IMP] mail: avoid email loops when Odoo reply automatically
Purpose
=======
In this commit, we will fix the loops that occur when you send an email
to an alias (to create a ticket e.g.). In that case Odoo can reply
"Your ticket has been created" and then the auto-replier of the user
can reply to this email and the loop occurs.

Specifications
==============
To solve this issues, we add 2 system parameters
- <mail.gateway.loop.minutes>, 120 minutes by default
- <mail.gateway.loop.threshold>, 20 by default

When an email is sent to an alias, we look on the last records created
<mail.gateway.loop.minutes> minutes ago. If we overcome the limit
<mail.gateway.loop.threshold> the email is ignored.

Alias creation detection
========================
To detect the number of records created by a specific email address we
use the `_primary_email` attribute set on the model.

Task-2294034

closes odoo/odoo#78597

Related: odoo/enterprise#21772
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2022-05-20 13:14:56 +02:00
Nicolas Bayet 4813f42997 [IMP] mail,*: replace jinja with qweb
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment

By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).

There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).

We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.

To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.

This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
  (for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
  in order to see only one at once
- a floating select input to switch visibility of a particular logical
  branching

Task-27033

X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
2021-09-28 23:42:54 +00:00
Thibault Delavallée c68f655782 [FIX] test_mail: fix activity tests
Fix some activity tests: use real test models, try to avoid date issues
by using freezegun.

Followup of odoo/odoo@a26e6e954c and odoo/odoo@06dee038dd .

Task-2654840

closes odoo/odoo#77120

X-original-commit: 2387c2ee6272a7c260386427c108208e747b819b
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-09-24 11:09:17 +00:00
Habib (ayh) 45ee18ce5b [IMP] mail: enable inheritance in the next activity values computation
values for next activities are currently prepared inside the `mail_activity._action_done()` method which makes inheritance very difficult. This commit delegates the value computation to a sub-method and thereby improves the extensibility of the module.

required to support changes to enterprise documents odoo/enterprise#20769
Task-2627837

closes odoo/odoo#77004

X-original-commit: a26e6e954c5e6c773f37e9239138a7b94ded821f
Related: odoo/enterprise#21079
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-09-22 20:38:59 +00:00
Jairo LlopisandThibault Delavallee 1efeffd297 [FW][IMP] test_mail: test models with type do not mess with attachment types
Purpose of this commit is to highlight an issue that may happens easily with
`crm` that is made generic here within `test_mail`.

`crm` alters the context when creating a new record adding in this case
`default_type` to it][1]. The returned record contains that altered context.
his results in other records created from it trying to assign that same default
value for `type`. This is a very common name for fields, and happens to exist
in `ir.attachment` too.

If you create an alias for incoming leads in your DB with default values
`{"type": "lead"}` (something very common) and then an email comes to that
alias that contains an inlined base64 image, the attachment creation process
would simply fail.

Obtained error is ``ValueError: Wrong value for ir.attachment.type: 'lead'`` .

[1]: https://github.com/odoo/odoo/blob/272602193f5647f7f2270ed6ec68777625a139dd/addons/crm/models/crm_lead.py#L310-L311

X-original-commit: 99434b2e8528c10fcc9cb6860765e0ddcaa364c8
Part-of: odoo/odoo#77005
Co-authored-by: Thibault Delavallee <tde@odoo.com>
2021-09-22 19:25:55 +00:00
Florent de LabarreandThibault Delavallee a8518ae75d [FIX] mail: handle parsing of void body in incoming emails
When body does not contain any tag or content parsing currently fails
with an ``lxml.etree.ParserError``. To avoid that we can improve condition
about void body: stripping void characters allows to avoid that traceback.

Task-2641572
PR odoo#76159
Closes odoo#75625

X-original-commit: 708fe3e74991c144a05c6bdcafa1931672e36ce9
Part-of: odoo/odoo#77005
Co-authored-by: Thibault Delavallee <tde@odoo.com>
2021-09-22 19:25:54 +00:00
Florent de LabarreandThibault Delavallee 1ca67905b4 [FIX] mail: handle wrong Final-Recipient header in incoming emails
Some emails are wrongly formatted mainly due to old servers. If Final-Recipient
header is void or wrongly encoded it currently crashes. This fix ensure there
is no crash, even if bounce detection could be incomplete.

Task-2641572
PR odoo/odoo#76159
Closes odoo/odoo#75618

X-original-commit: f437967a1fa4fca56c88e1cf79586805101ab712
Part-of: odoo/odoo#77005
Co-authored-by: Thibault Delavallee <tde@odoo.com>
2021-09-22 19:25:54 +00:00
Martin Trigaux d9e3aab69b [FIX] mail: convert res_model_id to selection
Following the removal of read access on ir.model (odoo/odoo#69120),
the mail.activity.type model was not accessible to non-admin users due
to the res_model_id many2one field.

Before this commit, a project user could not access the Activity Type
menu.

Convert it to a selection field with the selection values being
computed in sudo.

closes odoo/odoo#74981

Related: odoo/enterprise#20214
Related: odoo/upgrade#2734
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-08-23 06:56:00 +00:00
David Beguin 0ea27f430c [IMP] mail: add message_log with view helper
This commit allows to log a note and bypass notification process using a
template. The method _message_log_with_view has been added. This method
render the template using the given view and values (in kwargs) to make the
body before calling _message_log().

This process is done by the intermediary method _message_compose_with_view that
is now common to message_post_with_view and _message_log_with_view has it
follows the same template preparation process.

Task ID: 2459416
Linked ENT PR: odoo/enterprise#17553
2021-06-01 13:34:54 +00:00
Thibault Delavallée 1766e0147d [MOV] various: reorganize templates into their right files
Purpose is to have all mail template into a mail_template_data.xml file
when possible. It eases maintenance and update when having to work globally
on template records.

Also update some ``body_html`` declarations still using ``xml`` instead of
``html``.

Task ID-2534550 (Template usage improvement)
Task ID-2477164 (Composer mixin)
Prepares Task ID-27033 (QWeb in templates)
COM PR odoo/odoo#70889
ENT PR odoo/enterprise#18352
2021-06-01 09:19:45 +00:00
Jairo Llopis b8571322f9 [FIX] mail: add support for Text/RFC822-Headers content-type
WHY:

Some mail servers may provide `Message-Id` header in a section typed
`text/rfc822-headers` instead of a `message/rfc822`

According to https://tools.ietf.org/html/rfc6522#section-4 that section's body
actually contains the headers from the bounced email.

STEPS: a way to reproduce it should be using postfix v2.5+ with setting
bounce_size_limit=1. See
https://github.com/odoo/odoo/pull/42340#issuecomment-617605521

BEFORE: `bounced_message_id` is empty and thus the little email envelope icon
doesn't turn red, nor do the email resend features trigger in the UI. You could
be sending an invoice and never knowing it was bounced.

AFTER: bounces from mail servers that return the `text/rfc822-headers` part will
be handled properly.

---

@Tecnativa TT21170
opw-2162067
opw-2344252
closes #42340
closes #62551

closes odoo/odoo#62732

X-original-commit: 375ba5b37053a599922dd61c9e787553d6171291
Signed-off-by: Ivan Yelizariev // IEL <yelizariev@users.noreply.github.com>
2020-12-02 11:50:31 +00:00
Thibault Delavallée 20674c891d [IMP] test_mail: add new mail composer tests for its internals
Purpose is to have some low level tests for composer itself allowing to
ensure its behavior. We add tests notably about default values computation
in mass mailing mode as well as template_id change and values synchronization
in composer model.

In a near future composer will probably be improved (use computed fields,
rewrite part of its logic to improve performances). Having tests done before
those tasks ensure we notice every change that might happen.

Task ID-2390314
PR odoo/odoo#62061

X-original-commit: def98d2375a8a906092293f0aa1ce03bf948d2d3
2020-11-26 10:43:05 +00:00
Thibault Delavallée 7c3a0daaa1 [REF] test_mail: sun is shining, no more umbrellas
PURPOSE

Have a cleaner test_mail addons

SPECIFICATIONS

Umbrella -> Container, easier to understand that we target a ticket / project
like model using mail.test.ticket and mail.test.container .

LINKS

Prepares Task ID 2238597 (clean notification models)
Prepares Task ID 2083854 (improve mass mailing technical flows)
PR #49891
2020-04-22 08:40:03 +00:00
Thibault Delavallée ba068ee728 [REF] test_mail: rename test models to ease understanding
PURPOSE

Have a cleaner test_mail addons

SPECIFICATIONS

Keep only mail-related tests, move odoobot in test mail full, send "update
notification" tests in mail (specific to mail). Merge some test files to
lessen number of files, perform light file renaming.

Split test mail models file to prepare some cleaning in those models and tests.

LINKS

Prepares Task ID 2238597 (clean notification models)
Prepares Task ID 2083854 (improve mass mailing technical flows)
PR #49891
2020-04-22 08:39:46 +00:00
Victor Feyens d4e0fe018d [IMP] *: use ref= instead of eval="ref(' in xml field tags.
* Code cleanup
* Avoid a safe evaluation of the field value when loading those records.

closes odoo/odoo#44883

Related: odoo/enterprise#8283
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2020-02-11 13:07:06 +00:00
Yannick Tivisse bee0991528 [IMP] test_mail: Adapt tests to work with/without demo data 2019-11-05 16:18:10 +01:00
Julien Castiaux 18299d7e50 [REF] base,mail,tools: use new python email API
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules: `email.(charset|header|mime|utils)`.

The new `email.message.EmailMessage` API exposes a much easier interface
to create multiparted emails [1], is capable of doing all the necessary
headers value conversion (RFCs [2045], [2047], [2049]) [2] and get/set
different flavor of payload (text/bytes) in a straightforward way [3].

All headers are structured in a way to support python native types, i.e.
the `Date` header supports `datetime.datetime` objects and automatically
performs the required formatting. The same goes for multi-valued headers
like the `To` header, one can directly set a python list of values, it
will be automatically be formatted according to the RFCs.

The dedicated encoding and decoding functions are no more needed thus
has been removed has part of the refactor. FTR, [RFC2231] is an update
of [RFC2047] and based on tests we've just conducted, both GMail and
Thunderbird now support RFC2231 encoding just fine in all headers:
attachments names, From, etc.

[1]: http://docs.python.org/3/library/email.message.html
[2]: http://docs.python.org/3/library/email.headerregistry.html
[3]: http://docs.python.org/3/library/email.contentmanager.html
[2045]: https://tools.ietf.org/html/rfc2045
[2047]: https://tools.ietf.org/html/rfc2047
[2049]: https://tools.ietf.org/html/rfc2049
[2231]: https://tools.ietf.org/html/rfc2231

closes odoo/odoo#35929

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-19 14:24:31 +00:00
David Beguin f4524f03c3 [REF] mail: move and improve bounce management in mail gateway
PURPOSE

Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.

SPECIFICATIONS

Purpose
  * move bounce information detection in message parsing. It allows to have
    this information available in various steps of routing instead of having
    to manually re-compute them;
  * handle bounce in specific methods allowing easy override;
  * improve bounce management, notably when detecting a bounce not linked
    to the bounce alias configuration;
  * better integration with blacklist mechanism;

Specifications
  * compute bounce information in ``_message_parse_extract_bounce``.It parses
    bounce information and returns a dictionary allowing to update parsed email
    values;
  * remove override in mass_mailign that basically does what mail already
    does;
  * manage bounce in ``_routing_handle_bounce``;
  * when detecting a bounce, correctly call the bounce management method on
    all models inheriting from blacklist;
  * correctly update bounce counter;
  * bounced mailing traces and automatic blacklist in mass mailing should
    be done in ``_routing_handle_bounce``;
  * add some tests;

LINKS

Related to task 1893155
Linked to PR #33340
2019-08-09 14:30:55 +00:00
Thibault Delavallée e87189222d [IMP] test_mail: add tests for mail gateway and perform light cleaning
This commit add some tests related to the mail gateway: more bounce management
tests and some additional thread formation tests. Some test asserts about
bounce / blacklist management are commented as they are not completely working
currently. This will be improved in master soon.

Some cleaning in also done in all mail gateway tests. Notably some call to
tool methods are cleaned / simplified, duplicate tests are removed. Some
low-level checks are removed.

A new test model is added for mail gateway: mail.test.gateway. It is a
chatter model with blacklist enabled on it. It allows to tests the
various blacklist-related overrides and features as well as all basic
mail gateway features.

Sub-part of task 1853147 (pre-cleaning before implementing mail gateway
improvements)
Linked to PR #32974
2019-04-26 10:28:09 +00:00
XavierDo 92e9e84b63 [IMP] mail, *: don't track fields at create
*: project, crm, maintenance, helpdesk,

It is useless to track fields during create since they
have no initial value and future tracking message will
show changes on tracked field.

We can log a default creation message instead
(as it is now if there is no mail_create_nolog context key)

This change will implies
- less queries when creating record
- cleaner creation messages
- less occurence of mail_create_nolog ctx key

Removing tracking at create could break the creation subtypes
mechanism (example: following task creation subtype on project)
Instead of using _track_subtype to give a subtype at create,
a new _creation_subtype method can be override. If a creation
subtype is set on a specific modlel, creation messages will be
create by message_post instead of _message_log.

We also need to adapt the message_track_post_template in order to
keep this feature whithout tracking.

Task: #1916916

closes odoo/odoo#31945

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2019-03-29 15:28:43 +00:00
Christophe Simonis 644f0959d0 [MERGE] forward port branch saas-11.2 up to 1655202924 2018-05-17 14:23:37 +02:00
Christophe Simonis 1655202924 [MERGE] forward port branch 11.0 up to 59a8a1cd8e 2018-05-17 13:19:49 +02:00
Christophe Simonis bca769a1e8 [MERGE] forward port branch saas-11.2 up to e70e5b6cf3 2018-05-14 17:31:53 +02:00
Christophe Simonis e70e5b6cf3 [MERGE] forward port branch 11.0 up to 2787de8c68 2018-05-14 16:46:11 +02:00
Thibault Delavallée 60b81e00b2 [IMP] mail: add tool methods to manage activities directly from the mixin
This commit adds tools methods related to activities in the mail.activity
mixin. It gives to models inheriting from the activity mixin an easy-to-use
API to schedule, unlink or mark activities as done. Purpose of those methods
is to avoid having people manually managing activities in the code to hide
the technical details of the activities model, notably access rights
or activity types.

A field is added on activity model to indicate they have been generated
automatically. This way when rescheduling or unlinking based on some
specific activity types we do not change user-created activities.

Those methods include scheduling activities, changing their dates, marking
them as done or unlinking them. Future commits will use those methods
in various addons to automatically generate activities based on workflow
we want to implement.

This commit also adds tests for the newly added code. Future commits should
probably have a look at activity security and add some tests cases to check
it is correctly taken into account. It is considered a bit out of scope for
this task.

Thanks to @jem-odoo for its in-depth review of this commit. Well thanks for
other commits also.
2018-03-27 15:00:46 +02:00
Thibault Delavallée b971d9c4e4 [IMP] test_mail: add performance tests to cover more separate features
This commit is a manual forward-port of saas-11.2's 42659dc70c .

In test_mail there are performance tests involving several followers and give
counters for some heavy real-life-like use cases. Purpose of this commit is to
have more basic performance tests for main mail features, like simple post,
simple subscription of one follower. It allows to have an idea of the basic
cost of various features.

Adding use cases for message_post includes
 * posting without followers (aka, no notification)
 * posting with a ping (by email or by inbox)
 * logging a note with optimized method _message_log or with message_post

Adding use cases for activities includes setting an activity as done. This
action posts a message and is therefore interesting to evaluate.

Adding use cases for subscription includes
 * adding and re-adding one follower, with default or specified subtypes
 * updating responsible field triggering a simple tracking and an assignation
   email or notification (not completely supported in saas 11.2 meaning this
   counter will increase when forward-ported)
 * note that some subscription tests have already been added at a01933c9b4

Finally some heavier tests are added for assignation and tracking based
on QWeb view.

This commit is related to task ID 1824965 . This one is an ongoing task
and several commits may be linked to that task.
2018-03-09 12:53:49 +01:00
Thibault Delavallée 42659dc70c [IMP] test_mail: add performance tests to cover more separate features
In test_mail there are performance tests involving several followers and give
counters for some heavy real-life-like use cases. Purpose of this commit is to
have more basic performance tests for main mail features, like simple post,
simple subscription of one follower. It allows to have an idea of the basic
cost of various features.

Adding use cases for message_post includes
 * posting without followers (aka, no notification)
 * posting with a ping (by email or by inbox)
 * logging a note with optimized method _message_log or with message_post

Adding use cases for activities includes setting an activity as done. This
action posts a message and is therefore interesting to evaluate.

Adding use cases for subscription includes
 * adding and re-adding one follower, with default or specified subtypes
 * updating responsible field triggering a simple tracking and an assignation
   email or notification (not completely supported in saas 11.2 meaning this
   counter will increase when forward-ported)
 * note that some subscription tests have already been added at a01933c9b4

Finally some heavier tests are added for assignation and tracking based
on QWeb view.

This commit is related to task ID 1824965 . This one is an ongoing task
and several commits may be linked to that task.
2018-03-09 12:34:21 +01:00
Thibault Delavallée cde06d5f5c [IMP] test_mail: move some data into demo and update tests accordingly
Creating data directly in tests was done when tests were located in mail
module to avoid creating real data or demo. Now that mail tests have their
own module we can create demo data and use them in tests. It is simpler to
have demo data for things like subtype and email templates to ease
understanding and reuse.
2017-12-12 15:46:37 +01:00
Christophe Simonis 2f99470458 [MERGE] forward port branch 11.0 up to c201cf2b77 2017-12-01 12:55:02 +01:00
Thibault Delavallée d040bfde89 [MOV] (test_)mail: move tests into their dedicated module
Purpose: avoid having useless tables for test models in a
production environment.
2017-11-29 10:11:15 +01:00
Thibault Delavallée 1cc1ef5058 [SPLIT] test_performance: split between base and mail performance tests
Test performance will now hold the base class for performance tests as
well as tests related to the ORM, depending only on base. A new module
test_mail is introduced at this commit that contains performance tests
related to mail module. This commit contains only code move and should
not impact anything.

Future commits will move mail tests into test_mail so that all mail
related tests are located in the same optional module. This allows
notably to avoid creating a lot of unnecessary tables when installing
mail module on production databases.
2017-11-29 10:05:01 +01:00