Some mail clients set a wrong content-type for PDF attachments, they
set `binary/octet-stream` which doesn't exist[^1]. Odoo would crash
with a traceback in case it tried to extract the content of such
malformed attachment because Python doesn't know how to decode such
content-types.
Looking at the actual content of the emails we received, we would had
expected the content-type to be `application/octet-stream` instead. It
is fair to assume that the mail client software is buggy or poorly
configured and that it assumes that `application/octet-stream` and
`binary/octet-stream` represent the same thing.
Following Postel's Law[^2], it is better to still handle those malformed
emails, assuming the content-type is `application/octet-stream`.
opw-3030113
opw-2716507
[^1]: https://www.rfc-editor.org/rfc/rfc2046#section-3
[^2]: https://en.wikipedia.org/wiki/Robustness_principleclosesodoo/odoo#107317
X-original-commit: e6169325b968711a0a28444818b34aa1932a6bae
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Co-authored-by: Julien Castiaux <juc@odoo.com>
Warn user and alias responsible when model creation fails on incoming
message due to alias mis-configuration.
Alias custom default values that references archived or deleted records can
prevent the record to be created when receiving an email.
Unfortunately, correcting values on the fly has too many downsides:
- as the value cannot come from nowhere, we would probably end up with
corrupted record like a task without a project
- as errors are silent, lots of suboptimal record could be created before it
is corrected
- as the code would have to deal with different kind of updates, it would be
heavily depend on the framework meaning additional maintenance cost
- the attempt made had also a performance cost by using savepoint which we
want to avoid
For all those reasons, we have decided to warm the user rather than trying to
solve the problem automatically.
The users are warned:
- through a bounce email sent to the sender and alias responsible
- in the alias interface through an alias status present in the list and the
form view
Technical notes:
- They are multiple kind of errors: error in the message (ex.: user not
authorized to send to a specific alias), error in the alias (ex.: dangling
reference in alias_defaults), other error (ex: technical error like a
service unavailable when receiving the message). Here, we want only to detect
alias error and skip other errors. So rather than doing a big try except around
_message_route_process, we have selected the spot where we should detect alias
error:
-- in _alias_get_error_message where we distinct a message error from an alias
error
-- in inside _message_route_process where record are created using
alias_defaults (custom values with reference to record that might have been
deleted since the alias creation)
- In test, we call mail.thread message_process with sudo as it is the case in
real use case (processed when fetching mail). It is needed as in some test
alias status are modified.
Task-2675209
closesodoo/odoo#101019
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The goal of this revision is to add a group
`mail.group_mail_notification_type_inbox` which is granted
automatically when the user `notification_type` is set to `Inbox`
to be able to easily hide the filter `Unread messages` when
the user uses the `Email` notification type,
by simply adding `groups="mail.group_mail_notification_type_inbox"`
on the filter rather than overriding `_get_view` in every model.
The code overriding `_get_view` of `project.task`
to hide the filter "Unread messages" replaced
in this revision initially comes from
odoo/odoo@da868d28e2
> - In project.task search View:
> - the filter for unread messages should be visible only if the current user
> managing his notifications in Odoo
The above revision was part of the task 2844212
closesodoo/odoo#104328
Related: odoo/enterprise#33338
Related: odoo/upgrade#3996
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Update to current runbot state, in order to better spot changes potentially
introduced with this PR.
Task-2710804 (Mail: Clean MailThread Posting API)
closesodoo/odoo#106182
Related: odoo/enterprise#34197
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose of this commit is to add tests on translations when using templates
and composer, on both email content (body, subject) and email layouting
content (view content, groups-based buttons, ...). This better highlights
some missing part of translation support, notably
* when being in mass mode: as content is taken from the composer and rendered
dynamically, it does not take the original translation from the template
even if it matches template content exactly;
* access button title is not translated as it is added during a separate
part of the process;
We also improve tests for ``mail.composer.mixin``. The wizard used in test
(mail.test.composer.mixin) is improve to behave more like a real invite
wizard: it runs on records (mail.test.composer.source) which are records
on which dynamic content is rendered. This highlights one issue with
translations: translations are fetched on current model (aka: composer
body and subject fields). However when using templates, translations are
stored on template model (aka: mail.template). This will be improved soon
by trying to fetch template translations when composer content matches
content from template.
Task-3046371 (Mail: Improve language propagation in mail and layouting)
Part-of: odoo/odoo#106072
Purpose is to try to have code easier to read and to update by having a
common way of sorting / writing code. Reorder some fields definitions and
dictionaries, update docstrings, ... in mail applications or when invoking
mail thread API.
Additional stuff worth noting here
* add some tagged in tests helping debugging / choosing tests to execute;
* in a test about mail generation with server action: correctly check
body_html field, not body which comes from the mail.message inheritance
(currently filled due to a side effect but actual field to check is the
html one);
* add same check on input for ``_render_template_qweb`` as done on other
rendering methods (even if rendering on [False] should be supported);
* move code translation update from specific tests into main test class
(code update due to new translations, followup of odoo/odoo@f8c2b02abe);
Some test linting done here
* reorder mail_render tests, remove some duplicated tests;
* reorder mail_template tests, remove some duplicated tests;
Task-2710804 (Mail: Clean MailThread Posting API)
closesodoo/odoo#106025
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Notify the user when added as follower by the mean chosen in his/her
preference (inbox/email) and not exclusively by email as before.
Task-2960281
Part-of: odoo/odoo#99371
If source is something like
"admin@éxample.com" <admin@éxample.com>
candidates founds are
['"admin@\xc3\xa9xample.com"', 'admin@\xc3\xa9xample.com']
and the first one raises an error because of "".
Malformed addresses should be ignored.
opw-2982426
closesodoo/odoo#102353
X-original-commit: 35ad2dd630a8ed173c6a9275585eac46b9a19363
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
Currently when creating a composer in mass mode
the composer always shows the 'mass mailing name' option
even when a mailing cannot be created.
This silently creates a regular mass mail.
This hides that option when selecting records of models
that do not inherit from thread. So that users do not falsly
believe a mailing will be created.
task #2990447closesodoo/odoo#102105
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Website: the context install_filename='dummy' is used to prevent
arch_updated from becoming True while updating translations of
ir_ui_view.arch_db (if arch_update becomes True, test_inherit_specific
fails)
Fuzzy search for jsonb translated fields has been adapted in the case of
website. It may require some refactoring later.
Purpose of this commit is to add some test about MailThread helpers built on
top of ``message_post`` / ``composer`` (log, post with view, ...). Some tests
about batch are also added.
Some performance tests are also added for those helpers.
Counters are updated. Note that they did not change, it is just an update
based on current master counters.
Task-2710804 (MailThread Api Cleaning)
closesodoo/odoo#100184
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This change allows for adding and removing followers from threads
by directly setting the message_partner_ids field of threads.
This enables the use of many2many_tags to add followers to a thread.
Task-2804888
closesodoo/odoo#95645
Related: odoo/enterprise#29284
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Make mail_notification_layout as the default mail template and remove
message_notification_email. We now only have 2 main notification mail layout
templates (outside of a light one): the default one and one that inherits from
it to only change the signature to the responsible instead of the author one.
In this operation the layout of message_notification_email has been integrated
in mail_notification_layout to support outlook mail client for which
message_notification_email has just been reworked (odoo/odoo#92847).
Technical note: we have integrated the "internal communication" banner from
"message_notification_email" into the new default template
"mail_notification_layout" which cause one added query in the test for querying
the flag internal from mail.message.subtype.
Task-2801600
Part-of: odoo/odoo#88466
"mail_notification_paynow" mail template layout has been renamed to
"mail_notification_responsible_signature_layout" as it now only differ by the
signature. It allows to use it more widely.
Task-2801600
Part-of: odoo/odoo#88466
Ease the understanding of the notifications sent when users are assigned to an
activity or a model (task, lead, ...).
Technical note:
- the customization of data sent to the email layout template (here subtitles)
should be done by overriding
mail_thread._notify_by_email_prepare_rendering_context on the model.
But in this case, we send an activity (ex.: todo) for a model (ex.: crm.lead)
which involves 2 models. Data from both models must be sent to the template
layout. To solve this problem an optional parameter to
mail_thread.message_notify has been added: subtitles. This allows the
caller which knows about the 2 models to set the values for subtitles for
the template.
- mail_activity._render_notify_header has been added to render the subject and
subtitles using the language of the recipients through the context lang
variable.
- a generic mail notification template has been added and the specific
mail_notification_paynow has been derived from it as the only change is the
handling of the signature.
Task-2801600
Part-of: odoo/odoo#88466
- Remove `_default_log_exceptions` of our `Cursor` class (unused except
in one test)
- Deprecated `serialized` args of `__init__` (Cursor class) and
`cursor()` (of Connection class), our cursor is always serialized.
- Remove `sql_log` attribute of Cursor class and replace it with
appropriate code to do the same stuff dynamically.
- Simplify some code
- Update some docstring
- Clean import
task-2766494
closesodoo/odoo#85078
Signed-off-by: Rémy Voet <ryv@odoo.com>
When creating records with default keys in context those are currently
propagated through the whole call chain until message creation. However
default keys are meant to be used for the record creation, not its logs
or messages.
A protection against default_child_ids already exists. However better
remove all context keys by using a clean context.
Closesodoo/odoo#93797closesodoo/odoo#99670
X-original-commit: 421ee12a099220cb43f3c3f0bf8d51586e687a28
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
*: im_livechat, website_livechat
Access right should be based on channel type and membership instead.
Chat always private, group always private, channel private should
disapear and be a group instead (migration needed), and other channel
always public (but they can still be further restricted with
the "allowed groups" feature)
task-2632861
closesodoo/odoo#90415
Related: odoo/enterprise#30980
Related: odoo/upgrade#3850
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Some differences between community / enterprise seems to have vanished.
Task-2710804 (Mail: Clean MailThread API)
closesodoo/odoo#99566
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Not entirely sure about TestAllocationRights. For TestEsEdiCommon
issue is quite obviously that it's inherited by tests which are
external, so when the `post_install_l10n` tag gets applied those tests
get run during "normal" l10n and they break.
closesodoo/odoo#98814
Related: odoo/enterprise#30825
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Update counters after all previous performance commits. It is done as a single
update due to frequent conflicts and difficulty to keep updated counters for
each commit separately.
Task-2883589 (Activity performance and cleaning)
Part-of: odoo/odoo#93682
Purpose
=======
Currently, only email sending can be scheduled with the `scheduled_date` field
defined on on <mail.mail>. It's not possible to delay the sending of
notifications.
We want to be able to delay the sending of the emails, but also the inbox
and bus bus notifications.
Technical
=========
For that purpose, we created a new model which stores the message we need to
notify with the scheduled date. When a scheduled_datetime is given we skip the
notification process. Instead an entry in that new model is created. A cron
regularly polls the scheduled message and launch the notification process on
messages that are ready to be sent.
Task-2207626 (Rating: Delay rating notification to ease feedback)
Part-of: odoo/odoo#95623
Co-authored-by: Thibault Delavallée <tde@odoo.com>
Purpose of this commit is to correctly generate and support scheduled date
defined on template when using ``send_mail`` tool that send emails directly
from a MailTemplate.
It uses the parsing tool method defined on mail.mail in order to have a
datetime localized in UTC then set timezone agnostic as expected by the
ORM.
Task-2826699 (Mail: use datetime for scheduled_date mail field instead of char)
Part-of: odoo/odoo#95623
Purpose
=======
Move the code which updates the mail message from the message model to mail
thread. Most other thread methods (like `_message_update_content_after_hook`)
are defined at record model level. It makes sense to delegate the update to
documents and not to the message. Message is a low-level technical object
that should not really hold business code.
While modifying this code, an update is done in the update content. We now
also allow to update the body without removing all attachments.
Finally tests are added as this feature was added without really testing
model code.
Task-2207626 (Rating: Delay rating notification to ease feedback)
Part-of: odoo/odoo#95623
Co-authored-by: Thibault Delavallée <tde@odoo.com>
Purpose
=======
Historically, we use a char field on the <mail.mail> to match the field type
on the <mail.template>. But even it's useful to have a char field on the mail
template (which can contains QWeb code or Jinja previously to Odoo 15.0) it is
not that useful for the <mail.mail> to keep this char type, because the value
is already rendered. Moreover this forces to have some code convert and store
datetime values using the standard format and in UTC.
We now correctly use a datetime field, as all other fields of that kind in
Odoo.
Task-2826699 (Mail: use datetime for scheduled_date mail field instead of char)
Prepares Task-2207626 (Rating: Delay rating notification to ease feedback)
Part-of: odoo/odoo#95623
Co-authored-by: Thibault Delavallée <tde@odoo.com>
Steps to reproduce:
1.) Create a custom email domain and incoming email server on a database, set the
Actions to Perform on Incoming Mails to Create a new record: Helpdesk Ticket.
2.) Set an email alias for a Helpdesk team, set the assignment method to
balanced/random, assign some users to the team.
3.) Set another email address to forward emails to the alias for the Helpdesk team.
4.) Emails received directly by the email alias will create tickets and assign
properly, emails that are forwarded to the email alias will fall back on assignment
defaults.
Explanation:
When we get the "Delivered-To" field for the message dictionnary we use
decode_message_header and the message.get_all() function, this function
returns a list with two addresses but it is transformed back into a string
in decode_message_header with a space as separator. This create an issue
when we use email_split_and_format on this string as it uses
email.utils.getaddresses that expects a list of headers field or a text
where addresses are separated with a comma instead of a string with the
header fields separated by " ". Because of that getaddresses fails to get
the right addresses and the recipients field of the message dictionnary is
missing the right address. Hence when we check if the alias is in this
values it does not find it and use the default fall back.
Solution:
To solve the issue we set the separator as a comma in decode_message_header.
opw-2917543
closesodoo/odoo#98761
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
* = calendar, im_livechat, rating, snailmail, test_discuss_full, test_mail,
website_livechat
Distinction between "replace" and "insert-and-replace" can be guessed based on
the type of the provided data.
task-2957295
closesodoo/odoo#98404
Related: odoo/enterprise#30580
Signed-off-by: Alexandre Kühn (aku) <aku@odoo.com>
PURPOSE
Have more reliable tests.
Better spot side effects coming from sub addons.
Lessen non deterministic counters due to local db.
SPECIFICATIONS
Make crm, event and mail performance tests post install.
Update query counters with
* local values (install module only with enterprise activated);
* community / enterprise runbots (if value is different);
* some notes on non deterministic issue if known;
Task-2925606
closesodoo/odoo#96446
Related: odoo/enterprise#29726
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Some fields in composer model allows to control generated messages or emails
values. However they are not available in the form view. In this commit we
add them in form view as invisible, as a first step towards cleaning the
composer code and usage.
Next step will be to cleanup composer code to remove the onchange based on
template and have real computed fields. Those will require fields to be
available in views so adding them is a necessary first step.
Some tests are added to see the usage and purpose of some fields. This helps
having a better code coverage.
Query counters update when using the Form tool
* adding subtype_id: 2 queries
* adding author_id: 1 query
Task-2816845
Prepares Task-2088884
Part-of: odoo/odoo#98287
When sending a mass mail through the composer, if the field ``reply_to`` had to
fall back to being ``email_from``, reply_to would take the value of the template
syntax instead of the rendered value.
This is notably the case when mass-mailing invoices through the accounting app.
Resulting in reply_to fields such as: '{{user.email}}'
On some mail clients (including mailhog), this could also result in template
syntax being shown as part of the subject or sender field.
This commit fixes that by correctly taking the rendered value of 'email_from'
Task-2816845
X-original-commit: e320b852e8f1958ac3dccaa5333420bb7b8d8f3d
Part-of: odoo/odoo#98287
FWD-PORT: updated to new test files, and tests are more in-depth since v14
When sending a mass mail through the composer, if the field ``reply_to`` had to
fall back to being ``email_from``, reply_to would take the value of the template
syntax instead of the rendered value.
This is notably the case when mass-mailing invoices through the accounting app.
Resulting in reply_to fields such as: '{{user.email}}'
On some mail clients (including mailhog), this could also result in template
syntax being shown as part of the subject or sender field.
Task-2816845
X-original-commit: b87df6664908615bdf57bf16e239a10c3f7bf89b
Part-of: odoo/odoo#98287
Currently only an email action based on template exists in server actions when
having mail app installed. It basically sends an email based on a mail template.
However being able to post a message on record is also useful. Instead of
sending emails it post on a document as a comment or as a note, like what
users can do using the chatter. Notification flow for those cases is the
classic from post: followers, specified partners, Inbox/Email, ...
Task-2613245 (Server actions mail update / cleaning)
Closes#45640
Part-of: odoo/odoo#75906
Current behavior
Starred messages counter takes into account the starred messages of a private
channels even if we no longer have access to this channel. Happens with deleted
messages too.
Steps channels
- Install Discuss
- Create a Private Channel and invite Marc Demo to join it
then send him a message
- *As Marc Demo*, star the message then leave the channel
-> Starred counter still shows 1
Steps for deleted messages
- Join the private channel again
- Delete the starred message (with Mitchell Admin)
-> Starred counter still shows 1 for Marc Demo
Reasons
Starred message count is computed by a raw sql [1] which only counts partner's
occurrences without taking into account the message's state and/or the
associated channel.
Side records (stars, notifications) are not removed when emptying a message
content [2].
With changes
- It doesn't take into account messages from a private channel which we no
longer have access to by doing a search on mail.message instead of doing it
in SQL (which filters out invisible messages);
- It correctly voids side records when emptying messages;
Side effects
This somehow raises number of queries because we now check access on messages
and records. However this is necessary as bypassing ACLs means unreachable
notifications or stars.
OPW-2742092
Task-2813738
[1] : https://github.com/odoo/odoo/blob/2c1c6b1373c238216fda1e2d9d2f00b5d16c8ca3/addons/mail/models/res_partner.py#L49-L51
[2] : 776d1ee08bclosesodoo/odoo#97974
X-original-commit: b1e8f7d97e3a1fa624436d2dd6801bf5fe7d19aa
Related: odoo/enterprise#30415
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The render API was confusing as mixing the access to the report and
the rendering env.
The ambiguity was present for code such as
`report.sudo()._render(record_ids)` where it was not clear if the
`sudo()` is needed to access to `report` or to `record_ids`. For low
priviledge users (such as portal or public), it was common to use
`report.with_user(SUPERUSER_ID)._render(record_ids)`.
This PR changes the render methods signature to be `api.model`. The
`report_ref` can be:
- ir.actions.report external id
- ir.actions.report id
- ir.actions.report recod
- `report_name` value
This will allow to call the report methods with any user and no longer
need to use `with_user(1)` to render reports as public user.
Task-id 2670865
closesodoo/odoo#91341
Related: odoo/upgrade#3650
Related: odoo/enterprise#27323
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
This removes an extra request, targets fetched messages more reliably and
simplifies the code.
task-2847909
closesodoo/odoo#96840
Signed-off-by: Alexandre Kühn (aku) <aku@odoo.com>
Purpose of this commit is to add tests on ``failure_reason`` usage of
notification model. Tests already exists for MailMail and other fields
but not that specific one. Followup of odoo/odoo@d895f3514e
New tests are added to check the usage of email_to and email_cc when
sending emails. Notably wrong usage of email_cc (lost formatting, copy
of all sent emails) is asserted to be fixed afterwards.
A performance test about batch sending is also added.
Prepares Task-2684479 (Mail: Better send error storage and display)
Part-of: odoo/odoo#96223
In the web client, in a real use case, it's not possible
to write on fields which are invisible,
as it's not possible to write on fields which are readonly.
This is a first step in the goal to change the behavior
of the `groups=` attribute in the back-end views,
to remove them for the view instead of making them invisible.
This is mainly to reduce the diff of the revision that will introduce
the mentioned above behavior change.
As nodes with `groups=` will be removed from the view
when the user doesn't have the group, it's no longer possible
to set a value on a field having a `groups=` the user doesn't have
in the `Form` test class, as the field will no longer be at all in the
view.
However, these unit tests shouldn't have been able to set values
on invisible fields in the first place.
This revision therefore aims to correct the unit tests setting value
on fields which were invisible because the user executing the
test was not part of the required group(s) for these fields
to be visible in the view.
closesodoo/odoo#94337
Related: odoo/enterprise#28936
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
``scheduled_date`` field of ``mail.mail`` is a char field since its addition
in 2015 (see odoo/odoo@364b4ba06d). As its first usage was in combination with
mail templates, a char field was used to simplify its implementation.
However this technically allows to store whatever value in that field. Using
it in a filter with a datetime argument is quite strange. A workaround is
to try to parse as much as possible the inputs, remove timezone information,
try to localize it, and have it in regular server format to enable filtering
on it.
We consider value should be set in UTC. If we have a specific timezone set
on the input we localize it to UTC. Otherwise we consider the input was done
in UTC, as all datetime fields. It is the role of the business code generating
mail.mail to either give the timezone, either already convert into UTC.
This might solve the following bug
Step to reproduce:
activate the developer mode
go to settings - technical - emails
create a new email and set the Scheduled Send Date in the future
run the scheduled action Mail: Email Queue Manager
Current behavior:
the email is sent and the action does not consider the filter
Expected behavior:
the email is not sent and will only be sent when the scheduler detects
that scheduled_date exceeds the current time.
Task-2833300
opw-2823106
closesodoo/odoo#94937
X-original-commit: 5c113cb9d54e051132a9a50deb2fcf61f9c9dc2c
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
TestMail holds a tool to create test records in batch, notably with inline
partner creation. Allow its usage in some sub modules and tests depending
directly from the base mail test class. This allows notably usage of this
tool in performance tests (see enterprise PR).
Also explicitly set some test data to ease sub tests writing.
Prepares Task-2150462 (Mass Mailing: Unsubscribe flow improvement)
Part-of: odoo/odoo#94660
Use dict.get() instead of a subscriptable call. This way we let through
selection values that are not loaded into the registry, instead
of raising an error.
This is especially useful in the upgrade environment
where such values may be unavailable (because of being
lambda-defined in a custom module for instance).
closesodoo/odoo#94530
X-original-commit: 36a6943740f9b4fbd9a78986bee4cd84bb8469c7
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
In message_notify, when called on a recordset, call model methods instead of
base one defined on MailThread. This allows to use internal methods overrides.
Also perform some linting on calls to ``message_notify`` in order to better
spot calls, parameters, ...
Task-2852908
closesodoo/odoo#92868
Related: odoo/enterprise#28038
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
