The aim of this commit is to prevent reconciliation with credit note to be triggered when a reversed move is reset to draft
Context: reconciliation between credit note and invoice (same for vendor bill)
Previous to this commit:
Post invoice
Create and post credit note
Reset to draft both the invoice and the credit note
Post (confirm) again the credit note
-> User error due to attempt of reconciliation between credit note and draft invoice
After this commit:
The credit note is posted but not reconcilied
task-3492197
closesodoo/odoo#134526
X-original-commit: b61395337ca139b6a663628c70c52f8cce87d76f
Signed-off-by: Laurent Smet (las) <las@odoo.com>
Steps to reproduce:
- Set "To Check" for a bill in the Bill list view
- go back to the dashboard
Issue:
There won't be the "To Check" shortcut as it was the case in 15.0
opw-3455414
closesodoo/odoo#134432
X-original-commit: d8498091bc6e2cd441f5e4387e3833d1a2a1abc0
Signed-off-by: William André (wan) <wan@odoo.com>
Signed-off-by: Yolann Sabaux (yosa) <yosa@odoo.com>
Step:
- Setup one app free trial
- Create user on a trial database
- Go to the signup generated (token valid)
- Signup/Reset password
Actual result:
Token is "false"
Expected result:
Token is False/Null/None
Cause by #113753
opw-3482042
opw-3484705
closesodoo/odoo#134137
X-original-commit: e668e15f7ebc4e82cd08270733fa9c68e9944149
Signed-off-by: Vranckx Florian (flvr) <flvr@odoo.com>
Signed-off-by: Gaetan Vanden Bergh (gavb) <gavb@odoo.com>
For model_terms translated fields if a translation for a lang(fr_FR) has never
been defined
before this commit
when update translations for both en_US and fr_FR, the new translation for fr_FR
cannot be saved.
after this commit
new translations can be correctly saved when en_US and fr_FR are updated at the
same time.
closesodoo/odoo#135103
X-original-commit: ef4b195eeac178a14eb47f4e6cc61ddc97f78866
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Chong Wang (cwg) <cwg@odoo.com>
Since 0eff29409d, the behavior was
unpredictable when clicking on multiple companies before a reload.
This simplifies the implementation and the behavior:
* the behavior is always the same even if there were already multiple
companies selected: if one wants to add new companies they should use
the check box instead.
* there is no more confusion due to the storage of the symetrical
difference when toggling multiple companies and their children before
the reload.
closesodoo/odoo#135086
X-original-commit: b07b30cd4b2a3e821a24761075ae0aecee332b14
Related: odoo/enterprise#47265
Signed-off-by: Bastien Fafchamps (bafa) <bafa@odoo.com>
Signed-off-by: William André (wan) <wan@odoo.com>
Since commit [1], which converted the wysiwyg to Owl, trying to
customize the Google Map API key causes a traceback.
Steps to reproduce:
- In edit mode, go to the "Theme" tab.
- Click on the Google Map "Custom Key" button.
=> A traceback appears.
It happens because when getting the parent with the `this.getParent()`
call, the parent is now a `WysiwygAdapterComponent` component and so,
calling the `trigger_up` function on it does not work anymore.
This commit triggers the `gmap_api_key_request` event in a way that is
compatible with the Owl component.
[1]: https://github.com/odoo/odoo/commit/76d4f9811b756f0e57b0a33dbba534c900c7fe15
opw-3489976
closesodoo/odoo#135069
X-original-commit: 355a593fe2dea05a96d0491d684722770533ab7d
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Purpose of this merge is to further cleanup and prepare tests for
multi-company enabled aliases, notably
* activate multi-company environment by default in all mail tests;
* avoid multiple init of gateway parameters;
* fix leftover of bad alias management;
Followup of odoo/odoo# 130768 and odoo/enterprise# 45204
Followup of Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Followup of Task-3453343 (Mail: Cleanup Alias Usage)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
closesodoo/odoo#135055
Related: odoo/enterprise#47243
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When inheriting from the mail test class gateway initialization is done
automatically. No need to call it again.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#135055
Have all mail tests be in a controlled multi-company environment by default.
Remove extra calls to '_activate_multi_company' as it is now part of the
base 'MailCommon' test class.
Task-3453577 (TestMail: Update Alias/Gateway tests for MC)
Prepares Task-36879 (Mail: Support MultiCompany Aliases)
Part-of: odoo/odoo#135055
Currently, when a journal is created during import, we just use the journal name as the code.
However, that gets truncated to 5 characters on create, which can conflict with existing journals.
The import should be as flexible as possible, as it's better to let users import and fix things
after the import rather than debug the import file.
- If there is a journal with a shortcode that matches the truncated name of the new journal,
just use the existing journal (enterprise PR)
- If the shortcode conflict arises within the imported file
(so the journals don't exist before the import), keep track of the codes in `vals_list`.
The `get_next_bank_cash_default_code` is modified to avoid duplicating code.
See also odoo/enterprise#46101closesodoo/odoo#135051
X-original-commit: 1161307dc59d0c98eb1352fd80d95da7dc98e700
Related: odoo/enterprise#47242
Signed-off-by: Laurent Smet (las) <las@odoo.com>
Signed-off-by: Aliya Tastemirova (alta) <alta@odoo.com>
Have a form view with an x2many list with handle field and a
default_order containing id (e.g. sequence,id), but id not being
defined in the list view. Before this commit, on an existing
record, it was impossible to resequence records of the x2many.
The reason is that after resequencing (drag&drop), the model tried
to sort the records, and since the "id" field wasn't in the view,
but was in the default_order, records were reloaded and the changes
(the new sequences) were lost.
The "id" case is special because the field doesn't need to be set
in the view, it is always known by the model. However, it is only
available in record.data if it is defined in the view. This commit
treats the "id" field individually in the sort algorithm.
Bug reported in the new relational model feedback pad.
closesodoo/odoo#135045
Signed-off-by: Francois Georis (fge) <fge@odoo.com>
Steps to Reproduce the Bug:
- Create a storable product "P1" with 2 Bills of Materials:
- BoM 1:
- Component: C1
- BoM 2:
- Component: C2
- Create a MO:
- Select BoM 1.
- Save.
- Select BoM 2 without saving. Result: The component C1 is deleted
and replaced by the component of BoM 2.
- Select BoM 1.
Problem:
The component of BoM 2 is not cleared. Because we check if the new BoM
is different from the original one, but since we didn't save the change
when selecting BoM 2, the `move_raw_ids` are not cleared.
Solution:
Clear the `move_raw_ids` if any move with `bom_line` is not linked to
the current BoM.
OPW-3473387
closesodoo/odoo#135044
X-original-commit: f874fd46fcf8c2668c6d2ab42224196868ec0f0a
Signed-off-by: William Henrotin (whe) <whe@odoo.com>
When stock is uninstalled, the sequences used in the warehouses for the
picking types are kept. However, since [1], picking types cannot be
duplicated anymore. When the module is re-installed, the picking types
are created with the same name, which is not allowed by [1].
[1]: https://github.com/odoo/odoo/commit/9034602a028fb22fbd139cf41a2f5c71d9d8d032
opw-3484352
closesodoo/odoo#135036
X-original-commit: 7b0f38fea8bc9a541972d29f289e6175ee15919b
Signed-off-by: Arnold Moyaux (arm) <arm@odoo.com>
Signed-off-by: Walravens Mathieu (wama) <wama@odoo.com>
Before this commit, if an onchange returned an error, the faulty
value was kept in the UI and there was no feedback given to the
user besides the error being displayed in a dialog. However, the
former value was still the one stored in the model, which produced
a discrepancy between model and UI.
This commit tackles the issue by marking the field as invalid in
this case (like when you input letters inside a numeric field).
That way, the value in the model is still different from the one
in the UI, but the field is stored in the list of invalid fields,
it is displayed in red, and the view can't be saved (it can be
discarded though).
Task~3498849 (master version, which differs from 16.0, see [1])
[1] odoo/odoo#134792closesodoo/odoo#135017
Signed-off-by: Pierre Rousseau (pro) <pro@odoo.com>
Following AVW's review, the following changes have been made to the QR
code page:
- Generic QR codes are only added when in QR menu mode.
- Generic QR codes now fit on a single page.
- There is now a sample URL in the description.
- UI change.
- Description change.
Other fix:
Before, fields in res_config_settings.py were not `related` to fields in
pos_config. These have now been changed to `related` fields.
The compute function managing these old fields has also been removed.
closesodoo/odoo#133137
Taskid: 3478173
Related: odoo/upgrade#5112
Signed-off-by: Robin Heinz (rhe) <rhe@odoo.com>
website_*
* = forum, profile, slides, slides_forum, slides_survey
Purpose
=======
Make it possible for users to have an overview and search in all forums. This includes searching for posts of other users, and their own upvoted posts, much like /slides/all.
Give users a better experience when browsing website profiles looking for courses, certifications, forum questions, ...
Specifications
=============
Add route for `/forum/all`
change templates in `website_forum` to show posts even if `forum` is not specified to shows results and search in all forums
Add search bars for Questions, Answers and Upvotes, small UI changes here and there.
Task-2676848
Co-authored by: Fabio Barbero <faba@odoo.com>
closesodoo/odoo#129123
Related: odoo/enterprise#47207
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
*= forum, profile, slides, slides_survey
Purpose
=======
Give users a better experience when browsing website profiles
looking for courses, certifications, forum questions, ...
Specifications
=============
Add search bars for Questions, Answers and Upvotes, small UI changes
here and there.
Task-2676848
Based on #79372
Co-authored by: Fabio Barbero <faba@odoo.com>
Part-of: odoo/odoo#129123
Purpose
=======
Make it possible for users to have an overview and search in all forums.
This includes searching for posts of other users, much like /slides/all.
Specifications
=============
- add routes for `/forum/all`
- change templates in `website_forum` to show posts even if `forum` is
not specified to shows results and search in all forums
Task-2676848
Based on #79372
Co-authored by: Fabio Barbero <faba@odoo.com>
Part-of: odoo/odoo#129123
before this commit, in the import translation
wizard, if user try to import a file
currently it will show any file types to
upload even though supported formats are
csv and po
after this commit, if user try to import a
file only csv and po files will be shown
closesodoo/odoo#128807
Signed-off-by: Raphael Collet <rco@odoo.com>
When selecting an attribute value in POS, what we store is the `description`,
which is a string that represents the selected attribute values.
Ex: selecting `Size: M` and `Material: Leather` will result in the description:
`(L, Leather)`.
This choice does not lead to a logical API for dealing with product attibutes.
In this pr, we add a new field that stores the selected `ids` of
`"product.template.attribute.value"` and remove the
`selected_attributes` `Json` field from the `pos_self_order`
override of the `pos.order.line` model.
The `attributeHelper` function from the `tour_utils.js` file
from `pos_self_order` is improved such that it can now handle
both checking if a certain attribute is selected and actually
selecting an attribute.
closesodoo/odoo#126398
Task: 3378533
Related: odoo/upgrade#4992
Signed-off-by: Joseph Caburnay (jcb) <jcb@odoo.com>
Previously, when searching for leads from Belgium and from the state of
California, in the US, only companies from California would be returned.
This was due to the way the query was built, it looked like this:
`country IN ('Belgium', 'United States') AND state = 'California'`
instead of :
`(country = 'United States' AND state = 'California') OR country =
'Belgium'`
closesodoo/odoo#71617
Task: 2543169
Signed-off-by: Florian Daloze (fda) <fda@odoo.com>
Co-authored-by: Louis Baudoux <lba@odoo.com>
When calling track_prepare, an important part of the logic is getting
description in fields_get. We actually don't need the description,
fields_get is mainly use here to check for groups, but the dictionnary
is immediately transformed to a set making values irrelevant.
The same optimisation is done in _message_track, usefull to avoid an
additionnal query in test_recurring_order_creation_perf, because of a
value not in cache.
closesodoo/odoo#134689
X-original-commit: a7e7f90531a21af99a4c95abb829f8f6de94f68a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
In this commit, we moved all features related to the PWA and the PWA
itself to the community.
This includes:
* PWA
* Web Push Notification
* VCARD
Note from original commits:
===========================
PWA (part 1)
------------
This commit adds a ServiceWorker to complement the WebManifest to
complete the setup of the backend as a Progressive Web App.
More precisely, it adds the route, registration and the most basic
ServiceWorker to allow the backend to be recognized as an installable
PWA.
References:
- https://web.dev/install-criteria/
- https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Installable_PWAs
- https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers
Task ID: 3063485
PWA (part 2)
------------
This commit adds a WebManifest as a first step toward setuping the
backend as a Progressive Web App.
In a nutshell:
- the web app's name is configurable through a config parameter
(available in the Settings, in debug); defaulting to "Odoo".
- the web app's icon has been revamped to accommodate the required sizes;
also its design matches the one from the Android app.
- "theme-color" is used to color part of the browser/system UI to match
Enterprise brand color; also supports the dark mode.
References:
- https://web.dev/learn/pwa/web-app-manifest/
- https://web.dev/install-criteria/
- https://developer.mozilla.org/en-US/docs/Web/Manifest
Task ID: 3063485
PWA shortcuts
-------------
The main goal of this commit is like we did inside the `Android Odoo
Mobile App`, allowing users to have some Odoo application shortcuts.
We added the following apps in the key `shortcuts` on `web.manifest` in
these orders: `Discuss`, `CRM`, `Project`, `To-Do` (old `Notes`).
Links:
- https://w3c.github.io/manifest/#shortcuts-member
- https://developer.mozilla.org/en-US/docs/Web/Manifest/shortcuts
Task ID: 3123607
Offline mode
------------
This commit introduces a way to notify the user that he's "offline"
(aka. cannot reach its Odoo server) and that Odoo doesn't work in a
graceful way in this circumstance.
To do so, the Service-Worker will return the response of the
´web/offline´ route, which is cached at its setup.
Note: this screen is only show when launched while "offline" and fails
to load the requested page. It does not "interrupt" the WebClient to
show this screen when the connection drops off (cf. not a replacement
for the existing notification).
Task ID: 3203639
WebPush
-------
WebPush allows sending data to the user browser/app(PWA) even when
tab/app is closed. Web push is a "constant" link between the
ServiceWorker of browser/app and a WebPush server.
Note that each browser has its own custom WebPush server.
e.g.:
Chrome: https://fcm.googleapis.com/
Firefox: https://updates.push.services.mozilla.com/
Safari: https://web.push.apple.com/
Edge: https://wns2-ln2p.notify.windows.com/
WebPush introduces some cryptographic notion to ensure some the
reliability of the data sent:
VAPID: "Voluntary Application Server Identification" is the standard
used to generate the public and the private to sign the message
between the browser and the WebPush server
JWT: "JSON Web Token" is the standard used to sign the payload to the
WebPush server
ECE: "Encrypted Content-Encoding" is the standard used by WebPush to
encrypt the data of the payload to avoid sending RAW data
outside trusted network.
Simplified steps how to WebPush works:
The Javascript code of a web page subscribes to the WebPush server
(using the VAPID key generated at mail_entreprise install).
The WebPush server replay with a subscription (and some other info
like the unique URL endpoint per subscription where to send a
notification)
The application (odoo-bin in our case) sends a post request to the
WebPush server using the specific URL endpoint of the user (using JWT
and ECE).
The WebPush server sends back to the browser the encrypted payload.
The browser decrypts the payload and sends it to the ServiceWorker
linked to the subscription.
Here is a Sequence diagram of all interactions to process a web push
notification.
In Odoo, we use WebPush to send Notification to the user.
This commit aims to have a parity with the Android/iOS Mobile App at
the notification level.
Notes:
There are some ways to encrypt (ECE) the message for WebPush:
AESGCM128: this is a draft
AESGCM: very well documented
AES128GCM: RFC8188 Standard encoding
We implement only the RFC one as it is the only one implemented in all
major updated browsers (Chrome, Firefox, Safari, Edge, ...)
You need to allow the desktop "Notification" and "Push" inside your
browser. For iOS Devices, it only works on iOS 16.4+ and it's requiring
Odoo to first be added to the Home Screen. It's delivered silently,
meaning no sound, vibration, haptics or screen wake.
Note:
Notifications are sent directly if there are less than five
notifications, otherwise we use a cron triggered immediately.
Also, we have changed the value of the "QueryCount" as mail_enterprise
executes a new query to search the devices associated with the partner.
We have added a "try/except" for any Exception before the
push_to_end_point method as we want to avoid blocking a normal flow
just for a not mandatory push notification if something happens during
the push to the endpoint.
See: odoo/enterprise@d0ae70103d
Links:
https://www.rfc-editor.org/rfc/rfc8030https://www.rfc-editor.org/rfc/rfc8188https://www.rfc-editor.org/rfc/rfc8291https://www.rfc-editor.org/rfc/rfc8292https://w3c.github.io/push-api/index.htmlhttps://autopush.readthedocs.io/en/latest/http.htmlhttps://web.dev/push-notifications-web-push-protocol/https://github.com/web-push-libs/encrypted-content-encodinghttps://github.com/web-push-libs/pywebpushhttps://github.com/web-push-libs/vapidhttps://caniuse.com/push-api
Task ID: 3123678
VCARD
-----
In the process of replacing the native methods exposed in the mobile
apps, this commit implements the download of a vCard containing a
partner's information.
By using this standard format, both regular web users and mobile ones
are now able to save the partner's details to use them with their usual
address book software.
On a mobile device, the actual import of those informations is delegated
to the operating system.
References:
- https://datatracker.ietf.org/doc/html/rfc6350
- https://en.wikipedia.org/wiki/VCard
- https://github.com/eventable/vobject#vcards
Task ID: 2583916
===========
End of note
===========
Task ID: 3478014
closesodoo/odoo#133560
Related: odoo/enterprise#46530
Related: odoo/upgrade#5086
Signed-off-by: Adrien Dieudonné (adr) <adr@odoo.com>
Co-authored-by: Romeo Fragomeli <rfr@odoo.com>
Co-authored-by: Romain Estievenart <res@odoo.com>
Co-authored-by: Pierre Paridans <app@odoo.com>
Description of the issue/feature this PR addresses:
Prior to this, when multiple currencies with the same symbol where to appear
in a same document or be sent from a country to another with the same currency
symbol, the only information the recipient had was
the currency symbol leading to lack of clarity.
Desired behavior after PR is merged:
This set of two PR has for objective to give the user the possibility to edit
the symbol curency that is displayed everywhere so that if he feels like the
documents and views are lacking clarity, he can improve it.
This was already an option before but only when the debug mode was active.
This commit has the purpose of moving the currency symbol option out of the debug mode section and put it in the right column of the currency form view.
Adding this, the "Symbol" option is moved out of the debug mode.
task-3484335
closesodoo/odoo#133554
Related: odoo/enterprise#46527
Signed-off-by: Quentin De Paoli <qdp@odoo.com>
- Menu stock move renamed into 'moves analysis'
- the `moves analysis` menu presents an open default pivot view with
predefined filters and group by that facilitate the visualization of
stock variations based on quantity
- Add some default filters too in the moves history pivot view
closesodoo/odoo#126981
Signed-off-by: Arnold Moyaux (arm) <arm@odoo.com>
Bug
===
When we enrich a partner we don't have access to, it will try to read
it, but because of the ACL if will crash. To fix that issue, we don't
return the partner if we can't access it and we show an error message
in the UI instead of letting it crash.
Task-3344536
closesodoo/odoo#135025
X-original-commit: a3dcae1eebe7920ae788d5ff6d12c45e19a9628d
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Stéphane Debauche (std) <std@odoo.com>
This commit prevent the livechat visitor from executing actions
on frontend messages (that are not known by the server).
Steps to reproduce the issue:
- Open a livechat with an operator
- A welcome message is displayed ("How can I help you?")
- Add a reaction to this message
- A crash occurs
closesodoo/odoo#134987
Signed-off-by: Alexandre Kühn (aku) <aku@odoo.com>
Before this commit, the push to talk key combination was wrongly
detected. Indeed, it was dependent on which key was pressed first.
Steps to reproduce:
- Register "ctrl + alt" as the push to talk key
- Press control then alt, push to talk does not work
- Press alt then control, push to talk works
- If you unpress the alt key, the push to talk will stop as expected
- If you unpress the control key, the push to talk will keep going
This commit fixes the issue.
closesodoo/odoo#134981
X-original-commit: 3e4ed2a43ad23116c095af67cb7e5736a9854820
Signed-off-by: Thanh Son Dodeur (tso) <tso@odoo.com>
Signed-off-by: Matthieu Stockbauer (tsm) <tsm@odoo.com>
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
RATIONALE
Two main use case of corner case usage of 'email' fields are tested in this PR
and their support is improved
* formatted emails: `"Full Name" <email@domain.com>` stored into the 'email'
field;
* multi emails: `email1@domain.com, email2@domain.com` stored into a single
'email' field;
IMPLICATION
Email field is generally managed as "containing a valid email". This means
it is sometimes used as it in 'formataddr' as well as to perform searches or
identification checks. Example of issue: partner 'Raoul' has a formatted email
like "Raoul" <raoul@raoul.fr>. Using 'formataddr' in email_from leads to
from: "Raoul" <"Raoul" <raoul@raoul.fr>>
-> which is incorrect (but often dynamically corrected by email servers);
Email field holding multi-emails are not normalized, as current normalize
is done only if the field holds a single email. It means
* no easy finding based on 'email_normalized', e.g. various tools like
'_mail_find_partner_from_emails' or 'find_or_create' do not find partners
based on this email;
* no exclusion list management;
* issue with formatting, like
to: "Raoul" <raoul@raoul.fr,raoul.other@raoul.fr>
-> which is incorrect (but often dynamically corrected by email servers);
USAGE: OUTGOING EMAILS
Those use cases currently generate faulty outgoing emails. This is valid for
recipients ('email_cc', 'email_to') as well as author ('email_from').
For formatted emails: `email_to` is formatted again based on name and email
which leads to sending emails to `"Full Name" <"Other"<email@domain.com>>`.
Note that multi emails without formatting may work as it leads to email_to
`"Full name" <email1@domain.com,email2@domain.com>`. Some outgoing email
servers correctly send multiple emails. It depends on their fault
tolerance.
USAGE: FIND BASED ON EMAIL (NORMALIZED)
When searching for partners (e.g. using '_mail_find_partner_from_emails' or
'find_or_create') normalized version of input is used.
In case of multi emails sanitize is 'False', as normalization expects a single
email in the field. Therefore no partner is found. In processes that do a
"search or create" (e.g. using a template on a record) this leads to creating
a new partner (or several partners in case of multi emails) each time.
USAGE: OTHER FLOWS
Other flows are build on top of '_mail_find_partner_from_emails' / 'create'
of outgoing emails and are impacted by formatted email / multi email usage.
Those include notably
* mass_mailing: '_message_get_default_recipients' should be defensive to
give correct values when creating mailing emails;
* mass_mailing: faulty emails is based on normalize and multi-emails are
considered as faulty and ignored;
* after post hook: '_message_post_after_hook' tries to link messages without
author (but email_from) with newly-created partners, when partners are
created from chatter. It is therefore impacted by those corner cases;
* marketing_automation: built on top of mass_mailing and suffers from the
same issues;
USAGE: UNICODE
Unicode in emails should be supported. 'formataddr' and IrMailServer notably
received fixes to support unicode. Some check performed on email addresses
fail when unicode is involved, which leads to some emails not being sent
while they could.
USAGE: WRONG EMAIL FORMATTING
With input 'name email@domain.com' (missing chevrons allowing to clearly spot
the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
whole input is considered as being the email.
To improve the heuristic we can add a fallback by recalling 'getadresses'
on the input with spaces replaced by commas when it found only an email and
no name. The new email will be split into sub pairs allowing to find the real
email and various name parts, allowing to make a new name / email pair.
Emails should not contain spaces thus this is coherent with email formation.
This fallback actually comes from a specific code done in '_parse_partner_name'
of Partner model. Supporting it directly at tools level make the behavior
coherent for all models.
SPECIFICATIONS
This PR contains first tests to cover current support of those use cases. Then
we gradually improve support of multi-emails and formatted emails in various
layers of mail code, from mail.mail to mass_mailing or email formatting.
See individual commit for more explanation, each of them solving a specific
issue / use case.
Task-2612945 (Mail: Defensive email formatting)
closesodoo/odoo#134934
Forward-port-of: odoo/odoo#134888
Forward-port-of: odoo/odoo#74474
Related: odoo/enterprise#47183
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Formatted emails using our own formataddr contains quotes to correctly
separate name from emails, like'"Name" <email@domain.com'. However
JS parse function does not correctly handle quotes, assuming everything
being left of opening chevron is the name.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@c5b39cfb64
Part-of: odoo/odoo#134934
With input 'name email@domain.com' (missing chevrons allowing to clearly spot
the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
whole input is considered as being the email.
To improve the heuristic we can add a fallback by recalling 'getadresses'
on the input with spaces replaced by commas when it found only an email and
no name. The new email will be split into sub pairs allowing to find the real
email and various name parts, allowing to make a new name / email pair.
Emails should not contain spaces thus this is coherent with email formation.
This fallback actually comes from a specific code done in '_parse_partner_name'
of Partner model. Supporting it directly at tools level make the behavior
coherent for all models.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@18c71edf59
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
As of rfc5322 section 3.4.1 local-part is case-sensitive. However most main
providers do consider the local-part as case insensitive. With the introduction
of smtp-utf8 within odoo, this assumption is certain to fall short for
international emails. We now consider that
* if local part is ascii: normalize still 'lower' ;
* else: use as it, SMTP-UF8 is made for non-ascii local parts;
Concerning domain part of the address, as of v14 international domain (IDNA)
are handled fine. The domain is always lowercase, lowering it is fine as it
is probably an error. With the introduction of IDNA, there is an encoding
that allow non-ascii characters to be encoded to ascii ones, using 'idna.encode'.
Also remove usage of 'email_re' in mailing email check. It is too restrictive
compared to real formatting we support (or try to). Valid outgoing emails
were directly canceled, notably when containing unicode.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@3ce5fb3072
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS: MAIL COMPOSER IN MAILING
When using the composer with a mailing, it currently skips recipients whose
email is a multi-email due to the strict usage of 'email_normalize'.
We can improve multi-email support by effectively checking for the first
email found, using the "less strict" mode of normalize. It means more emails
are detected as valid, and therefore sent.
Due to lower support of multi-emails when sending emails, this even allows
to send multiple emails as all emails are mailed.
SPECIFICATIONS: DEFAULT RECIPIENTS
Mailings are generally done using default recipients, aka using a model method
that returns the people to mail: customers ('partner_id'), customer emails
('email_from'), specific implementation, ...
This is implementation using '_message_get_default_recipients' that returns
'partner_ids', 'email_to' and 'email_cc' that are then used in the mail
composer to generate final recipients.
In this commit we better handle the content of email fields to avoid issues
with multi-emails. For that purpose we correctly split the content of those
fields. We now have several 'email_to' for records having multi-emails instead
of a single badly-formatted 'email_to'.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@4a0d87d44f
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
Post-message hook is used in various apps to link records to a newly created
created partner. This is the case notably when used with a template as it
creates partner on the fly based on emails to always handle partners.
We now check either the complete 'email', either the normalized version of
it to avoid comparison issues with multi emails and formatted emails. As
'email_normalized' now supports multi-emails by storing the first found one
it helps finding the partner.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@e5ad9e0371
Part-of: odoo/odoo#134934
As it already normalizes returned emails some manual calls to 'email_normalize'
are not necessary. Some variable names are updated to be clearer about the
email being normalized.
Parsing contact name and email is also moved into a tool function to avoid
using a partner environment just for a tool parsing method.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@f7add44c28
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
Tool method '_mail_find_partner_from_emails' that searches for partners based
on email is improved to support multi-emails input. Instead of skipping
multi-email (current behavior of 'email_normalize') we now consider first
found email in the input.
It is used notably to match incoming email / partners or suggest recipients
in Chatter.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@e0582b9e1f
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When having multi-emails input in an email field, 'email_normalized' field is
currently 'False', as they expect the field to contain a single email. This
has several drawbacks
* searching partners or fetching information based on emails does not work as
most tool methods use 'email_normalized' which is False (see e.g.
'_message_partner_info_from_emails', '_mail_find_partner_from_emails'
or 'find_or_create');
* blacklist is not available as it is based on 'email_normalized';
* mass_mailing wrongly considers those emails as invalid and cancel their
mail and related trace, as it tries to skip sending emails to invalid
emails;
Be more defensive and use first found email in case of multi-emails field.
Other emails are ignored. It is already an improvement that does not break
flows in stable and allow more emails to be sent.
before
-> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
-> email_normalized: False
after
-> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
-> email_normalized: raoul1@raoul.fr
A side effect is that it helps finding back some partners, as indicated in
tests where less phantom partners are created. It also helps suggested
partners / emails flow in discuss.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@90218186c5
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When building the final 'from' of outgoing emails using 'formataddr' we
have issues if email contains multi emails or formatted email. Having a
wrongly formatted email in 'email_from' leads to issues as it is badly
recognized by email providers, could be considered as being phishing and
also breaks reply_to mechanism.
Main fix of this commit is to extract emails and rebuild the 'email_from'
based on found emails. 'from' of sent emails is now the first found email
in 'email_from' field of related <mail.mail> record like
-> before: email_from: '"Raoul" <raoul@raoul.fr>, raoul2@raoul.fr'
-> after: email_from: '"Raoul" <raoul@raoul.fr>' and raoul2 is ignored
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1453db1a74
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When possible, use 'email_formatted' field on partner, instead of calling
'format_addr'. That way management of corner case input (multi emails and
double encapsulation) is managed by the computed field itself.
When 'format_addr' has to be used, ensure email part is normalized to avoid
formatting issues.
Also use tools to extract emails instead of 'email_re' regex when possible.
That way all code extracting and managing emails go through the same stack.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1c06014531
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When building the final 'email_to' of outgoing emails using 'formataddr' we
have issues if email contains multi emails or formatted email. Main fix of
this commit is to extract emails and rebuild the 'email_to' list based on
found emails.
E.g. partner Raoul - email: "Raoul" <raoul@raoul.fr>
-> before: to: "Raoul" <"Raoul" <raoul@raoul.fr>> (double format)
-> after: to: "Raoul" <raoul@raoul.fr>
E.g. partner Raoul - email: raoul1@raoul.fr, raoul2@raoul.fr
-> before: to: "Raoul" <raoul1@raoul.fr, raoul2@raoul.fr>
single email with multiple emails, depends on server fault tolerance)
-> after: to: "Raoul" <raoul1@raoul.fr>, "Raoul" <raoul2@raoul.fr>
multi emails
Fix that computation by using all normalized emails found in 'email' fields
and rebuilding a formatted email based on name + those emails. We do not
use `email_formatted` as it is not really multi-enabled. We prefer a local
defensive approach to be as tolerant as possible with respect to user inputs.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1c4b704149
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
Main fix in this commit: fix multiple nested formatting in 'email_formatted'
computation for <res.partner>. Other use cases are mainly left untouched as
we let users deal with their input. In summary :
* double format: if email already holds a formatted email, we should not use
it to compute email_formatted, like
name: Name / email: 'Format' <email@domain.com>
-> before '"Name" <"Name" <email@domain.com>>"
-> after '"Name" <email@domain.com>''
* multi emails: sometimes this field is used to hold several addresses
like email1@domain.com, email2@domain.com. We currently let this value
globally untouched by extracting emails and joining them, as we do not
expect email_formatted to be a list of emails. Extractin emails allows
to filter out extra text stored in email field, like
name: Name / email: text, email1@domain.com, email2@domain.com
-> before: "Name" <text, email1@domain.com, email2@domain.com>
-> after: "Name" <email1@domain.com,email2@domain.com>
* invalid email: if something is wrong, better keep it in email_formatted
than harcoding "False". Indeed this eases management and understanding
of failures at mail.mail, mail.notification and mailing.trace level. This
behavior does not change as it was already implemented like that even if
not sure it was intended;
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@9175bbd8e2
Part-of: odoo/odoo#134934
When having several possible emails to contact the first one is saved on the
trace model. Indeed a trace is normally linked to a unique recipient but
in case of multi-emails in an email field, several emails are available.
This commit ensure computation keeps the original mail_to when making emails
adresses uniques before sending mails and creating traces.
Followup of odoo/odoo@213d5d396c
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@b9b6fae9c5
Part-of: odoo/odoo#134934
This method is still not really tested, except its override of 'mail.thread.cc'.
So better have a test, especially in CRM that has some specific overrides.
Formatted email and multi-email input are tested, to check their current
support. Next commits will try to improve that, notably avoid formatting
issues.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@ce1fb51352
Part-of: odoo/odoo#134934
Move tests currently in mail about helpers defined in 'tools/mail.py' directly
into base. No need to test it in mail, there is no specific override or
behavior change compared to base.
Regroup partner related tests in base into 'test_res_partner'. This breaks
part of test history but it helps knowing which features are tested.
Task-2612945 (Mail: Defensive email formatting)
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
RATIONALE
Add tests related to not standard usage of email field. Two main use cases
are tested here
* formatted emails: `"Full Name" <email@domain.com>` stored into the 'email'
field;
* multi emails: `email1@domain.com, email2@domain.com` stored into a single
'email' field;
Additional tests: tests with unicode / ascii / case / wrong formatting are also
added to check the support in normalize and format methods.
IMPLICATION
Email field is generally managed as "containing a valid email". This means
it is sometimes used as it in 'formataddr' as well as to perform searches or
identification checks. Example of issue: partner 'Raoul' has a formatted email
like "Raoul" <raoul@raoul.fr>. Using 'formataddr' in email_from leads to
from: "Raoul" <"Raoul" <raoul@raoul.fr>>
-> which is incorrect (but often dynamically corrected by email servers);
Email field holding multi-emails are not normalized, as current normalize
is done only if the field holds a single email. It means
* no easy finding based on 'email_normalized', e.g. various tools like
'_mail_find_partner_from_emails' or 'find_or_create' do not find partners
based on this email;
* no exclusion list management;
* issue with formatting, like
to: "Raoul" <raoul@raoul.fr,raoul.other@raoul.fr>
-> which is incorrect (but often dynamically corrected by email servers);
USAGE: OUTGOING EMAILS
Those use cases currently generate faulty outgoing emails. This is valid for
recipients ('email_cc', 'email_to') as well as author ('email_from').
For formatted emails: `email_to` is formatted again based on name and email
which leads to sending emails to `"Full Name" <"Other"<email@domain.com>>`.
Note that multi emails without formatting may work as it leads to email_to
`"Full name" <email1@domain.com,email2@domain.com>`. Some outgoing email
servers correctly send multiple emails. It depends on their fault
tolerance.
USAGE: FIND BASED ON EMAIL (NORMALIZED)
When searching for partners (e.g. using '_mail_find_partner_from_emails' or
'find_or_create') normalized version of input is used.
In case of multi emails sanitize is 'False', as normalization expects a single
email in the field. Therefore no partner is found. In processes that do a
"search or create" (e.g. using a template on a record) this leads to creating
a new partner (or several partners in case of multi emails) each time.
USAGE: OTHER FLOWS
Other flows are build on top of '_mail_find_partner_from_emails' / 'create'
of outgoing emails and are impacted by formatted email / multi email usage.
Those include notably
* mass_mailing: '_message_get_default_recipients' should be defensive to
give correct values when creating mailing emails;
* mass_mailing: faulty emails is based on normalize and multi-emails are
considered as faulty and ignored;
* after post hook: '_message_post_after_hook' tries to link messages without
author (but email_from) with newly-created partners, when partners are
created from chatter. It is therefore impacted by those corner cases;
* marketing_automation: built on top of mass_mailing and suffers from the
same issues;
USAGE: UNICODE
Unicode in emails should be supported. 'formataddr' and IrMailServer notably
received fixes to support unicode. Some check performed on email addresses
fail when unicode is involved, which leads to some emails not being sent
while they could.
SPECIFICATIONS
Add tests related to those corner cases. Also add tests for computation of
`email_formatted` field of Partner model. It currently generates wrong email
values for the same corner cases (multi emails, formatted emails).
Tests are also added for the computation of `email_normalized` field used
notably for blacklists. It is not computed currently when being in multi
email mode which prevents from any blacklist mechanism as well as make
email finding harder. `_mail_find_partner_from_emails` tool method is also
tested with multi email as it uses the same heuristic as normalized email
field.
Tests are also added for mass mailing, when having to mail documents that
have a partner with formatted emails / multi-emails, or that have an email
field with formatted emails / multi-emails.
Also restore a test removed at odoo/odoo@afcb734908 while it should have been
updated to state that email addresses containing non-ascii characters are
supported.
Add some tests for tools methods used in various email processing flows.
Unicode tests are also added.
In future commits we will try to make email usage a bit more defensive to
try to lessen issues with that kind of use cases.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@fc8442f133
Part-of: odoo/odoo#134934
Add 'id' in partner ordering, to be sure searching on duplicated partners is
deterministic.
In mail, use standard ordering when searching on users to be deterministic.
As ordering on users is based on name then login which is unique, this is a
safe ordering and can be used as it.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1e6d50a141
Part-of: odoo/odoo#134934
MailTemplate model has a 'partner_to' field is dynamically rendered to contain
partners being recipients. After rendering it should hold a comma-separated
list of partner IDs.
However as we are unsure it was correctly written better be defensive. We now
check that we can effectively transform items to IDs using 'isdigit()'.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@a4e7f9b9dd
Part-of: odoo/odoo#134934
Sohail Jaidi (soja)