*: tools
Steps to reproduce the bug:
- Add an image on the website.
- Replace it by a "jpeg". Note that the mimetype of the image is
"image/webp" at the upload since [1].
- Add a shape on the image.
- Save and Edit.
-> If you check on the available "Format", the mimetype of the
"original" is "webp" but it should be "jpeg".
Before this commit, there were two types of mimetype data attribute:
- `mimetype`: the current mimetype of the image.
- `originalMimetype`: the mimetype of the image without a shape.
Before [1], it was also the mimetype of the original image. However,
since [1], the user has the possibility to change the mimetype of the
image so the "originalMimetype" attribute does not always refer to the
mimetype of the original image anymore.
To resolve the problem, another data attribute has to be introduced.
Here is a summary of the mimetype related attribute:
- `mimetype`: the current mimetype of an image.
- `originalMimetype`: the mimetype of the image before a shape has
been applied. It is needed when removing a shape to recover the correct
mimetype.
- `mimetypeBeforeConversion`: the mimetype of the original image. It is
needed in order to be able to change the format of an image and come
back to the original one.
In the case of an uploaded "jpeg" image on which a shape has been
applied, `mimetypeBeforeConversion` is "image/jpeg", `originalMimetype`
is "image/webp" (since [1]) and `mimetype` is "image/svg+xml".
The `loadImageInfo()` has been adapted to also handle the case of an
image that has already been loaded but that does not have the
`mimetypeBeforeConversion` attribute (for example all the images that
were uploaded on the website before this commit). In this case, the
mimetype attribute is kept and not set to the original one as the user
could have changed it.
[1]: https://github.com/odoo/odoo/commit/0449fe85cb0e1d639a4e1aeba26e90906f79254d
task-3449866
closesodoo/odoo#137424
X-original-commit: 730588b802506844e6ca54df312333bfd8df1d52
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
With input 'name email@domain.com' (missing chevrons allowing to clearly spot
the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
whole input is considered as being the email.
To improve the heuristic we can add a fallback by recalling 'getadresses'
on the input with spaces replaced by commas when it found only an email and
no name. The new email will be split into sub pairs allowing to find the real
email and various name parts, allowing to make a new name / email pair.
Emails should not contain spaces thus this is coherent with email formation.
This fallback actually comes from a specific code done in '_parse_partner_name'
of Partner model. Supporting it directly at tools level make the behavior
coherent for all models.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@18c71edf59
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
As of rfc5322 section 3.4.1 local-part is case-sensitive. However most main
providers do consider the local-part as case insensitive. With the introduction
of smtp-utf8 within odoo, this assumption is certain to fall short for
international emails. We now consider that
* if local part is ascii: normalize still 'lower' ;
* else: use as it, SMTP-UF8 is made for non-ascii local parts;
Concerning domain part of the address, as of v14 international domain (IDNA)
are handled fine. The domain is always lowercase, lowering it is fine as it
is probably an error. With the introduction of IDNA, there is an encoding
that allow non-ascii characters to be encoded to ascii ones, using 'idna.encode'.
Also remove usage of 'email_re' in mailing email check. It is too restrictive
compared to real formatting we support (or try to). Valid outgoing emails
were directly canceled, notably when containing unicode.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@3ce5fb3072
Part-of: odoo/odoo#134934
As it already normalizes returned emails some manual calls to 'email_normalize'
are not necessary. Some variable names are updated to be clearer about the
email being normalized.
Parsing contact name and email is also moved into a tool function to avoid
using a partner environment just for a tool parsing method.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@f7add44c28
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
When having multi-emails input in an email field, 'email_normalized' field is
currently 'False', as they expect the field to contain a single email. This
has several drawbacks
* searching partners or fetching information based on emails does not work as
most tool methods use 'email_normalized' which is False (see e.g.
'_message_partner_info_from_emails', '_mail_find_partner_from_emails'
or 'find_or_create');
* blacklist is not available as it is based on 'email_normalized';
* mass_mailing wrongly considers those emails as invalid and cancel their
mail and related trace, as it tries to skip sending emails to invalid
emails;
Be more defensive and use first found email in case of multi-emails field.
Other emails are ignored. It is already an improvement that does not break
flows in stable and allow more emails to be sent.
before
-> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
-> email_normalized: False
after
-> email: '"Raoul" <raoul1@raoul.fr>, raoul2@raoul.fr'
-> email_normalized: raoul1@raoul.fr
A side effect is that it helps finding back some partners, as indicated in
tests where less phantom partners are created. It also helps suggested
partners / emails flow in discuss.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@90218186c5
Part-of: odoo/odoo#134934
PURPOSE
Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.
SPECIFICATIONS
Main fix in this commit: fix multiple nested formatting in 'email_formatted'
computation for <res.partner>. Other use cases are mainly left untouched as
we let users deal with their input. In summary :
* double format: if email already holds a formatted email, we should not use
it to compute email_formatted, like
name: Name / email: 'Format' <email@domain.com>
-> before '"Name" <"Name" <email@domain.com>>"
-> after '"Name" <email@domain.com>''
* multi emails: sometimes this field is used to hold several addresses
like email1@domain.com, email2@domain.com. We currently let this value
globally untouched by extracting emails and joining them, as we do not
expect email_formatted to be a list of emails. Extractin emails allows
to filter out extra text stored in email field, like
name: Name / email: text, email1@domain.com, email2@domain.com
-> before: "Name" <text, email1@domain.com, email2@domain.com>
-> after: "Name" <email1@domain.com,email2@domain.com>
* invalid email: if something is wrong, better keep it in email_formatted
than harcoding "False". Indeed this eases management and understanding
of failures at mail.mail, mail.notification and mailing.trace level. This
behavior does not change as it was already implemented like that even if
not sure it was intended;
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@9175bbd8e2
Part-of: odoo/odoo#134934
Current behaviour:
When sending an email from Odoo,
can be seen in plaintext.
Steps to reproduce:
1. Install sale_management
2. Head over to Sales > Quotations
3. Create a new quotation
4. Enter a partner
5. Click on Send by email
6. [...] S00021 amounting in $ 12.00 [...]
Fix:
When parsing to plaintext, replacing
html character by unicode character
opw-3389602
closesodoo/odoo#132203
X-original-commit: 3e3f1e67dd5894a41c830344ed41c91a5e44159e
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Antoine Demany (ande) <ande@odoo.com>
This commit fixes the malformed comment that would sometimes comment out
the rest of the html resulting in an improper display.
this is due to the new html5 notation --!> not behing understood by
our parser.
this commit replaces any --!> into -->.
this commit also remove <!--> or <!--->
opw-2812488
closesodoo/odoo#126148
X-original-commit: 4fed0c36fedf271acc3520d8219763964753dabe
Signed-off-by: Vranckx Florian (flvr) <flvr@odoo.com>
Current behaviour:
If we add a blockquote in the `website_description` of a product on
the e-shop, we cannot checkout the product. Silent HTTP 400 error
code, due to an exception raised by
https://github.com/odoo/odoo/blob/bf772181933ce5334da35c8368455963b2478399/odoo/fields.py#L1987-L1993
Expected behaviour:
You should be able to checkout products even if they have blockquote
in their `website_description`.
Steps to reproduce:
- Install eCommerce, sale_quotation_builder (issue is present only
after installing sale_quotation_builder)
- On a product, with the website editor, add a `blockquote` to the
description of the product > Save
- In a private browser window, as public user, visit the product on
the e-shop and try to checkout with it.
- Observe there is no visible error, and we do not proceed in the
checkout process.
Reason for the problem:
The exception mentioned above is triggered when there is a
difference between the html content that is saved in the DB and after
sanitization, meaning that someone with escalated privilege saved
the HTML content by overriding the sanitization with
`sanitize_overridable`. In our use case the only diff is the
presence of the attribute `data-o-mail-quote-node` which is removed
after the sanitization.
Fix:
This issue can be resolved two ways:
1) Adding `data-o-mail-quote-node` to the list of save attributes,
meaning it will not be removed during the sanitization process.
Since this is an attribute that we add on `<blockquote>` nodes,
it can be considered safe, just like `data-o-mail-quote`.
2) Remove the attribute sanitization of the `website_description`,
just like it is done in the website_sale module.
Since the `website_description` and `quotation_description` are both
computed from one-another, they should have the same sanitization
level.
I am implementing both solutions, 1) because adding the attribute to
the safe list seems safe in general, and may prevent future
issues of this sort. 2) because it is the root cause of the issue,
since the bug is present only after installation of the
`sale_quotation_builder` module.
Affected versions:
- 16.0
- saas-16.1
- saas-16.2
- master
opw-3297237
closesodoo/odoo#122154
X-original-commit: 23022144cb1a338db05870b28f17360b92c46a9c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Piryns Victor (pivi) <pivi@odoo.com>
With the introduction of Knowledge Behavior Component, came a need to create
html nodes which would have limited interactions with the editor. i.e. an Odoo
view already has everything it needs to function properly, and when it is
inserted in the editor, any manipulation on the selection or on the style that
could be done with it should be prevented. Another example would be the
/template block (will be renamed /clipboard in the future) that has a
non-editable part (buttons which have a definite action in Odoo, and which
should not be interacted with) as well as an editable part inside of it).
To solve this use case, this commit proposes to mark specific html nodes with
a `data-oe-protected` attribute which could have one of three values:
- "true"
- Only mutations of type "attributes" can be registered on the node itself
which has the `data-oe-protected="true"` attribute
- Prevent mutations of children (and sub-children) from being registered by
the mutationObserver of the editor
- Prevent the selection handling when its anchor is inside a
`data-oe-protected="true"` element, even if it is `contenteditable="false"`
- Prevent the command hint
- Prevent the usage of the wysiwyg toolbar
- Prevent the dblClick tooltip
- Prevent the editor sanitization `Sanitize.js`
- "false"
- Designed to be contained inside a node with `data-oe-protected="true"`
- Re-enable all features disabled by a parent node with
`data-oe-protected="true" for the children of a node with
`data-oe-protected="false"
- ("")
- This is considered equivalent to have the `data-oe-protected` attribute
set to "true" (like other html attributes).
Another attribute is added: `data-oe-transient-content`, with the following
values:
- "true"
- Prevent the serialization of the children of the node, so they are not
shared during a collaboration.
- Transient nodes will be removed during `cleanForSave`, meaning that they
will never be part of the html_field value in the database
- ("")
- equivalent to "true"
The use case is an embedded view: there is a large quantity of nodes that
are not relevant to share nor to save, since it will be recreated with the
lastest data from the database, with the information relevant to the
currently active user each time it has to be rendered.
Note:
This commit does not handle the dynamic switch from a specific value for
`data-oe-protected` to another (i.e. switching from "false" to "" or "true").
This could cause a number of problems like:
- some mutations from when the value was "true" are not yet handled when the
switch (to "false") happens => those mutations will be registered as if they
were always under the "false" value, even though it is not the case.
- in collaborative, some nodes with oids that were not relevant (under the value
"true") won't necessarily have the same oids in between collaborators.
Therefore we cannot suddently listen to their mutations and expect the changes
to be shared by switching to "false".
In conclusion: the `data-oe-protected` attribute value should stay the same
during the entire edition.
Task-2821374
Part-of: odoo/odoo#104680
This commit add a mechanism to ensure that someone could never save
changes from an history that diverge (in case there is a partition in
the RTC network or a person A was disconnected while another person B
saved changes that were not transmitted to person A).
task-3002163
closesodoo/odoo#112099
Signed-off-by: David Monjoie (dmo) <dmo@odoo.com>
Before this change URLs would get their url-safe encoding replaced
with regular characters when cleaning up
html content.
This caused signed URLs that use some of those characters to become
invalid if the target did not accept equivalent characters.
Concrete example:
/markers=size%3Alittle%7CAddress
is not the same as
/markers=size%3Alittle|Address
for google maps signed static urls
Do not add special handling for MAKO patterns
as they are not used in templates anymore
task - 3079113
Part-of: odoo/odoo#107200
The fix#14569 was not complete. This fix solve the issue explained in
that PR.
The lxml cleaner transform any utf8 character `U+00A0` to a string
` `. This cause a comparison in the editor to be erroneous
inside `HtmlFieldWysiwygAdapterComponent.updateWidget` when comparing
a value coming from the server with the value coming from the editor.
task-3138358
closesodoo/odoo#110264
X-original-commit: 05e0f65c4120ec7e1420dacab8dc678b8a30f70a
Signed-off-by: David Monjoie (dmo) <dmo@odoo.com>
Change class="oe-blackbox" to data-oe-protected="true" (same purpose), because
the name was decided in a rush before 16.0
When the editor history management and sanitization (sanitize.js) handle nodes
of an embedded view, OWL can become corrupted, which will lead to a traceback.
This commit uses the attribute data-oe-protected="true" to prevent
historyRollbacks or sanitization, and ensure that protected elements children
are removed during the cleanForSave operation.
The collaborative DOMPurify sanitization is untouched, for security reasons. To
properly use `data-oe-protected` in collaborative mode, a more complex
mechanism should be put in place to ensure that the protected element children
are properly reconstructed for each client, and that it is not part of the
collaboration. (i.e.: embedded view in knowledge do not support the
collaborative mode currently).
This commit also blocks the selection handling in tables which would be in a
data-oe-protected="true" node.
Task-3012552
closesodoo/odoo#106255
X-original-commit: 394e4fe567bb2eecbd0865de82914aa7396fde63
Related: odoo/enterprise#34245
Signed-off-by: David Beguin (dbe) <dbe@odoo.com>
Signed-off-by: Abeloos Damien (abd) <abd@odoo.com>
Steps to reproduce:
- in any app log a note using full composer
- add styles to the text (underline, strike-through, italic)
Bug:
styles except bold are removed
Fix:
added missing styles to the whitlelist
opw-2956374
closesodoo/odoo#102056
X-original-commit: eb07ab104d5f47574e14db811108a7d0cbbbe4b2
Signed-off-by: David Monjoie (dmo) <dmo@odoo.com>
Purpose is to allow to have ``data-behavior-props`` and ``data-prop-name``
on HTML nodes and keep it even when HTML content is sanitized. Those attributes
will be used notably in Knowledge application to store some structured content
e.g. used to populate props.
Prepares Task-2796156
Prepares odoo/enterprise#29423
X-original-commit: 7ddb4685567c115ca4e14fa7e5f845b0cb33fd7c
Part-of: odoo/odoo#101694
Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Currently, the 'similar lead detection' mechanism only considers email,
contact name, and partner name while finding duplicate leads in CRM.
After this commit, it will also consider the mobile/phone number for
the same. Note that the mobile numbers and phone numbers both will be
matched with each other while finding duplicate leads.
Task-2817884
closesodoo/odoo#88372
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Prepare an informative and nice-looking preview from the main content of the
email body, avoiding buttons/images alt etc.
Links, images, tables are removed.
Whitespace is added after the preview to avoid including the full message in
the preview (with markup).
Tags and attributes are also added to increase compliance with HTML5 standard,
including accessibility.
One additional SQL request is required to fetch the preview sub-template.
Task-2413355
closesodoo/odoo#86266
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
Fix bugs in link replacement (html and text version)
1/ The replacement of urls in text (not html) was buggy when
* a string includes multiple urls, and
* `base_url` is one of them and not the first
* Also in SMS marketing when adding the sms `id`
When it came to base_url's turn, the replace function for the content would
replace the `base_url` part of a previously shortened url instead of the
distinct `base_url` link.
2/ Ampersand character
A) The ampersand character prevented replacement of an url inside a `Markup`.
Previous tests passed as the tracker was created but the url was not replaced.
B) The ampersand character was not recognized as part of an url for simple
strings.
3/ Replacement of already short links
A faulty logic made it possible to replace "/r/" urls when no "blacklist" was
passed.
Commit
* Ensures that only the base_url link would be replaced instead in this case.
* Adds support for urls with "&"
* Does prevent shortening short urls
* Improves link conversion performance (regex + avoid duplication when there
are several occurrences of the same url)
* Adds multiple unit tests
Task-2783844
closesodoo/odoo#91164
X-original-commit: 61026806d831b49a7cf5c44ffef3d762986781ce
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When the system broadcasts an email response to document followers,
if the config parameters `mail.force.smtp.from` or
`mail.dynamic.smtp.from` are defined, it will rewrite the `From`
address to avoid spoofing the sender's domain.
**NOTE**: As of 15.0, this is based on the `from_filter` setting on the
corresponding ir.mail_server, rather than the abovementioned config
parameters, but the rest of the discussion stands.
For example, if the `mail.catchall.domain` is set to `example.com` and
an email response comes from:
"John D" <john@doe.com>
it will rewrite it to:
"John D (john@doe.com)" <notifications@example.com>
This will make sure the system never sends outgoing email for an external
domain, as it has no authority for doing so, and that could
break mail filtering/authentication rules (SPF, DMARC, etc.)
During this "encapsulation rewrite step", both the original Sender name
and their email are preserved, and put into the quoted "name" field of
the rewritten address. It seems sensible to preserve as much information
as possible about the original sender.
Unfortunately, the inclusion of the Sender email in the final name makes
it appear to some inbox providers as if the message is trying to
deceptively impersonate another person (as many phishing schemes would).
As of November 2021 GMail at least does this, and will hide the name in
the UI when it happens. It will keep only the rewritten email, which is not
very useful in the case of a notification (even though it's more
technically correct, of course).
This patch removes the original email from the rewritten notification,
keeping only the name, considering that the email is not the most
important part, and it's better to have one of the two than none.
So after the patch, the rewritten address is now:
"John D" <notifications@example.com>
When there is no name in the original address, we keep only the local
part of the email, to avoid the same display issue. The recipient will
have to identify the sender based on the context / past messages.
closesodoo/odoo#81807
X-original-commit: 3c65ec5a8191a392980ceb0a8c584767eae405f1
Signed-off-by: Olivier Dony <odo@odoo.com>
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
fields) are out of scope for the lint but my editor catches
closesodoo/odoo#74604
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
Purpose
=======
This commit improves the UI of the mail group module (backend and portal
view), fix some bugs and clean the code.
Bug 1
=====
If a user click 2 times on a confirmation link, an error is raised.
Bug 2
=====
A bug about the website snippets has been fixed;
- Add the snippet to a group in which you are member
- Change the group of the snippet to a group in which you are not member
- The snippet still show "Unsubscribe"
Bug 3
=====
In the portal view, when messages are filtered by month, messages which
are created in the last day of the month appear also in the next month.
Specification
=============
UI
--
Show the Allow / Ban buttons whatever is the state of the message, so
the user can ban / whitelist a message even if there's no pending email.
When we receive an email, remove the "Mailing List" footer. This was a
big issue because when some users reply many times, the footer was
duplicated on each answer.
Improve the form view of the mail group:
- On non-moderated group, show all messages instead of only pending
message
- Show the guidelines stuff on non-moderated groups
Improve the portal view
- show the attachments under every message (not only on opened message)
- improve the template to confirm the subscription of the user
- improve the portal view for mobile device
- do not show rejected messages, even for admin
- do not show the button "X replies" when the mode is not thread
Technical
---------
Split the route "/groups/subscription" into "/group/subscribe" and
"/group/unsubscribe"
Correctly quote the parent email for the emails sent by web version of
Outlook (with the HTML id "divRplyFwdMsg").
ACLs
----
Now only administrators or responsible / moderators can write / unlink
the group.
Fix an ACL issue on the group member where the responsible of a
non-moderated group could not access the members of the group.
Members with same email
-----------------------
Improve the behavior when multiple members have the same email address
(this situation is possible because there's no unique constraint on the
email field of the <res.partner> model.) We now return the most
appropriate member for the given email address and partner.
When unsubscribing by clicking on the confirmation link that we received
by email, remove all members with the same email address.
Links
=====
Task-2599676
See odoo/odoo/pull/73640
closesodoo/odoo#73640
Related: odoo/upgrade#2774
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose is to better understand the use of email_re regex (find emails in a
text) and email_split_tuples (find name and email in a string used in email
headers or as email_from / email_to / email_cc).
Task ID-2377974
Community PR odoo/odoo#61467
This tool function is deprecated. As explained in docstring: "since OpenERP 6.1
please use ir.mail_server.send_email()". Seems we are allowed to finally remove
it.
Task ID-2377974
Community PR odoo/odoo#61467
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
It seems quite easy to have a void content in the currently new editor
that actually holds a lot of undesired information, notably a font
tag. This is annoying when having behavior based on an html field
being empty or not. In this commit we therefore improve definition
of an 'empty' html field.
Task ID-2532529
PR odoo/odoo#71793
*: tools
The ImageOptimizer option can now set shapes on images to use those
shapes as clip path and background. It uses a flexible path so that the
shape will always fit the image proportionally. We also added an html
file alongside a javascript file to semi-convert the shape from
illustrator into a usable shape for this usecase (the clip path must
have values between one and zero therefore we must do some computation
before the shape is ready to use).
Thanks to Samuel for this specific shape-converter tool and other
technical points about svgs and clip-paths.
Thanks to Mehdi for remaining development post-testing and post-reviews
and for the many fixes.
Thanks to Brieuc for the actual shape SVG files.
Part of https://github.com/odoo/odoo/pull/69179
task-2327045
closesodoo/odoo#69179
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: xO-Tx <mou@odoo.com>
Co-authored-by: Brieuc-brd <brd@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
- Change an image option in mass mailing editor (e.g. Quality)
- save
- edit -> The option can't get the new applied value.
The body_arch's field used in mass mailing editor is
sanitizing attributes and as a consequence, option related data
attrs are removed on save.
task-2327045
closesodoo/odoo#72311
X-original-commit: 7c5666611363a73bc6fef070565039cf065a0176
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Before this commit there were few cases in which
is_html_empty method was not working as our expectation.
In cases such as "<p class=""><br/></p>", "<p id=""><br/></p>"
and many other cases.
In this commit we improves regex of is_html_empty method.
Mow from this commit all kind of attributes will be consider
in this method.
In this commit we also have added is_html_empty in mail template, portal
values and in report values also for rendering templates.
Task id: 2499504
X-original-commit: fd0a05f2955b9f7e9ae7233afebfd6240c9244dd
In `prepend_html_content`, if body `html_body` and `html_content` are
`Markup` objects, the processing of `html_content` through `re.sub`
will strip away the `Markup` flag, leading to the later concatenation
of the body and content first escaping the content (to safely convert
it to a `Markup`) before performing the concatenation proper.
As a result, the `preview` of a `mailing.mailing` would be
double-escaped, and the markup hiding it would instead be printed as
part of the body.
Assume that the type of `html_content` is str-compatible, and
immediately rewrap the value after processing it through
`re.sub`. This should do nothing if `html_content` was an actual
`str`, and will re-flag it as a `Markup` if it was one.
Also change the finding of the insertion point to use `re.search`, I
don't understand why this uses `finditer`, it just makes the code more
complicated.
closesodoo/odoo#71090
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Make the email body take the entire space to avoid having some wasted space,
this will also make the user have more focus when designing an email.
Revamp the settings notebook page in order to give more clarity to the user,
and move some fields from the main form have been moved to this section to
have more space in the bottom for the email body.
In the mailing form, when the mailing is sent or is being sent, set fields
which are no longer useful for the user to change to readonly mode.
Add a wizard that enables the user to schedule a mailing, the schedule field is
still kept in the form for the user to be able to change the date when the
mailing is in the queue (if they want to send it sooner).
Display an action helper-style content when the email is empty, because,
currently, the user is left with a big white screen when the email has no
content which is not desirable.
Update the html_empty function to take into account style attributes to better
match the editor's void content.
Hide A/B testing fields from SMS mailing form view as these are not supported
for SMS marketing.
Task-2469409
closesodoo/odoo#68882
Ent-pr: https://github.com/odoo/odoo/pull/68882
Related: odoo/enterprise#18391
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
We want to be able to force the FROM headers when we sent email in
SMTP. So we can avoid the emails to be considered as spam.
Specifications
==============
This is done with 2 system parameters.
If the system parameter `mail.force.smtp.from` is set we encapsulate all
outgoing email from with the given value.
If the previous system parameter is not set and if both
`mail.dynamic.smtp.from` and `mail.catchall.domain` are set, we
encapsulate the FROM only if the domain of the email is not the same as
the domain of the catchall parameter.
Otherwise we do not encapsulate the email (same behavior as before this
commit).
Task 2367946
See odoo/odoo/pull/61853
closesodoo/odoo#70980
X-original-commit: 08a561505b92d23c4c4ec4094b0cee209ece8753
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
* The result of `plaintext2html` is fully controlled and
markup-safe (the first thing we do is escape the input).
* For `append_content_to_html`, we assume the inputs are HTML and the
output is thus always properly HTML.
Alternatively, we may want to `Markup("%s%s") % ...` and require the
inputs to be properly marked? That seems like a good idea.
* In `_replace_local_links`, applying re.sub will strip out the markup
mark, so store it and reapply it on output if necessary.
That one is a big gnarly, because if the input to ustr is
markup-safe bytes (e.g. qweb rendering output) then the output is a
Markup object, but if the input is str then the output is str, so we
need to check before and after unless... we update ustr to check for
subclasses instead of exact type?
* In `_prepend_preview` the issue is similar to that of
`append_content_to_html`, though in this case we should *not* trust
the input, so we can flag the "parent document" as Markup and format
the preview bit in.
* And since we're marking mail's jinja output as safe, do the same for
web and iot.
Sadly there doesn't seem to be any hook for doing that at the
environment level of jinja, so every `Template.render` site has to
be marked.
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
Scenario:
- add a file in mass mailing editor => an icon is added linking the file
- save
- edit => the icon is replaced by a generic icon
- save
- send mail => no icon is shown in email received
The system is using eg. `data-mimetype="pdf"` to show the icon, before
12.0 this worked but in saas-12.3 the system uses mailing.mailing
body_arch's field that is sanitizing attributes and remove it on save,
so the icon is only working in received email if you save one and only
one time after adding file inside the mass mailing.
opw-2474053 (ticket for similar issue in 14.0)
closesodoo/odoo#70057
X-original-commit: 7e14515b84d8081557517df7e9dc7d8c0d17f2be
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
The mail signature of an user did not inline the CSS like the body of a
message, so for example icons seemed to work but would not be seen
(unless the mail reader had the same font-awesome system).
With this change we inline the signature too.
On edition there was also an issue since the original icon was saved as
data-class which was stripped: so eg. when editing a mail.message with
working icon, the icons would disappear.
opw-2453912
closesodoo/odoo#66372
X-original-commit: 5e093b7a37984c8cc5aa73d1c0f7aa9c9345e6b1
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
In order to make a wow effect right after DB install we enrich the base
company based on the given email address or company website at install.
In order to achieve that goal we use the partner_autocomplete service from
IAP. It uses free credits that are offered to new clients on saas platform.
Only the fields that are not filled yet are enriched to avoid erasing user
entered data, except logo. Indeed as partner_autocomplete is probably installed
as a core app (mail -> iap -> partner_autocomplete auto install chain) it is
unlikely that people already updated their company logo.
We consider that having a call to IAP consuming a token is ok for a standard
use case, especially that
* if no iap service is configured call to IAP won't add much timing;
* on Odoo SaaS free credits are given and company is enriched;
* on custom SaaS with custom iap service, at db creation probably no
credits are given and call will simply give no results back;
We decided to call IAP asynchronously at client web load. Session combined to
a boolean field on company model allows to do this call only once per company.
Doing this allows to avoid adding yet another post init hook. It also eases
behavior tweak through inheritance.
This call is limited to admin for obvious security reasons as well as
performance reason (limiting calls to external providers). As call will be
done once per company generally admin is the first person to log into the
its newly created Odoo.
LINKS
Task ID-2322455
PR odoo/odoo#64600
PR odoo/upgrade#2086
Co-Authored-By: David Beguin <dbe@odoo.com>
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
What are the steps to reproduce your issue ?
1. Install website_blog
2. Go to website and add a new blog post
3. Remove the block content
4. Save and go to Blog page
What is currently happening ?
Crash:
Template fallback
An error occured while rendering the template website_blog.post_teaser
opw-241937
closesodoo/odoo#63844
X-original-commit: d9097c2b6244154ea64142b5e8fd31cfdd34e66e
Signed-off-by: Achraf <abz-odoo@users.noreply.github.com>
PURPOSE
When displaying an email in a list the mail client (gmail, outlook...) computes
a preview based on the content. This preview is generally not well computed as
it contains a lot of garbage and tags while it should contain only relevant
text.
SPECIFICATIONS
To build this preview, all mail clients read the content of the email.
The only way to be able to customize the preview is to add an invisible
HTML element at the beginning of the email with the wanted preview text.
We add at the end of the preview `‌` (zero-width non-joiner) to
fill the end of the preview in order to not have the beginning of the
mail at the end of the preview. It doesn't work with simple space as
the mail clients trim each HTML element content.
Task ID-2172125
PR #49886
Since a recent commit[1], we have a utility method in tools named
'is_html_empty' that checks whether the given html content is
void(containing only formatting tags) or not. However, the re from
this method does not consider the case of self closing tags, fox ex
`<br/>`. In such cases, the method returns Falsy value even if the
content is void.
This commit fixes the issue by considering self-closing void tags
in the regular expression.
commit[1] - https://github.com/odoo/odoo/commit/974f512f5f5d3b9f80a8c3fcde290e4f55cf1230
Task - 2267689
closesodoo/odoo#53167
X-original-commit: 0fd6c9388dc959a9b6b7828850683175edb05b9d
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Provide a follow-up/overview of the Mailing 24 hours after its been sent.
Specifications
==============
Send an email to the responsible of the mailing 24 hours the last email
of the mass mailing.
During the link trackers creation, extract the button label if exists
(so we can display them in the statistics email).
Technical remarks
=================
The statistics email is sent to the responsible of each mailing in the
CRON of mass mailing.
Task-2227411
closesodoo/odoo#49836
Related: odoo/upgrade#1094
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
We already have BDO tag as allowed tag in sanitized HTML that is
similar to BDI tag, so this commit also adds BDI in allowed tags.
BDI* tags make sense in email since email client can have different
orientation than Odoo (alternative is to use `<div dir="auto"></div>`).
*: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/bdi
opw-2256982
closes#52787closesodoo/odoo#52801
X-original-commit: 68edebb0d862acb3dc14f839aaef744306c5e269
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Some tags are not necessary in the HTML template and we can remove them to
have a proper HTML output.
Task 2171753
closesodoo/odoo#51757
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Louis (loco)