Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
Purpose
=======
This commit improves the UI of the mail group module (backend and portal
view), fix some bugs and clean the code.
Bug 1
=====
If a user click 2 times on a confirmation link, an error is raised.
Bug 2
=====
A bug about the website snippets has been fixed;
- Add the snippet to a group in which you are member
- Change the group of the snippet to a group in which you are not member
- The snippet still show "Unsubscribe"
Bug 3
=====
In the portal view, when messages are filtered by month, messages which
are created in the last day of the month appear also in the next month.
Specification
=============
UI
--
Show the Allow / Ban buttons whatever is the state of the message, so
the user can ban / whitelist a message even if there's no pending email.
When we receive an email, remove the "Mailing List" footer. This was a
big issue because when some users reply many times, the footer was
duplicated on each answer.
Improve the form view of the mail group:
- On non-moderated group, show all messages instead of only pending
message
- Show the guidelines stuff on non-moderated groups
Improve the portal view
- show the attachments under every message (not only on opened message)
- improve the template to confirm the subscription of the user
- improve the portal view for mobile device
- do not show rejected messages, even for admin
- do not show the button "X replies" when the mode is not thread
Technical
---------
Split the route "/groups/subscription" into "/group/subscribe" and
"/group/unsubscribe"
Correctly quote the parent email for the emails sent by web version of
Outlook (with the HTML id "divRplyFwdMsg").
ACLs
----
Now only administrators or responsible / moderators can write / unlink
the group.
Fix an ACL issue on the group member where the responsible of a
non-moderated group could not access the members of the group.
Members with same email
-----------------------
Improve the behavior when multiple members have the same email address
(this situation is possible because there's no unique constraint on the
email field of the <res.partner> model.) We now return the most
appropriate member for the given email address and partner.
When unsubscribing by clicking on the confirmation link that we received
by email, remove all members with the same email address.
Links
=====
Task-2599676
See odoo/odoo/pull/73640
closesodoo/odoo#73640
Related: odoo/upgrade#2774
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose is to better understand the use of email_re regex (find emails in a
text) and email_split_tuples (find name and email in a string used in email
headers or as email_from / email_to / email_cc).
Task ID-2377974
Community PR odoo/odoo#61467
This tool function is deprecated. As explained in docstring: "since OpenERP 6.1
please use ir.mail_server.send_email()". Seems we are allowed to finally remove
it.
Task ID-2377974
Community PR odoo/odoo#61467
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
It seems quite easy to have a void content in the currently new editor
that actually holds a lot of undesired information, notably a font
tag. This is annoying when having behavior based on an html field
being empty or not. In this commit we therefore improve definition
of an 'empty' html field.
Task ID-2532529
PR odoo/odoo#71793
*: tools
The ImageOptimizer option can now set shapes on images to use those
shapes as clip path and background. It uses a flexible path so that the
shape will always fit the image proportionally. We also added an html
file alongside a javascript file to semi-convert the shape from
illustrator into a usable shape for this usecase (the clip path must
have values between one and zero therefore we must do some computation
before the shape is ready to use).
Thanks to Samuel for this specific shape-converter tool and other
technical points about svgs and clip-paths.
Thanks to Mehdi for remaining development post-testing and post-reviews
and for the many fixes.
Thanks to Brieuc for the actual shape SVG files.
Part of https://github.com/odoo/odoo/pull/69179
task-2327045
closesodoo/odoo#69179
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: xO-Tx <mou@odoo.com>
Co-authored-by: Brieuc-brd <brd@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
- Change an image option in mass mailing editor (e.g. Quality)
- save
- edit -> The option can't get the new applied value.
The body_arch's field used in mass mailing editor is
sanitizing attributes and as a consequence, option related data
attrs are removed on save.
task-2327045
closesodoo/odoo#72311
X-original-commit: 7c5666611363a73bc6fef070565039cf065a0176
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Before this commit there were few cases in which
is_html_empty method was not working as our expectation.
In cases such as "<p class=""><br/></p>", "<p id=""><br/></p>"
and many other cases.
In this commit we improves regex of is_html_empty method.
Mow from this commit all kind of attributes will be consider
in this method.
In this commit we also have added is_html_empty in mail template, portal
values and in report values also for rendering templates.
Task id: 2499504
X-original-commit: fd0a05f2955b9f7e9ae7233afebfd6240c9244dd
In `prepend_html_content`, if body `html_body` and `html_content` are
`Markup` objects, the processing of `html_content` through `re.sub`
will strip away the `Markup` flag, leading to the later concatenation
of the body and content first escaping the content (to safely convert
it to a `Markup`) before performing the concatenation proper.
As a result, the `preview` of a `mailing.mailing` would be
double-escaped, and the markup hiding it would instead be printed as
part of the body.
Assume that the type of `html_content` is str-compatible, and
immediately rewrap the value after processing it through
`re.sub`. This should do nothing if `html_content` was an actual
`str`, and will re-flag it as a `Markup` if it was one.
Also change the finding of the insertion point to use `re.search`, I
don't understand why this uses `finditer`, it just makes the code more
complicated.
closesodoo/odoo#71090
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Make the email body take the entire space to avoid having some wasted space,
this will also make the user have more focus when designing an email.
Revamp the settings notebook page in order to give more clarity to the user,
and move some fields from the main form have been moved to this section to
have more space in the bottom for the email body.
In the mailing form, when the mailing is sent or is being sent, set fields
which are no longer useful for the user to change to readonly mode.
Add a wizard that enables the user to schedule a mailing, the schedule field is
still kept in the form for the user to be able to change the date when the
mailing is in the queue (if they want to send it sooner).
Display an action helper-style content when the email is empty, because,
currently, the user is left with a big white screen when the email has no
content which is not desirable.
Update the html_empty function to take into account style attributes to better
match the editor's void content.
Hide A/B testing fields from SMS mailing form view as these are not supported
for SMS marketing.
Task-2469409
closesodoo/odoo#68882
Ent-pr: https://github.com/odoo/odoo/pull/68882
Related: odoo/enterprise#18391
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
We want to be able to force the FROM headers when we sent email in
SMTP. So we can avoid the emails to be considered as spam.
Specifications
==============
This is done with 2 system parameters.
If the system parameter `mail.force.smtp.from` is set we encapsulate all
outgoing email from with the given value.
If the previous system parameter is not set and if both
`mail.dynamic.smtp.from` and `mail.catchall.domain` are set, we
encapsulate the FROM only if the domain of the email is not the same as
the domain of the catchall parameter.
Otherwise we do not encapsulate the email (same behavior as before this
commit).
Task 2367946
See odoo/odoo/pull/61853
closesodoo/odoo#70980
X-original-commit: 08a561505b92d23c4c4ec4094b0cee209ece8753
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
* The result of `plaintext2html` is fully controlled and
markup-safe (the first thing we do is escape the input).
* For `append_content_to_html`, we assume the inputs are HTML and the
output is thus always properly HTML.
Alternatively, we may want to `Markup("%s%s") % ...` and require the
inputs to be properly marked? That seems like a good idea.
* In `_replace_local_links`, applying re.sub will strip out the markup
mark, so store it and reapply it on output if necessary.
That one is a big gnarly, because if the input to ustr is
markup-safe bytes (e.g. qweb rendering output) then the output is a
Markup object, but if the input is str then the output is str, so we
need to check before and after unless... we update ustr to check for
subclasses instead of exact type?
* In `_prepend_preview` the issue is similar to that of
`append_content_to_html`, though in this case we should *not* trust
the input, so we can flag the "parent document" as Markup and format
the preview bit in.
* And since we're marking mail's jinja output as safe, do the same for
web and iot.
Sadly there doesn't seem to be any hook for doing that at the
environment level of jinja, so every `Template.render` site has to
be marked.
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
Scenario:
- add a file in mass mailing editor => an icon is added linking the file
- save
- edit => the icon is replaced by a generic icon
- save
- send mail => no icon is shown in email received
The system is using eg. `data-mimetype="pdf"` to show the icon, before
12.0 this worked but in saas-12.3 the system uses mailing.mailing
body_arch's field that is sanitizing attributes and remove it on save,
so the icon is only working in received email if you save one and only
one time after adding file inside the mass mailing.
opw-2474053 (ticket for similar issue in 14.0)
closesodoo/odoo#70057
X-original-commit: 7e14515b84d8081557517df7e9dc7d8c0d17f2be
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
The mail signature of an user did not inline the CSS like the body of a
message, so for example icons seemed to work but would not be seen
(unless the mail reader had the same font-awesome system).
With this change we inline the signature too.
On edition there was also an issue since the original icon was saved as
data-class which was stripped: so eg. when editing a mail.message with
working icon, the icons would disappear.
opw-2453912
closesodoo/odoo#66372
X-original-commit: 5e093b7a37984c8cc5aa73d1c0f7aa9c9345e6b1
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
In order to make a wow effect right after DB install we enrich the base
company based on the given email address or company website at install.
In order to achieve that goal we use the partner_autocomplete service from
IAP. It uses free credits that are offered to new clients on saas platform.
Only the fields that are not filled yet are enriched to avoid erasing user
entered data, except logo. Indeed as partner_autocomplete is probably installed
as a core app (mail -> iap -> partner_autocomplete auto install chain) it is
unlikely that people already updated their company logo.
We consider that having a call to IAP consuming a token is ok for a standard
use case, especially that
* if no iap service is configured call to IAP won't add much timing;
* on Odoo SaaS free credits are given and company is enriched;
* on custom SaaS with custom iap service, at db creation probably no
credits are given and call will simply give no results back;
We decided to call IAP asynchronously at client web load. Session combined to
a boolean field on company model allows to do this call only once per company.
Doing this allows to avoid adding yet another post init hook. It also eases
behavior tweak through inheritance.
This call is limited to admin for obvious security reasons as well as
performance reason (limiting calls to external providers). As call will be
done once per company generally admin is the first person to log into the
its newly created Odoo.
LINKS
Task ID-2322455
PR odoo/odoo#64600
PR odoo/upgrade#2086
Co-Authored-By: David Beguin <dbe@odoo.com>
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
What are the steps to reproduce your issue ?
1. Install website_blog
2. Go to website and add a new blog post
3. Remove the block content
4. Save and go to Blog page
What is currently happening ?
Crash:
Template fallback
An error occured while rendering the template website_blog.post_teaser
opw-241937
closesodoo/odoo#63844
X-original-commit: d9097c2b6244154ea64142b5e8fd31cfdd34e66e
Signed-off-by: Achraf <abz-odoo@users.noreply.github.com>
PURPOSE
When displaying an email in a list the mail client (gmail, outlook...) computes
a preview based on the content. This preview is generally not well computed as
it contains a lot of garbage and tags while it should contain only relevant
text.
SPECIFICATIONS
To build this preview, all mail clients read the content of the email.
The only way to be able to customize the preview is to add an invisible
HTML element at the beginning of the email with the wanted preview text.
We add at the end of the preview `‌` (zero-width non-joiner) to
fill the end of the preview in order to not have the beginning of the
mail at the end of the preview. It doesn't work with simple space as
the mail clients trim each HTML element content.
Task ID-2172125
PR #49886
Since a recent commit[1], we have a utility method in tools named
'is_html_empty' that checks whether the given html content is
void(containing only formatting tags) or not. However, the re from
this method does not consider the case of self closing tags, fox ex
`<br/>`. In such cases, the method returns Falsy value even if the
content is void.
This commit fixes the issue by considering self-closing void tags
in the regular expression.
commit[1] - https://github.com/odoo/odoo/commit/974f512f5f5d3b9f80a8c3fcde290e4f55cf1230
Task - 2267689
closesodoo/odoo#53167
X-original-commit: 0fd6c9388dc959a9b6b7828850683175edb05b9d
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Provide a follow-up/overview of the Mailing 24 hours after its been sent.
Specifications
==============
Send an email to the responsible of the mailing 24 hours the last email
of the mass mailing.
During the link trackers creation, extract the button label if exists
(so we can display them in the statistics email).
Technical remarks
=================
The statistics email is sent to the responsible of each mailing in the
CRON of mass mailing.
Task-2227411
closesodoo/odoo#49836
Related: odoo/upgrade#1094
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
We already have BDO tag as allowed tag in sanitized HTML that is
similar to BDI tag, so this commit also adds BDI in allowed tags.
BDI* tags make sense in email since email client can have different
orientation than Odoo (alternative is to use `<div dir="auto"></div>`).
*: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/bdi
opw-2256982
closes#52787closesodoo/odoo#52801
X-original-commit: 68edebb0d862acb3dc14f839aaef744306c5e269
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Some tags are not necessary in the HTML template and we can remove them to
have a proper HTML output.
Task 2171753
closesodoo/odoo#51757
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Currently a custom override of Cleaner.allow_element()
exists in tools to allow object tags for SVG images.
This was due to the first prototype of website builder
and its first implementation of image and svg management.
This is not necessary anymore as using the tag is sufficient.
We can safely clean code in the cleaner, allowing to speedup
its performances. As it is used in most html fields and email
parsing each unnecessary code removed is time gainged.
Task 2215228
closesodoo/odoo#51294
X-original-commit: 4a0b09fd3b0d3b63f2df00f002e4d6825cac4eef
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
It has been a recurrent request from customers to be able to send email
messages to email addresses containing non-ascii characters. [IDNA] is a
domain extension to allow unicode characters in domain names. [SMTPUTF8]
is a SMTP extension to allow unicode in any header.
IDNA defines the [punycode] encoding which translates unicode to an
ascii representation. This encoding MUST be used to encode domains.
SMTPUTF8 is an SMTP extension that allow utf-8 in all headers on the
envelope.
[IDNA] https://tools.ietf.org/html/rfc5890
[SMTPUTF8] https://tools.ietf.org/html/rfc6531
[punycode] https://tools.ietf.org/html/rfc3492
Task: 2116928
opw-2229906
opw-2248251
closesodoo/odoo#47709
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
A plaintext email is displayed in a `<pre/>` tag to conserve spacing.
But since there is no escaping, if in this text there was XML tags or
HTML entities, they would appear as HTML in browser which is not wanted.
Do note that this was not a security issue since the content will still
be subjected to the checks and foundling of HTML emails.
Without the change, the added test would fail because character &,<,>
were not escaped.
opw-2242323
closes#50003closesodoo/odoo#50123
X-original-commit: 932532b5b59e0b71c8e16dadfb2ff36c38764208
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
* = event, website_event_track, website_sale, website_hr_recruitment,
website_livechat, website_sale, website_slides
The option to sanitize or not the forms was not available, this will
allow better flexibility on whether forms should be sanitized or not on
an HTML field.
Also we use this new param to allow forms to be added on some already
existing html fields where forms where sanitized out.
task-2209554
closesodoo/odoo#47318
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
PURPOSE
Move links shortening tools on mail.render.mixin to have rendering and
post procesing tools available on that mixin.
SPECIFICATIONS
Move html and text links shortening methods on mail.render.mixin class.
That way rendering and post processing is done in that mixin instead of
being specific to link.tracker model. Indeed those methods are tools and
not really business related methods.
Future commits will improve the way mail and sms composer use the render
mixin. This commit is therefore both a cleaning and preparing commit.
LINKS
Task ID 1963529
Community PR odoo/odoo#32397
PURPOSE
When incoming emails bounce due to alias security bounce email is quite
generic. Purpose of this task is to ease its customization and update.
SPECIFICATIONS
In order to improve the flexibility of alias, add a customizable html field
on the alias model. This html content will be send as bounce email core content
in case of bounced/unauthorized mail received for this alias,
Obviously it has no effect on 'everyone' security setting as no email will
bounce due to that issue.
If it is not set a default generic mail will be send depending on security
setting. It allows to keep void html fields when no specific bounce content
is required
In HR, an old template allowing some light customization for employee based
security option is removed as it is completely replaced by the new feature.
Also add references message-id of the mail received to the answer so that
threads are correctly set.
LINKS
Task ID 2126509
In odoo empty html is not always completely empty. This tool method allow
to consider html holding void tags as empty html. This happens notably when
using the editor that may store <p><br><p> even if displayed content is void.
Task ID 2126509
PR #41827
When we send a SMS in the contact form view, exact body is sent by SMS and
displayed in chatter. If some manual HTML is added chatter will display it
as HTML while sms receive HTML tags.
We want that the SMS content in the chatter is the same as the SMS sent and
that HTML tags are removed to avoid being interpreted.
To achieve that goal we call html2plaintext in ``_message_sms`` and in
``prepare_log_body_value`` that are two entry points to send SMS.
We also update ``html2plaintext`` to strip result in order to avoid having
unnecessary spaces left.
Task ID 2126123
PR #40441closesodoo/odoo#46362
X-original-commit: 4b7b14a5ea5e6ab521838c01813fa478154682fd
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose of this commit is to better support custom styling used notably
for mass mailing and outgoing emails. Indeed styling is whitelisted when
storing html. Finding it in html requires some parsing. This commit improves
the regex used to find it when semicolons are involved. It allows more use
cases to be correctly supported.
Task ID #2125856
PR #40433closesodoo/odoo#41969
X-original-commit: 5235a49a5cd0f79f5590c96afa26dce25465fe06
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Python 2 `email.tools.formataddr` doesn't encode not RFC-2822 compliant
realname to base64 or quoted-printable. This is the wanted behavior to
output formatted email addresses on screen.
Python 3 `email.tools.formataddr` does encode the realname to be
RFC-2822 compliant. This is the wanted behavior when connecting to a
smtp server to send emails.
That method has been deprecated by pep-594 as the new python email API
is capable of automatically formatting email address in a RFC compliant
way. As the method was still in use in a lot of modules as the
preferred way to format email addresses, a refined P3-like
implementation as been included in the tool suite.
This commit changes the default behavior so it mimics P2 implementation
with an easy way to use the P3 behavior.
Task: 2003936
It seems the emails > < are missing.
By the way one manual formataddr is replaced by an email_formatted field.
Result is the same but let us use fields doing it for us.
These functions convert plain text links into clickable hyperlinks.
These open in new tabs but did not have the "noreferrer noopener" rel
attributes, which made them susceptible to reverse tabnabbing.
A lot of attack vectors were available to unregistered, uninvited
anonymous users and presented a significant phishing threat (such as
posting links in the instant-chat, through a mail-alias, in a forum post
or in a twitter post) and leading the operator to believe he had been
disconnected from odoo in the original tab, prompting them to enter
their credentials.
while these three places will add the noreferrer and noopener attributes
on the anchor tags generated by them, there are still many places that
create hyperlinks without the use of these functions, although most of
them are static links, they still represent a transitive security
vulnerability to the linked sites.
There are also a few modules and widgets that roll out their own links
or open new tabs unsafely using window.open(), these will need to be
patched separately.
closesodoo/odoo#37591
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Since it appears that email validation feature from Flanker v0.9.0
is completelly broken (the lib explodes when calling
address.validate_address('some@address.xxx')), Flanker is then removed
and email validation will be done only using the email_normalize method.
Task ID: 2044539
PR #37413
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules: `email.(charset|header|mime|utils)`.
The new `email.message.EmailMessage` API exposes a much easier interface
to create multiparted emails [1], is capable of doing all the necessary
headers value conversion (RFCs [2045], [2047], [2049]) [2] and get/set
different flavor of payload (text/bytes) in a straightforward way [3].
All headers are structured in a way to support python native types, i.e.
the `Date` header supports `datetime.datetime` objects and automatically
performs the required formatting. The same goes for multi-valued headers
like the `To` header, one can directly set a python list of values, it
will be automatically be formatted according to the RFCs.
The dedicated encoding and decoding functions are no more needed thus
has been removed has part of the refactor. FTR, [RFC2231] is an update
of [RFC2047] and based on tests we've just conducted, both GMail and
Thunderbird now support RFC2231 encoding just fine in all headers:
attachments names, From, etc.
[1]: http://docs.python.org/3/library/email.message.html
[2]: http://docs.python.org/3/library/email.headerregistry.html
[3]: http://docs.python.org/3/library/email.contentmanager.html
[2045]: https://tools.ietf.org/html/rfc2045
[2047]: https://tools.ietf.org/html/rfc2047
[2049]: https://tools.ietf.org/html/rfc2049
[2231]: https://tools.ietf.org/html/rfc2231closesodoo/odoo#35929
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules. Among them the `email.utils.formataddr`
function, this function takes a `(name, email)` pair and returns a
string value suitable for From, To and Cc headers.
The stdlib function is capable of handling several character encoding
and two binary-to-ascii encoding. Odoo only uses `utf-8` which is
compatible with the base64 b2a encoding. The re-implementation of that
function has been simplified to only support base64-ed utf-8 and ascii.
[PEP 594]: https://www.python.org/dev/peps/pep-0594/\#email-lagacy-api
A sanitize fields by default remove all attributes outside of a
whitelist (if sanitize_attributes on field is not set to false).
This prevent to use parallax widget in Html sanitized field since
attribute "data-scroll-background-ratio" define parallax effect speed
but was not in whitelist.
note: also keep data-interval for gallery widget
opw-2053983
closes#35961
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
PURPOSE
Add some improvements in mail gateway: remove private discussion, improve
bounce management, allow resetting bounce counters, improve automatic set or
reset of blacklists and ease mass mailing inheritance.
SPECIFICATIONS
Parsing email values: make message_parse effectively prepare all required
values for later processing
* update message_parse to add some more data directly in parsed dictionary
and avoid having to parse value again in later processing and computation.
* notably recipients computation is now done directly in parsing. It
includes email check and reconstruction using tools methods. That way
routing just has to handle those values and not compute part of them;
* from and cc are correctly managed with parsed and cleaned (sanitized)
versions in dictionary;
* remove all code decoding message header and replace them by fetching
the parsed value instead;
* filter parsed values before calling message_post to ensure right values
are given to it instead of having to manage them in message_post. Indeed
caller has to give the right input;
* make parsing sub-methods (extract payload, post process payload) have
a dictionary as input and output to always manage a dictionary of values
through the parsing methods and update their namespace;
Routing: remove deprecated code parts
* remove use of email_references regex. Indeed emails are routed using their
messageId and not the regex anymore since several versions [1];
* clean some unnecessary variables;
* simplify variable type input, like message_parse accepts only valid
email.message instances and to avoid several decode / encode. Tests are
updated accordingly;
Update docstrings of those methods.
LINKS
Related to task 1893155
Linked to PR #33340
[1] See df037e2959 and 0028c42136
As Flanker can be used to validate an email address and was originally
used in website_crm_score module, the Flanker usage is now moved into tools.
This commit prepares the next Predictive Lead Scoring (aka PLS) ones
as PLS will use the email state as computation variable,
it will need this flanker validation to compute first the email state.
We use also the already implemented normalized email as a double validation.
Flanker is not in the required dependencies of Odoo :
So if flanker is installed, it will check if email is valid
(checking DNS and other standard rules).
If Flanker is not installed, email_validate will only normalize the email.
Anyway, the email must at least be normalizable.
Task ID : 1925439
PR #33589
Nicolas Bayet