Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Currently, the 'similar lead detection' mechanism only considers email,
contact name, and partner name while finding duplicate leads in CRM.
After this commit, it will also consider the mobile/phone number for
the same. Note that the mobile numbers and phone numbers both will be
matched with each other while finding duplicate leads.
Task-2817884
closesodoo/odoo#88372
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Prepare an informative and nice-looking preview from the main content of the
email body, avoiding buttons/images alt etc.
Links, images, tables are removed.
Whitespace is added after the preview to avoid including the full message in
the preview (with markup).
Tags and attributes are also added to increase compliance with HTML5 standard,
including accessibility.
One additional SQL request is required to fetch the preview sub-template.
Task-2413355
closesodoo/odoo#86266
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
Fix bugs in link replacement (html and text version)
1/ The replacement of urls in text (not html) was buggy when
* a string includes multiple urls, and
* `base_url` is one of them and not the first
* Also in SMS marketing when adding the sms `id`
When it came to base_url's turn, the replace function for the content would
replace the `base_url` part of a previously shortened url instead of the
distinct `base_url` link.
2/ Ampersand character
A) The ampersand character prevented replacement of an url inside a `Markup`.
Previous tests passed as the tracker was created but the url was not replaced.
B) The ampersand character was not recognized as part of an url for simple
strings.
3/ Replacement of already short links
A faulty logic made it possible to replace "/r/" urls when no "blacklist" was
passed.
Commit
* Ensures that only the base_url link would be replaced instead in this case.
* Adds support for urls with "&"
* Does prevent shortening short urls
* Improves link conversion performance (regex + avoid duplication when there
are several occurrences of the same url)
* Adds multiple unit tests
Task-2783844
closesodoo/odoo#91164
X-original-commit: 61026806d831b49a7cf5c44ffef3d762986781ce
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When the system broadcasts an email response to document followers,
if the config parameters `mail.force.smtp.from` or
`mail.dynamic.smtp.from` are defined, it will rewrite the `From`
address to avoid spoofing the sender's domain.
**NOTE**: As of 15.0, this is based on the `from_filter` setting on the
corresponding ir.mail_server, rather than the abovementioned config
parameters, but the rest of the discussion stands.
For example, if the `mail.catchall.domain` is set to `example.com` and
an email response comes from:
"John D" <john@doe.com>
it will rewrite it to:
"John D (john@doe.com)" <notifications@example.com>
This will make sure the system never sends outgoing email for an external
domain, as it has no authority for doing so, and that could
break mail filtering/authentication rules (SPF, DMARC, etc.)
During this "encapsulation rewrite step", both the original Sender name
and their email are preserved, and put into the quoted "name" field of
the rewritten address. It seems sensible to preserve as much information
as possible about the original sender.
Unfortunately, the inclusion of the Sender email in the final name makes
it appear to some inbox providers as if the message is trying to
deceptively impersonate another person (as many phishing schemes would).
As of November 2021 GMail at least does this, and will hide the name in
the UI when it happens. It will keep only the rewritten email, which is not
very useful in the case of a notification (even though it's more
technically correct, of course).
This patch removes the original email from the rewritten notification,
keeping only the name, considering that the email is not the most
important part, and it's better to have one of the two than none.
So after the patch, the rewritten address is now:
"John D" <notifications@example.com>
When there is no name in the original address, we keep only the local
part of the email, to avoid the same display issue. The recipient will
have to identify the sender based on the context / past messages.
closesodoo/odoo#81807
X-original-commit: 3c65ec5a8191a392980ceb0a8c584767eae405f1
Signed-off-by: Olivier Dony <odo@odoo.com>
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
fields) are out of scope for the lint but my editor catches
closesodoo/odoo#74604
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
Purpose
=======
This commit improves the UI of the mail group module (backend and portal
view), fix some bugs and clean the code.
Bug 1
=====
If a user click 2 times on a confirmation link, an error is raised.
Bug 2
=====
A bug about the website snippets has been fixed;
- Add the snippet to a group in which you are member
- Change the group of the snippet to a group in which you are not member
- The snippet still show "Unsubscribe"
Bug 3
=====
In the portal view, when messages are filtered by month, messages which
are created in the last day of the month appear also in the next month.
Specification
=============
UI
--
Show the Allow / Ban buttons whatever is the state of the message, so
the user can ban / whitelist a message even if there's no pending email.
When we receive an email, remove the "Mailing List" footer. This was a
big issue because when some users reply many times, the footer was
duplicated on each answer.
Improve the form view of the mail group:
- On non-moderated group, show all messages instead of only pending
message
- Show the guidelines stuff on non-moderated groups
Improve the portal view
- show the attachments under every message (not only on opened message)
- improve the template to confirm the subscription of the user
- improve the portal view for mobile device
- do not show rejected messages, even for admin
- do not show the button "X replies" when the mode is not thread
Technical
---------
Split the route "/groups/subscription" into "/group/subscribe" and
"/group/unsubscribe"
Correctly quote the parent email for the emails sent by web version of
Outlook (with the HTML id "divRplyFwdMsg").
ACLs
----
Now only administrators or responsible / moderators can write / unlink
the group.
Fix an ACL issue on the group member where the responsible of a
non-moderated group could not access the members of the group.
Members with same email
-----------------------
Improve the behavior when multiple members have the same email address
(this situation is possible because there's no unique constraint on the
email field of the <res.partner> model.) We now return the most
appropriate member for the given email address and partner.
When unsubscribing by clicking on the confirmation link that we received
by email, remove all members with the same email address.
Links
=====
Task-2599676
See odoo/odoo/pull/73640
closesodoo/odoo#73640
Related: odoo/upgrade#2774
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose is to better understand the use of email_re regex (find emails in a
text) and email_split_tuples (find name and email in a string used in email
headers or as email_from / email_to / email_cc).
Task ID-2377974
Community PR odoo/odoo#61467
This tool function is deprecated. As explained in docstring: "since OpenERP 6.1
please use ir.mail_server.send_email()". Seems we are allowed to finally remove
it.
Task ID-2377974
Community PR odoo/odoo#61467
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
It seems quite easy to have a void content in the currently new editor
that actually holds a lot of undesired information, notably a font
tag. This is annoying when having behavior based on an html field
being empty or not. In this commit we therefore improve definition
of an 'empty' html field.
Task ID-2532529
PR odoo/odoo#71793
*: tools
The ImageOptimizer option can now set shapes on images to use those
shapes as clip path and background. It uses a flexible path so that the
shape will always fit the image proportionally. We also added an html
file alongside a javascript file to semi-convert the shape from
illustrator into a usable shape for this usecase (the clip path must
have values between one and zero therefore we must do some computation
before the shape is ready to use).
Thanks to Samuel for this specific shape-converter tool and other
technical points about svgs and clip-paths.
Thanks to Mehdi for remaining development post-testing and post-reviews
and for the many fixes.
Thanks to Brieuc for the actual shape SVG files.
Part of https://github.com/odoo/odoo/pull/69179
task-2327045
closesodoo/odoo#69179
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: xO-Tx <mou@odoo.com>
Co-authored-by: Brieuc-brd <brd@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
- Change an image option in mass mailing editor (e.g. Quality)
- save
- edit -> The option can't get the new applied value.
The body_arch's field used in mass mailing editor is
sanitizing attributes and as a consequence, option related data
attrs are removed on save.
task-2327045
closesodoo/odoo#72311
X-original-commit: 7c5666611363a73bc6fef070565039cf065a0176
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Before this commit there were few cases in which
is_html_empty method was not working as our expectation.
In cases such as "<p class=""><br/></p>", "<p id=""><br/></p>"
and many other cases.
In this commit we improves regex of is_html_empty method.
Mow from this commit all kind of attributes will be consider
in this method.
In this commit we also have added is_html_empty in mail template, portal
values and in report values also for rendering templates.
Task id: 2499504
X-original-commit: fd0a05f2955b9f7e9ae7233afebfd6240c9244dd
In `prepend_html_content`, if body `html_body` and `html_content` are
`Markup` objects, the processing of `html_content` through `re.sub`
will strip away the `Markup` flag, leading to the later concatenation
of the body and content first escaping the content (to safely convert
it to a `Markup`) before performing the concatenation proper.
As a result, the `preview` of a `mailing.mailing` would be
double-escaped, and the markup hiding it would instead be printed as
part of the body.
Assume that the type of `html_content` is str-compatible, and
immediately rewrap the value after processing it through
`re.sub`. This should do nothing if `html_content` was an actual
`str`, and will re-flag it as a `Markup` if it was one.
Also change the finding of the insertion point to use `re.search`, I
don't understand why this uses `finditer`, it just makes the code more
complicated.
closesodoo/odoo#71090
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Make the email body take the entire space to avoid having some wasted space,
this will also make the user have more focus when designing an email.
Revamp the settings notebook page in order to give more clarity to the user,
and move some fields from the main form have been moved to this section to
have more space in the bottom for the email body.
In the mailing form, when the mailing is sent or is being sent, set fields
which are no longer useful for the user to change to readonly mode.
Add a wizard that enables the user to schedule a mailing, the schedule field is
still kept in the form for the user to be able to change the date when the
mailing is in the queue (if they want to send it sooner).
Display an action helper-style content when the email is empty, because,
currently, the user is left with a big white screen when the email has no
content which is not desirable.
Update the html_empty function to take into account style attributes to better
match the editor's void content.
Hide A/B testing fields from SMS mailing form view as these are not supported
for SMS marketing.
Task-2469409
closesodoo/odoo#68882
Ent-pr: https://github.com/odoo/odoo/pull/68882
Related: odoo/enterprise#18391
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
We want to be able to force the FROM headers when we sent email in
SMTP. So we can avoid the emails to be considered as spam.
Specifications
==============
This is done with 2 system parameters.
If the system parameter `mail.force.smtp.from` is set we encapsulate all
outgoing email from with the given value.
If the previous system parameter is not set and if both
`mail.dynamic.smtp.from` and `mail.catchall.domain` are set, we
encapsulate the FROM only if the domain of the email is not the same as
the domain of the catchall parameter.
Otherwise we do not encapsulate the email (same behavior as before this
commit).
Task 2367946
See odoo/odoo/pull/61853
closesodoo/odoo#70980
X-original-commit: 08a561505b92d23c4c4ec4094b0cee209ece8753
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
* The result of `plaintext2html` is fully controlled and
markup-safe (the first thing we do is escape the input).
* For `append_content_to_html`, we assume the inputs are HTML and the
output is thus always properly HTML.
Alternatively, we may want to `Markup("%s%s") % ...` and require the
inputs to be properly marked? That seems like a good idea.
* In `_replace_local_links`, applying re.sub will strip out the markup
mark, so store it and reapply it on output if necessary.
That one is a big gnarly, because if the input to ustr is
markup-safe bytes (e.g. qweb rendering output) then the output is a
Markup object, but if the input is str then the output is str, so we
need to check before and after unless... we update ustr to check for
subclasses instead of exact type?
* In `_prepend_preview` the issue is similar to that of
`append_content_to_html`, though in this case we should *not* trust
the input, so we can flag the "parent document" as Markup and format
the preview bit in.
* And since we're marking mail's jinja output as safe, do the same for
web and iot.
Sadly there doesn't seem to be any hook for doing that at the
environment level of jinja, so every `Template.render` site has to
be marked.
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
Scenario:
- add a file in mass mailing editor => an icon is added linking the file
- save
- edit => the icon is replaced by a generic icon
- save
- send mail => no icon is shown in email received
The system is using eg. `data-mimetype="pdf"` to show the icon, before
12.0 this worked but in saas-12.3 the system uses mailing.mailing
body_arch's field that is sanitizing attributes and remove it on save,
so the icon is only working in received email if you save one and only
one time after adding file inside the mass mailing.
opw-2474053 (ticket for similar issue in 14.0)
closesodoo/odoo#70057
X-original-commit: 7e14515b84d8081557517df7e9dc7d8c0d17f2be
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
The mail signature of an user did not inline the CSS like the body of a
message, so for example icons seemed to work but would not be seen
(unless the mail reader had the same font-awesome system).
With this change we inline the signature too.
On edition there was also an issue since the original icon was saved as
data-class which was stripped: so eg. when editing a mail.message with
working icon, the icons would disappear.
opw-2453912
closesodoo/odoo#66372
X-original-commit: 5e093b7a37984c8cc5aa73d1c0f7aa9c9345e6b1
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
In order to make a wow effect right after DB install we enrich the base
company based on the given email address or company website at install.
In order to achieve that goal we use the partner_autocomplete service from
IAP. It uses free credits that are offered to new clients on saas platform.
Only the fields that are not filled yet are enriched to avoid erasing user
entered data, except logo. Indeed as partner_autocomplete is probably installed
as a core app (mail -> iap -> partner_autocomplete auto install chain) it is
unlikely that people already updated their company logo.
We consider that having a call to IAP consuming a token is ok for a standard
use case, especially that
* if no iap service is configured call to IAP won't add much timing;
* on Odoo SaaS free credits are given and company is enriched;
* on custom SaaS with custom iap service, at db creation probably no
credits are given and call will simply give no results back;
We decided to call IAP asynchronously at client web load. Session combined to
a boolean field on company model allows to do this call only once per company.
Doing this allows to avoid adding yet another post init hook. It also eases
behavior tweak through inheritance.
This call is limited to admin for obvious security reasons as well as
performance reason (limiting calls to external providers). As call will be
done once per company generally admin is the first person to log into the
its newly created Odoo.
LINKS
Task ID-2322455
PR odoo/odoo#64600
PR odoo/upgrade#2086
Co-Authored-By: David Beguin <dbe@odoo.com>
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
What are the steps to reproduce your issue ?
1. Install website_blog
2. Go to website and add a new blog post
3. Remove the block content
4. Save and go to Blog page
What is currently happening ?
Crash:
Template fallback
An error occured while rendering the template website_blog.post_teaser
opw-241937
closesodoo/odoo#63844
X-original-commit: d9097c2b6244154ea64142b5e8fd31cfdd34e66e
Signed-off-by: Achraf <abz-odoo@users.noreply.github.com>
PURPOSE
When displaying an email in a list the mail client (gmail, outlook...) computes
a preview based on the content. This preview is generally not well computed as
it contains a lot of garbage and tags while it should contain only relevant
text.
SPECIFICATIONS
To build this preview, all mail clients read the content of the email.
The only way to be able to customize the preview is to add an invisible
HTML element at the beginning of the email with the wanted preview text.
We add at the end of the preview `‌` (zero-width non-joiner) to
fill the end of the preview in order to not have the beginning of the
mail at the end of the preview. It doesn't work with simple space as
the mail clients trim each HTML element content.
Task ID-2172125
PR #49886
Since a recent commit[1], we have a utility method in tools named
'is_html_empty' that checks whether the given html content is
void(containing only formatting tags) or not. However, the re from
this method does not consider the case of self closing tags, fox ex
`<br/>`. In such cases, the method returns Falsy value even if the
content is void.
This commit fixes the issue by considering self-closing void tags
in the regular expression.
commit[1] - https://github.com/odoo/odoo/commit/974f512f5f5d3b9f80a8c3fcde290e4f55cf1230
Task - 2267689
closesodoo/odoo#53167
X-original-commit: 0fd6c9388dc959a9b6b7828850683175edb05b9d
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Provide a follow-up/overview of the Mailing 24 hours after its been sent.
Specifications
==============
Send an email to the responsible of the mailing 24 hours the last email
of the mass mailing.
During the link trackers creation, extract the button label if exists
(so we can display them in the statistics email).
Technical remarks
=================
The statistics email is sent to the responsible of each mailing in the
CRON of mass mailing.
Task-2227411
closesodoo/odoo#49836
Related: odoo/upgrade#1094
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
We already have BDO tag as allowed tag in sanitized HTML that is
similar to BDI tag, so this commit also adds BDI in allowed tags.
BDI* tags make sense in email since email client can have different
orientation than Odoo (alternative is to use `<div dir="auto"></div>`).
*: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/bdi
opw-2256982
closes#52787closesodoo/odoo#52801
X-original-commit: 68edebb0d862acb3dc14f839aaef744306c5e269
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Some tags are not necessary in the HTML template and we can remove them to
have a proper HTML output.
Task 2171753
closesodoo/odoo#51757
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Currently a custom override of Cleaner.allow_element()
exists in tools to allow object tags for SVG images.
This was due to the first prototype of website builder
and its first implementation of image and svg management.
This is not necessary anymore as using the tag is sufficient.
We can safely clean code in the cleaner, allowing to speedup
its performances. As it is used in most html fields and email
parsing each unnecessary code removed is time gainged.
Task 2215228
closesodoo/odoo#51294
X-original-commit: 4a0b09fd3b0d3b63f2df00f002e4d6825cac4eef
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
It has been a recurrent request from customers to be able to send email
messages to email addresses containing non-ascii characters. [IDNA] is a
domain extension to allow unicode characters in domain names. [SMTPUTF8]
is a SMTP extension to allow unicode in any header.
IDNA defines the [punycode] encoding which translates unicode to an
ascii representation. This encoding MUST be used to encode domains.
SMTPUTF8 is an SMTP extension that allow utf-8 in all headers on the
envelope.
[IDNA] https://tools.ietf.org/html/rfc5890
[SMTPUTF8] https://tools.ietf.org/html/rfc6531
[punycode] https://tools.ietf.org/html/rfc3492
Task: 2116928
opw-2229906
opw-2248251
closesodoo/odoo#47709
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
A plaintext email is displayed in a `<pre/>` tag to conserve spacing.
But since there is no escaping, if in this text there was XML tags or
HTML entities, they would appear as HTML in browser which is not wanted.
Do note that this was not a security issue since the content will still
be subjected to the checks and foundling of HTML emails.
Without the change, the added test would fail because character &,<,>
were not escaped.
opw-2242323
closes#50003closesodoo/odoo#50123
X-original-commit: 932532b5b59e0b71c8e16dadfb2ff36c38764208
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
* = event, website_event_track, website_sale, website_hr_recruitment,
website_livechat, website_sale, website_slides
The option to sanitize or not the forms was not available, this will
allow better flexibility on whether forms should be sanitized or not on
an HTML field.
Also we use this new param to allow forms to be added on some already
existing html fields where forms where sanitized out.
task-2209554
closesodoo/odoo#47318
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
PURPOSE
Move links shortening tools on mail.render.mixin to have rendering and
post procesing tools available on that mixin.
SPECIFICATIONS
Move html and text links shortening methods on mail.render.mixin class.
That way rendering and post processing is done in that mixin instead of
being specific to link.tracker model. Indeed those methods are tools and
not really business related methods.
Future commits will improve the way mail and sms composer use the render
mixin. This commit is therefore both a cleaning and preparing commit.
LINKS
Task ID 1963529
Community PR odoo/odoo#32397
PURPOSE
When incoming emails bounce due to alias security bounce email is quite
generic. Purpose of this task is to ease its customization and update.
SPECIFICATIONS
In order to improve the flexibility of alias, add a customizable html field
on the alias model. This html content will be send as bounce email core content
in case of bounced/unauthorized mail received for this alias,
Obviously it has no effect on 'everyone' security setting as no email will
bounce due to that issue.
If it is not set a default generic mail will be send depending on security
setting. It allows to keep void html fields when no specific bounce content
is required
In HR, an old template allowing some light customization for employee based
security option is removed as it is completely replaced by the new feature.
Also add references message-id of the mail received to the answer so that
threads are correctly set.
LINKS
Task ID 2126509
In odoo empty html is not always completely empty. This tool method allow
to consider html holding void tags as empty html. This happens notably when
using the editor that may store <p><br><p> even if displayed content is void.
Task ID 2126509
PR #41827
When we send a SMS in the contact form view, exact body is sent by SMS and
displayed in chatter. If some manual HTML is added chatter will display it
as HTML while sms receive HTML tags.
We want that the SMS content in the chatter is the same as the SMS sent and
that HTML tags are removed to avoid being interpreted.
To achieve that goal we call html2plaintext in ``_message_sms`` and in
``prepare_log_body_value`` that are two entry points to send SMS.
We also update ``html2plaintext`` to strip result in order to avoid having
unnecessary spaces left.
Task ID 2126123
PR #40441closesodoo/odoo#46362
X-original-commit: 4b7b14a5ea5e6ab521838c01813fa478154682fd
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose of this commit is to better support custom styling used notably
for mass mailing and outgoing emails. Indeed styling is whitelisted when
storing html. Finding it in html requires some parsing. This commit improves
the regex used to find it when semicolons are involved. It allows more use
cases to be correctly supported.
Task ID #2125856
PR #40433closesodoo/odoo#41969
X-original-commit: 5235a49a5cd0f79f5590c96afa26dce25465fe06
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Python 2 `email.tools.formataddr` doesn't encode not RFC-2822 compliant
realname to base64 or quoted-printable. This is the wanted behavior to
output formatted email addresses on screen.
Python 3 `email.tools.formataddr` does encode the realname to be
RFC-2822 compliant. This is the wanted behavior when connecting to a
smtp server to send emails.
That method has been deprecated by pep-594 as the new python email API
is capable of automatically formatting email address in a RFC compliant
way. As the method was still in use in a lot of modules as the
preferred way to format email addresses, a refined P3-like
implementation as been included in the tool suite.
This commit changes the default behavior so it mimics P2 implementation
with an easy way to use the P3 behavior.
Task: 2003936
It seems the emails > < are missing.
By the way one manual formataddr is replaced by an email_formatted field.
Result is the same but let us use fields doing it for us.
These functions convert plain text links into clickable hyperlinks.
These open in new tabs but did not have the "noreferrer noopener" rel
attributes, which made them susceptible to reverse tabnabbing.
A lot of attack vectors were available to unregistered, uninvited
anonymous users and presented a significant phishing threat (such as
posting links in the instant-chat, through a mail-alias, in a forum post
or in a twitter post) and leading the operator to believe he had been
disconnected from odoo in the original tab, prompting them to enter
their credentials.
while these three places will add the noreferrer and noopener attributes
on the anchor tags generated by them, there are still many places that
create hyperlinks without the use of these functions, although most of
them are static links, they still represent a transitive security
vulnerability to the linked sites.
There are also a few modules and widgets that roll out their own links
or open new tabs unsafely using window.open(), these will need to be
patched separately.
closesodoo/odoo#37591
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Since it appears that email validation feature from Flanker v0.9.0
is completelly broken (the lib explodes when calling
address.validate_address('some@address.xxx')), Flanker is then removed
and email validation will be done only using the email_normalize method.
Task ID: 2044539
PR #37413
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules: `email.(charset|header|mime|utils)`.
The new `email.message.EmailMessage` API exposes a much easier interface
to create multiparted emails [1], is capable of doing all the necessary
headers value conversion (RFCs [2045], [2047], [2049]) [2] and get/set
different flavor of payload (text/bytes) in a straightforward way [3].
All headers are structured in a way to support python native types, i.e.
the `Date` header supports `datetime.datetime` objects and automatically
performs the required formatting. The same goes for multi-valued headers
like the `To` header, one can directly set a python list of values, it
will be automatically be formatted according to the RFCs.
The dedicated encoding and decoding functions are no more needed thus
has been removed has part of the refactor. FTR, [RFC2231] is an update
of [RFC2047] and based on tests we've just conducted, both GMail and
Thunderbird now support RFC2231 encoding just fine in all headers:
attachments names, From, etc.
[1]: http://docs.python.org/3/library/email.message.html
[2]: http://docs.python.org/3/library/email.headerregistry.html
[3]: http://docs.python.org/3/library/email.contentmanager.html
[2045]: https://tools.ietf.org/html/rfc2045
[2047]: https://tools.ietf.org/html/rfc2047
[2049]: https://tools.ietf.org/html/rfc2049
[2231]: https://tools.ietf.org/html/rfc2231closesodoo/odoo#35929
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
[PEP 594] is going to depreciate the legacy `email.message.Message` API
and its related modules. Among them the `email.utils.formataddr`
function, this function takes a `(name, email)` pair and returns a
string value suitable for From, To and Cc headers.
The stdlib function is capable of handling several character encoding
and two binary-to-ascii encoding. Odoo only uses `utf-8` which is
compatible with the base64 b2a encoding. The re-implementation of that
function has been simplified to only support base64-ed utf-8 and ascii.
[PEP 594]: https://www.python.org/dev/peps/pep-0594/\#email-lagacy-api
