Description of the issue/feature this PR addresses:
duplicate mandatory field in popup when save the contact record.
Current behavior before PR:
Its showing same field name twice in popup which asking invalid fields.
Desired behavior after PR is merged:
it should show 'name' field only once.
Fixes#79753closesodoo/odoo#81727
X-original-commit: eb4b49e7078f4aa90f9d652de10386da38ded2dc
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
When trying to save a new res.partner without name, an error appeared saying
Invalid fields:
- Name
- Name
Add condition to the domain to make it required only when displayed
Fixes#77976closesodoo/odoo#79727
X-original-commit: f65e3d9d21d269058dd842a4319b5b2242df103e
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Purpose
=======
Hide non-relevant fields for a portal user. E.G. we want to hide the
notification type, the menu customization... Because those fields
make no sense for a portal user.
Force the non-internal user to receive notifications by emails since
they can not open Discuss.
Task-2508521
Part-of: odoo/odoo#77766
Co-authored-by: nounoubensebia <neb@odoo.com>
base: The two elements were not distinguishable by an xpath.
partner_autocomplete: Use new ids in xpath
The class `o_text_overflow` set on the fields was also
preventing the dropdown menu from appearing. This class is
now moved to the `input` tag instead of the while `div`.
closesodoo/odoo#78141
Task: 2638570
X-original-commit: fd7bfbccd45f874d5a826e695858bc75c8467700
Signed-off-by: Florian Daloze (fda) <fda@odoo.com>
PURPOSE
Help people setuping their mail server with clear labels and form view.
SPECIFICATIONS
Rename Description to Name, as Description indicates a secondary text
field. Add a placeholder to indicate it is used as a functional name
and not a technical field.
Relabel the field for filtering to FROM Filtering. Current From Filter
could lead to think it filters incoming emails which is not the case.
Move button for testing in header as on all form views.
Use radio buttons for authentication and encryption to display available
settings directly to user. This is more user friendly than selection boxes.
Split connection information in two groups: authentication and security.
Each group comes with its options below main radio-based field.
Task-2628092
closesodoo/odoo#76301
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
For the "Archive"/"Unarchive" button to appear in the contextual "Actions" dropdown,
the field must be present in the view (which wasn't the case until now).
This commit also fixes the indentation of the concerned view.
closesodoo/odoo#77596
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
PR #75856 introduced a default implementation of group_expand for
Selection fields that set the value of group_expand to True.
However the PR lacked the necessary bits that allow the same behavior
for fields created on the fly instead of through code.
This commit allows the propagation of the group_expand attribute for
fields of type Selection, this commit also exposes the checkbox in the
UI to enable/disable the setting.
closesodoo/odoo#76775
X-original-commit: 1cacc3e53cf722a31bedff7a76f8e5cc6ed45ce0
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
By activating the profiler debugger, the two qweb options is added. The
qweb templates that need to be rendered are compiled into a new function
to add instructions for saving data.
`Add qweb directive context`
It's a sub-option of "Record sql" or "Record traces", add some context on
thread at current call stack level. This context stored by collector
beside stack and is used by Speedscope to add a level to the stack with
this qweb directive information.
```
directive=t-call='website.layout', xpath=/t/t
t_call_content
directive=t-foreach='5' t-as="'a', xpath=/t/t/div/div
directive=t-esc='website.search([])', xpath=/t/t/div/div/t
execute
```
`Record qweb`
Add profiling data used by ProfilingQwebView widget. In the `ir.profile`
form view, the widget display the duration and number of sql of every qweb
directives with the xml of templates. Every xml is recorded to be
consulted even if the user change the xml templates.
```xml
<t t-call="website.layout">
<div>
<div t-foreach="5" t-as="a">
<t t-esc="website.search([])"/> <!-- will display 5 separate requests -->
</div>
</div>
</t>
```
closesodoo/odoo#74712
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When using profiling on a fresh test database, it can be tedious to
access settings to enable the feature. This commit adds a wizard to help
enabling profiling without accessing the settings when trying to
activate profiling on a administrator session.
closesodoo/odoo#75967
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
- if the alias is configured for the team, the action helper
with the link of that email will be displayed. Otherwise, it will
simply indicate user to either create lead manually, or to configure
email alias.
- This commit improves action helper for the tags and adds
sample data for better onboarding.
- This commit adds action helper on action 'sale.action_invoice_salesteams'
for better onboarding.
- This commit updates the action helper for 'CRM > Customers'
and 'Contacts' actions and make the helper message consistent.
TaskID-2582208
Part-of: odoo/odoo#73530
Make the default height of the editor quite smaller in res.partner view.
To avoid "breaking" the design and prevent scrollbars
task-2586137
Part-of: odoo/odoo#75863
Since c53724ebc3, the contact image in contact kanban view for a shipping
or delivery contact type would be the regular placeholder instead of the truck
or money image.
Now, by using the new avatar system, we restore that behavior.
+ we also use that new avatar system to cleanup some code in the kanban in the
form view (`child_ids` field in `Contacts & Addresses` tab). That part was
working fine but wasn't using the new system and needed extra code.
closesodoo/odoo#75709
X-original-commit: a97cbdd033391af75628bed50accfef11c73687c
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Signed-off-by: Romain Derie <rdeodoo@users.noreply.github.com>
Purpose
=======
Review the UX of the 2-factor authentication flow in order to make it more clear
and easy to use.
Specifications
==============
This commit applies multiple rewording of instructions, button, etc. Tests have
been adapted accordingly.
It also adds an 'invite to use two-factor authentication' flow that will
send an email to the selected used to redirect them their account security
settings.
- If portal is not installed yet, the user is redirected to his account security
settings in backend.
- If portal is installed, the user is redirected to /my/profile if them are
portal user. Otherwise, the redirection is still done at backend side.
As the backend view of auth_totp wizard is used at frontend side, copyclipboard
widget has to be rebuilt at frontend side (click event, style etc..).
As API key section is now displayed only on debug mode, test urls have been
adapted accordingly.
Task-2487630
Part-of: odoo/odoo#71142
This commit let you choose a Point Relais® as your shipping address.
It instantiates the Widget only.
It will not handle the print label.
Part-of: odoo/odoo#75281
Before commit[1] there was only one field image_1920 in group with 'colspan=1'.
In commit[1] a new invisible field avatar_128 is added in this group. But due
to this an empty 'tr' element is added in view and it breaks the view.
With this commit we move avatar_128 field outside of the group and placed it
with other invisible fields, so it doesn't break view. And also adding comment
field in new group because it is html field now so it displays nicely.
Followup of odoo/odoo@cb4f597410 and odoo/enterprise@971f3ed9ae
commit[1]: c53724ebc3
LINKS
Task Id-2558824
PR odoo/odoo#74795
PR odoo/enterprise#2011
Automatically activate `group_multi_currency` if there is more than one active currency,
deactivate it if there is only one active currency
retain sale/pos feature of activating group_product_pricelist
adapt tests accordingly
Task 2610735
closesodoo/odoo#74396
Related: odoo/enterprise#20106
Signed-off-by: Laurent Smet <smetl@users.noreply.github.com>
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
PURPOSE
=======
We want to be able to authenticate our servers with a certificate
for the entire domain name instead of using a username and a password.
SPECIFICATIONS
==============
Add 2 fields on the `ir.mail_server`, which are
- the SSL certificate
- the SSL private key
When we uploaded both files, we use them to authenticate the client of
the SSL connection.
Add 2 options on the Odoo binary, so we can provide the filenames of both
files (like we do for the SMTP username/password).
SETTINGS
========
Note that this type of authentication doesn't work locally for Microsoft
office 365. It seems like Microsoft is blocking non-static IP address
(not able to ping the host locally, but it works on the server).
The host name of the server is defined in the MX DNS record. Then, on
Office 365 you must create an SMTP relay based on a certificate and not
based on a hard coded IP address. The certificate must be valid for your
domain name.
e.g.
Host: openerp-org.mail.protection.outlook.com
Port: 25
Username: <keep it blank>
Password: <keep it blank>
Security: STARTTLS
Email: admin@odoobe.com
New Python dependence
=====================
The standard SSL python library only takes a filename to the certificate
/ private key.
But, we do not want to use attachments and take the full path to the
file (in the filestore) or to create temporary file.
So, we need to use a new library "PyOpenSSL" which allows you to load
a certificate / private key from a byte array.
To make this library work with SMTPLIB we use a wrapper developed
in urllib3 (PyOpenSSLContext).
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
Currently, empty screen of the settings is not very beautiful
without proper action helper and decoration. It displays only
'no record found' string when no setting is found for the
search string.
With this commit, we make the no content helper for the settings
consistent with other apps. It now displays empty folder and
proper helper message.
TaskID-2588387
closesodoo/odoo#74570
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
This commit aims to improve the 'Contact Tags' sections of the 'Contact'
module by adding a color picker (on the list and the form view).
SPECS
- Add a color picker on the 'Contact Tags' sections (i.e: list and form view)
- Add a placeholder for the 'name' field of the form view.
- Update the label of the 'color' field.
LINKS
Task id: 2608410
closesodoo/odoo#74033
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Define `data-hotkey` on most used action buttons.
For the modals, the following keys are dedicated for "special"
actions:
- Alt+G: add
- Alt+V: save
- Alt+Z: cancel
closesodoo/odoo#73275
Taskid: 2588233
Related: odoo/enterprise#19464
Signed-off-by: Kevin Baptiste <kba@odoo.com>
With commit odoo/odoo@83ffe8b we ensured that while creating a child contact, it
takes language from its parent by default if any, otherwise falls back to DB
lang.
The fix was done with help of `default_get` method. However after merge of
odoo/odoo#55995 `default_get` is now called through the onchange for o2m
fields. Now we don't get the default value of `parent_id` here which
re-introduced the bug.
This commit fixes the behavior by setting the language from onchange
while creating the child contact and thus (once again) making the
language from parent contact prevail on DB lang.
We also re-use default_lang coming from parent in form view, which partially
reverts odoo/odoo@83ffe8b .
TaskID-2416922
closesodoo/odoo#72960
X-original-commit: 4d9b85bd07e487e7843dc458332dede36c2889c2
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Scenario:
- Go to Contacts
- Open a contact
- In "Contacts & Addresses" tab, add a child and configure its image
- Once child form is saved, image of the child is not displayed in the
kanban view used for children
It will only appear once the main contact form is saved.
The same issue occurs when editing a child image.
The new image will only be shown once the main contact form is saved.
The issue at the creation is due to the fact the function retrieving
the URL of the image tries to generate the URL from record id, which
is not set. It should use raw data of image in this case.
For the edition, it is due to the fact that the child form is changing
image_1920 field while the kanban view used for children is displaying
image_128 field. image_128 is a related field to image_1920, but it
does not appear in the child form. It is therefore not recomputed (by
onchange) when image_1920 is modified. Adding it in the child form view
solves the issue.
opw-2516188
closesodoo/odoo#72375
X-original-commit: 7eac23573c77415c84cb9c234c8063d4c68be425
Signed-off-by: Anh Thao PHAM <kitan191@users.noreply.github.com>
Currently, in My Profile all the users have access to language settings
even when the user does not have the rights to activate/deactivate
languages. Also, the language setting list is not user friendly which
needs to be imporved.
so in this commit hide "More Languages" button in My Profile for users
that does not have the rights and the language settings list is
improved by adding seperate buttons to "activate", "disable" and "update" a
language.
PS: groups does not work on My profile so added the compute field based
on group.
closesodoo/odoo#69292
Taskid: 2466771
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
Entry count can be useful to estimate the size on disk of a profile
file.
The entry count would be quite expensive to compute and thus should be
stored. It is impossible to use a compute stored in this case, since
ir.profile shouldn't have any "business" logic upon insert.
closesodoo/odoo#71673
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Currently, to identify records based on state, in most of the list views we
use 'badge' widget on 'state' fields which can be decorated as needed.
However, for the list views, best approach would be to also add list
decoration along with 'badge' widget so that the whole line is colored
and it's easy for user to 'scan' the information
This commit improves the behavior adding decorations to several list-views.
To check the list of changes, kindly see the task specification
Apart from that, this commit also improves label of the module state from
'Not Installed' to 'Uninstalled'.
TaskID-2527119
closesodoo/odoo#71367
Related: odoo/enterprise#18587
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
What are the steps to reproduce your issue ?
1. Create new contact
2. Set compagnie type as Individual
3. Set address type as Contact
What is currently happening ?
Address label is shown as "Company Address"
What are you expecting to happen ?
Show "Address" in case of Individual company type
opw-2504878
closesodoo/odoo#71618
X-original-commit: 0248f1735c31d039446f2a7f25148dcb99c09153
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Achraf <abz-odoo@users.noreply.github.com>
The profiling tools can be useful to profile a test of some execution
point but this is not convenient to identify a problem on a running
instance.
With this commit, an option available in the debug menu allows to add a
flag on the user sessions to enable profiling of all requests. Each
request will be saved in a different 'ir.profile' entry, but will be
grouped under the same session.
The profiling can be activated on all sessions, even for a public user,
but only if profiling is enabled on the database globally.
This commits also adds a speedscope view to visualize saved results in
the web client.
closesodoo/odoo#66590
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This commit adds tooling to profile performance and save execution by
saving stack traces and queries to a file/database in specific format.
----------
Collectors
----------
For now, three different profiling modes (aka Collectors) are available
even if a last once should be introduced by @Gorash to profile qweb
execution.
- SQLCollector (or 'sql'): Saves the current stack trace and the query
every time Cursor.execute() is called. Any query executed on the thread
will be collected, no matter the cursor.
- PeriodicCollector (or 'traces_async'): Saves the stack trace every
'interval' seconds using a parallel thread to profile the caller thread.
The python implementation was optimized to minimize impact on
performance while remaining portable and easy to enable/disable
inside a odoo execution. Higher the frequency (lower the interval),
more impactful the profiling will become on the execution and increase
memory usage. From last experiments, 1ms looks to be a good minimum for
short executions.
- SyncCollector (or 'traces_sync'): Saves the stack trace every function
call/return. This collector is obviously quite impactful on performance
and can quickly overload the memory for long executions, but this is
quite useful to understand the precise path followed by some short
executions. Any time related information will be almost irrelevant with this
collector.
A base Collector defining minimal collectors features can easily be
extended to create custom collectors if needed.
----------------
Profiler & Usage
----------------
Collectors are not supposed to be used by themselves, but should be
given to a Profiler. The Profiler will synchronize collectors starts and
stop, and manage saving them to a file of in a ir_profile in the
database.
Exemple of usage:
```
with Profiler():
do_stuff()
```
This simple example will use the default collectors (sql and
traces_async) and save them to the database. The database is defined
automatically from current_thread 'dbname' if available.
Example of usage:
```
with Profiler(collectors=['sql'], db=False, path=/home/user/logs/do_stuff_profile/{time}):
do_stuff()
```
This more complex example disable the default behavior consisting
to save to the database, gives a path where the profile will be saved
and specify to only use the 'sql' collector. Note that
collectors=[SQLCollector()] would have the same behavior since
Collectors can be either a Collector instance or a string describing the
desired collector. This allows to define custom params for the
collectors and use custom collectors if needed.
Note that it is always possible to get results after execution without
saving it since they are available on the profiler.
```
with Profiler(collectors=['sql'], db=False) as p:
do_stuff()
print(len([None for entry in p.collectors[0].entries if ...]))
```
Profiler will also save the stack below the profiler start point, and
collectors will only collect the part of the stack over this stack.
This is a good way to reduce collectors CPU and memory usage.
Collected entries will be saved as follows:
```
[{
'start': 2.0,
'context': {},
'stack': [
['path_to_file', lno, 'func_name', 'line_content'],
...
],
},
...
]
```
SQLCollector will add three additional keys on each entry:
- query (query without parameters)
- full_query (mogrified query with parameters)
- time (the 'exact' execution time of the query)
----------------
ExecutionContext
----------------
A last tool, ExecutionContext, allows to define some context on some block of code:
Example of usage:
```
def process_modules(modules)
for module in modules:
with ExecutionContext(module=module): # note the 'not linter frienldy but still convenient' 2 spaces indentation
do_stuff(module):
```
This context will automatically be added in the stack as a virtual frame between
process_modules and do_stuff in order to split do_stuff from one single frame to
one frame per module.
----------
Speedscope
----------
The saved data are in a simple json format easy to analyze, but can't be visualized in
speedscope as they are. A utility class `Speedscope` can be used to generate a format
readable by speedscope. The used format is actually the format defined by speedscope,
meaning that all features should be available using it.
The output format is evented, meaning that we need to transform a list of samples
(a list of stack) to a list of event (going in/out a frame).
This is the main task of the Speedscope, as well as combining samples from different
sources, to display SQLCollector and PeriodicCollector results mixed together.
When stored on an ir_profile, the default speedscope generation can easily be generated
with the speedscope computed field.
This class can be used as it is but will mainly be useful for the next commit.
Special thanks to @rco-odoo for the in depth review and @Gorash for support.
Description of the issue/feature this PR addresses:
It is currently quite difficult to differentiate users. Most of the time, people
don't take the time to upload an actual avatar so everybody looks the same. This
PR generates a custom avatar with the users initials and random color to
differentiate them. For res.users, res.partner and hr.employee, image fields now
hold the binary image and avatar are used to show the image or svg.
Current behavior before PR:
Avatar had only random colors and was being saved in database, being inefficient
Desired behavior after PR is merged:
A new mixin defines image fields and in case no image is set, it generates an
SVG image with the user's initials and random color.
closesodoo/odoo#69819
Task: 2404630
Related: odoo/enterprise#18199
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
In order to be able to add a button to activate contextual merge action on the
target model, the ir.model form view had to be adapted to add the header.
Task ID: 2459416
ENT PR: odoo/enterprise#16975
Currently, clicking on the "upgrade" button from Apps in community
opens the corresponding page in the same tab. As a consequence, the
user completely leaves his database and it can quickly become tedious
to get back to it.
so in this commit, clicking on the 'upgrade' button should open the
corresponding webpage in a new tab
closesodoo/odoo#70636
Taskid: 2513785
Signed-off-by: Arnaud Joset <arj-odoo@users.noreply.github.com>
Currently, across Odoo, there are around 40+ many2one fields defined with a
'selection' widget. Since the many2one widget has options to limit record
creation and opening, there is no reason to define a many2one field with a
selection widget. The selection widget does not allow for searching, and is
limited to 100 records.
PURPOSE
to update the definition of any many2one on which we applied a 'selection'
widget, and instead use the standard many2one widget with disabled
opening/creation instead.
after this commit,
for each many2one field defined with widget="selection", widget="selection" is
replaced with options="{'no_open': True, 'no_create': True}"
Task : 2476488
closesodoo/odoo#68387
Related: odoo/enterprise#17316
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Following the recent reorganisation of the documentation in 12.0+,
the majority of the documents have been moved and their old links are no longer valid.
Some redirection rules will soon be deployed, but those rules might be dropped in some years
and we want the links to still work, which is why we still replace the links to the new ones.
FW-Port of odoo/odoo#70675 (13.0)
closesodoo/odoo#70920
X-original-commit: bc9c1eef538ba6095e74c19d5d9ed9e01625ec7c
Related: odoo/enterprise#18361
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
Currently, in the res.company form view, name field
is extremely short.and in mobile view name field
is hidden by other element because of placeholder
is too long.
after this commit the length of name field is should
be as long as the res.partner one and change the
placeholder (e.g. My Company) instead of (e.g. My Awesome
Company) for idle mobile look.
closesodoo/odoo#70241
Taskid: 2520268
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
- Currently all menus are out of order in app switcher.
- For example, Sales app is 16 menu away from Accounting,
Social Marketing app is 25 menu away from Email Marketing, etc.
So, all menus should be reordered.
- This commit will reorder the menus of the app switcher in order to reduce
the distance between correlated applications,
and bring the most common apps upward.
- And in this commit we have left gap of 5 subsequent sequence for further new menus.
PR: #69984
TASK ID: 2513082
Related: odoo/enterprise#17989
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
According to the app-store guidelines we are not allowed to sell from
the iOS app.
Our updates are currently blocked because of this.
In the rush, we have no choice but to hide the 'learn more' button
displayed on each app. This redirects the customer to the documentation
page where you can find subscription links.
And this is unfortunately not allowed by Apple...
Source: https://developer.apple.com/app-store/review/guidelines/#business
Related PR:
https://github.com/odoo/mobile-ios/pull/102https://github.com/odoo/mobile-ios/pull/103
Task-id: 2483253
X-original-commit: a812660e284068802e24d9dd422e4dd15341e716
* QWeb bodies should be markup-safe so `0` should always be
markup-safe.
* `head` is qweb-rendered so the same.
* The `json` pseudo-module in qweb templates is `json.scriptsafe`,
which should be markup-safe.
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
Aktiv Software