Commit Graph
735 Commits
Author SHA1 Message Date
Jeremy KerstenandOkan SUMER be43710ab8 [IMP] tools: xpath - support mode inner for position replace
This commit adds a new attribute mode for position 'replace' to the xpath feature.

This mode can take 2 values:

- 'outer' (default mode if not provided) that will replace the sibling target
- 'inner' that will preserve the sibling target

If base arch is:
```html
<p>
   <field name='x'>yyy</field>
</p>
```

`<field name='x' position="replace" (mode="outer")>zzz</field>`
```html
<p>
   zzz
</p>
```

`<field name='x' position="replace" mode="inner">zzz</field>`
```html
<p>
   <field name='x'>
      zzz
   </field>
</p>
```

Mode inner is useful for theme or render flat content, but not recommanded to
be used in page editable since we ignore the inherit-branding part until now.

task-2172208

closes odoo/odoo#48679

Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Co-authored-by: Okan SUMER (osu) <osu@odoo.com>
Co-authored-by: Jérémy Kersten <jke@odoo.com>
2021-07-29 07:04:55 +00:00
Nicolas Bayet 4600086e7a [IMP] web_editor: move odoo-editor into odoo
The code of the Odoo editor was on another repository
and that created unnecessary overhead. This commit move
the code inside Odoo and slightly change the folder
structure.

Related PR:
saas-14.3: #73345
saas-14.4: #73348

closes odoo/odoo#73272

Master: #73272
Signed-off-by: Antoine Guenet <Zinston@users.noreply.github.com>
2021-07-08 15:31:45 +00:00
Thibault Delavallée 28d6468bc4 [FIX] tools: better support 'almost empty' content from editor
It seems quite easy to have a void content in the currently new editor
that actually holds a lot of undesired information, notably a font
tag. This is annoying when having behavior based on an html field
being empty or not. In this commit we therefore improve definition
of an 'empty' html field.

Task ID-2532529
PR odoo/odoo#71793
2021-07-06 13:30:47 +00:00
Mohammed Shekha d8134350b8 [IMP] base,stock,web,website_sale: replace create_text with add-label
before this commit: create_text was passed in node options and due to which it
was not parsed by translate.py, pass add-label as attribute on field so that
translate.py parse it and it is translated.

after this commit: create_text will be passed as field attribute instead of
node options.

task-1923433

closes odoo/odoo#59713

Related: odoo/enterprise#19418
Signed-off-by: Simon Genin (ges@odoo) <ges@odoo.com>
2021-07-05 11:11:21 +00:00
Romain Derie 3ea17f597b [FIX] translate: get class attribute safely
58d3b670221b3 was not correctly forward ported, it misses the `''` fallback part
of the original commit dc20ab9c02

Without this, traceback is shown.
Step to reproduce:
 - Open HTML Editor (or edit a backend view)
 - Add an input with no class but a value attribute (no type or type text)

closes odoo/odoo#73057

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-07-01 09:27:20 +00:00
Sébastien Mottet (oms) dc20ab9c02 [FW][FIX] translate: get class attribute safely
The 'class' attribute was not safely accessed using get. Since this attribute is not always
present the condition evaluation failed in some cases.

task-2276724

closes odoo/odoo#72894

X-original-commit: 58d3b670221b37c5c4c67ee53fbc545f6fb70395
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-06-29 07:18:50 +00:00
bbh-odooandxavierbol 842845bf46 [IMP] project,web: update the translation for some terms
It seems some terms are no translatable so purpose of the task is
to find those terms ans update to make them translatable.

So in this commit, translate the text attribute when the node is field
tag and this node contains widget='url' in its attributes. and
also make the project name as translatable.

closes odoo/odoo#71406

Taskid: 2487710
Signed-off-by: LTU-Odoo <IT-Ideas@users.noreply.github.com>
Co-authored-by: xavierbol <xbo@odoo.com>
2021-06-23 05:02:47 +00:00
cd403480f9 [IMP] web_editor, mass_mailing, *: add shape clipping option on images
*: tools

The ImageOptimizer option can now set shapes on images to use those
shapes as clip path and background. It uses a flexible path so that the
shape will always fit the image proportionally. We also added an html
file alongside a javascript file to semi-convert the shape from
illustrator into a usable shape for this usecase (the clip path must
have values between one and zero therefore we must do some computation
before the shape is ready to use).

Thanks to Samuel for this specific shape-converter tool and other
technical points about svgs and clip-paths.

Thanks to Mehdi for remaining development post-testing and post-reviews
and for the many fixes.

Thanks to Brieuc for the actual shape SVG files.

Part of https://github.com/odoo/odoo/pull/69179
task-2327045

closes odoo/odoo#69179

Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: xO-Tx <mou@odoo.com>
Co-authored-by: Brieuc-brd <brd@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
2021-06-21 16:31:07 +00:00
xO-Tx 79685b5bdf [FIX] tools: keep image option related attributes
- Change an image option in mass mailing editor (e.g. Quality)
- save
- edit -> The option can't get the new applied value.

The body_arch's field used in mass mailing editor is
sanitizing attributes and as a consequence, option related data
attrs are removed on save.

task-2327045

closes odoo/odoo#72311

X-original-commit: 7c5666611363a73bc6fef070565039cf065a0176
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2021-06-17 16:26:16 +00:00
Denis Ledoux b06e4454d5 [FIX] registry: check_foreign_keys, constraint names are limited to 63 chars
When computing the foreign key name,
`check_foreign_keys` didn't take into account the limit of 63 characters
for constraint names.

Because of this, some constraints were dropped and recreated
over and over while they were correct, during install and upgrades.

For instance, when installing `base`
when adding the foreign key for which the name was computed
`base_partner_merge_automatic_wizard_res_partner_rel_base_partner_merge_automatic_wizard_id_fkey`
Postgresql created the constraint under the name
`base_partner_merge_automatic__base_partner_merge_automatic_fkey`
and therefore, as the name did not match,
the constraint was dropped and re-created.

closes odoo/odoo#72234

X-original-commit: 43a4738ebf8a74a389b99f8f58330b3044beaa0c
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2021-06-16 13:15:26 +00:00
Julien Castiaux 5815ce7753 [FIX] base: some fixes for ir.ui.view
Task: 2463632
2021-06-11 17:09:05 +00:00
dht-odoo 57261b100f [IMP] base, mail, portal: improves is_html_empty method
Before this commit there were few cases in which
is_html_empty method was not working as our expectation.
In cases such as "<p class=""><br/></p>", "<p id=""><br/></p>"
and many other cases.

In this commit we improves regex of is_html_empty method.
Mow from this commit all kind of attributes will be consider
in this method.

In this commit we also have added is_html_empty in mail template, portal
values and in report values also for rendering templates.

Task id: 2499504

X-original-commit: fd0a05f2955b9f7e9ae7233afebfd6240c9244dd
2021-06-07 05:21:40 +00:00
Xavier-Do 245029c02a [IMP] base, profiler: add entry count
Entry count can be useful to estimate the size on disk of a profile
file.

The entry count would be quite expensive to compute and thus should be
stored. It is impossible to use a compute stored in this case, since
ir.profile shouldn't have any "business" logic upon insert.

closes odoo/odoo#71673

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-06-03 14:24:50 +00:00
Xavier-Do 73ac78f072 [IMP] profiler: remove parameter 'path'
Even if only accessible when writing some custom code, the path option
of Profiler looked a little dangerous from a security point of view,
since this option allows to write the result of the collector anywhere.
This shouldn't be a problem, since Profiler shoudn't be accessible
inside a safe_eval() and the path param cannot be user defined, but this
is still a risk.

An alternative option would have been to give a file descriptor instead,
so that the user must have access to `open()`, which is unlikely inside
a server action.  This alternative has some disadvantages, though,
because the file descriptor should be given when creating the profiler,
or at least before `__exit__()`, removing the possibility to add the
number of entries in the filename.

This commit proposes another solution: add a generic way to output the
profiler as a json text.  The result can thus be saved the way the
developer chooses.  A utility method format_path() will help to format
path the same way the profiler did before this change.
2021-06-03 14:24:50 +00:00
Xavier-Do 3b861dfe7f [IMP] http: better request context manager
The usage of an additional optional context manager leads to the usage
of ExitStack. Unfortunately, this changes the initial format quite a
lot, decreasing readability and adding a loop on context managers, even
if there should be only one most of the time (request).

This commit proposes to a nesting utility for the profiler, allowing to
nest another context manager inside a profiler.

This solution allows an easier integration into http.py, avoids to
manage the "enter_context" case when recovering the init stack and can
be useful in other cases.
2021-06-03 14:05:01 +00:00
Xavier-Do 4444475ef4 [ADD] base, core: built-in profiling tool in Odoo
This commit adds tooling to profile performance and save execution by
saving stack traces and queries to a file/database in specific format.

----------
Collectors
----------

For now, three different profiling modes (aka Collectors) are available
even if a last once should be introduced by @Gorash to profile qweb
execution.

- SQLCollector (or 'sql'): Saves the current stack trace and the query
every time Cursor.execute() is called. Any query executed on the thread
will be collected, no matter the cursor.

- PeriodicCollector (or 'traces_async'): Saves the stack trace every
'interval' seconds using a parallel thread to profile the caller thread.
The python implementation was optimized to minimize impact on
performance while remaining portable and easy to enable/disable
inside a odoo execution. Higher the frequency (lower the interval),
more impactful the profiling will become on the execution and increase
memory usage. From last experiments, 1ms looks to be a good minimum for
short executions.

- SyncCollector (or 'traces_sync'): Saves the stack trace every function
call/return. This collector is obviously quite impactful on performance
and can quickly overload the memory for long executions, but this is
quite useful to understand the precise path followed by some short
executions. Any time related information will be almost irrelevant with this
collector.

A base Collector defining minimal collectors features can easily be
extended to create custom collectors if needed.

----------------
Profiler & Usage
----------------

Collectors are not supposed to be used by themselves, but should be
given to a Profiler. The Profiler will synchronize collectors starts and
stop, and manage saving them to a file of in a ir_profile in the
database.

Exemple of usage:
```
    with Profiler():
        do_stuff()
```

This simple example will use the default collectors (sql and
traces_async) and save them to the database. The database is defined
automatically from current_thread 'dbname' if available.

Example of usage:
```
    with Profiler(collectors=['sql'], db=False, path=/home/user/logs/do_stuff_profile/{time}):
        do_stuff()
```

This more complex example disable the default behavior consisting
to save to the database, gives a path where the profile will be saved
and specify to only use the 'sql' collector. Note that
collectors=[SQLCollector()] would have the same behavior since
Collectors can be either a Collector instance or a string describing the
desired collector. This allows to define custom params for the
collectors and use custom collectors if needed.

Note that it is always possible to get results after execution without
saving it since they are available on the profiler.

```
    with Profiler(collectors=['sql'], db=False) as p:
        do_stuff()
    print(len([None for entry in p.collectors[0].entries if ...]))
```

Profiler will also save the stack below the profiler start point, and
collectors will only collect the part of the stack over this stack.
This is a good way to reduce collectors CPU and memory usage.

Collected entries will be saved as follows:

```
    [{
        'start': 2.0,
        'context': {},
        'stack': [
            ['path_to_file', lno, 'func_name', 'line_content'],
            ...
        ],
    },
    ...
    ]
```
SQLCollector will add three additional keys on each entry:
- query      (query without parameters)
- full_query (mogrified query with parameters)
- time       (the 'exact' execution time of the query)

----------------
ExecutionContext
----------------

A last tool, ExecutionContext, allows to define some context on some block of code:

Example of usage:
```
    def process_modules(modules)
        for module in modules:
          with ExecutionContext(module=module): # note the 'not linter frienldy but still convenient' 2 spaces indentation
            do_stuff(module):
```

This context will automatically be added in the stack as a virtual frame between
process_modules and do_stuff in order to split do_stuff from one single frame to
one frame per module.

----------
Speedscope
----------
The saved data are in a simple json format easy to analyze, but can't be visualized in
speedscope as they are. A utility class `Speedscope` can be used to generate a format
readable by speedscope. The used format is actually the format defined by speedscope,
meaning that all features should be available using it.

The output format is evented, meaning that we need to transform a list of samples
(a list of stack) to a list of event (going in/out a frame).
This is the main task of the Speedscope, as well as combining samples from different
sources, to display SQLCollector and PeriodicCollector results mixed together.

When stored on an ir_profile, the default speedscope generation can easily be generated
with the speedscope computed field.

This class can be used as it is but will mainly be useful for the next commit.

Special thanks to @rco-odoo for the in depth review and @Gorash for support.
2021-06-02 07:47:48 +00:00
Ivan Yelizariev 00a8f2457f [FIX] core: fix formatting issues for --test-tags docs
`./odoo-bin -h` prints unnecessary spaces between sentences.

---

task-2431630
Finetuning of #71130

closes odoo/odoo#71364

X-original-commit: fcc60e219fd024394f8642509a20755f964546b6
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2021-05-27 17:30:36 +00:00
Ivan Yelizariev 691fa54cc8 [IMP] core: clarify docs about config --test-tags
Technical name is not added to test_tags since https://github.com/odoo/odoo/commit/95b4f2ab4b5698ab3a28c9c35ac8da6fb6def983

at_install tag is added by default since introducing @tagged decorator: https://github.com/odoo/odoo/commit/b356b190338e3ee032b9e3a7f670f76468965006

Clarify how special tags at_install/post_install work.

Also, add dots for @tagged doc, because otherwise we have a mess in sphinx docs.

---

task-2431630

closes odoo/odoo#71329

X-original-commit: e4bf1e7dca2a9ef6dabf3f201d086ae77ff993d9
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2021-05-27 11:32:05 +00:00
Xavier Morel a1f45aab0e [FIX] core, mass_mailing: preserve markup status in prepend_html_content
In `prepend_html_content`, if body `html_body` and `html_content` are
`Markup` objects, the processing of `html_content` through `re.sub`
will strip away the `Markup` flag, leading to the later concatenation
of the body and content first escaping the content (to safely convert
it to a `Markup`) before performing the concatenation proper.

As a result, the `preview` of a `mailing.mailing` would be
double-escaped, and the markup hiding it would instead be printed as
part of the body.

Assume that the type of `html_content` is str-compatible, and
immediately rewrap the value after processing it through
`re.sub`. This should do nothing if `html_content` was an actual
`str`, and will re-flag it as a `Markup` if it was one.

Also change the finding of the insertion point to use `re.search`, I
don't understand why this uses `finditer`, it just makes the code more
complicated.

closes odoo/odoo#71090

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-05-20 10:37:09 +00:00
nounoubensebia 171eea3fae [IMP] mass_mailing[_sms], tools: make mass mailing form focus on body
Make the email body take the entire space to avoid having some wasted space,
this will also make the user have more focus when designing an email.

Revamp the settings notebook page in order to give more clarity to the user,
and move some fields from the main form have been moved to this section to
have more space in the bottom for the email body.

In the mailing form, when the mailing is sent or is being sent, set fields
which are no longer useful for the user to change to readonly mode.

Add a wizard that enables the user to schedule a mailing, the schedule field is
still kept in the form for the user to be able to change the date when the
mailing is in the queue (if they want to send it sooner).

Display an action helper-style content when the email is empty, because,
currently, the user is left with a big white screen when the email has no
content which is not desirable.

Update the html_empty function to take into account style attributes to better
match the editor's void content.

Hide A/B testing fields from SMS mailing form view as these are not supported
for SMS marketing.

Task-2469409

closes odoo/odoo#68882

Ent-pr: https://github.com/odoo/odoo/pull/68882
Related: odoo/enterprise#18391
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-05-20 09:40:40 +00:00
std-odoo a757cab857 [IMP] mail: allow to force the FROM headers of the outgoing mail server
Purpose
=======
We want to be able to force the FROM headers when we sent email in
SMTP. So we can avoid the emails to be considered as spam.

Specifications
==============
This is done with 2 system parameters.

If the system parameter `mail.force.smtp.from` is set we encapsulate all
outgoing email from with the given value.

If the previous system parameter is not set and if both
`mail.dynamic.smtp.from` and `mail.catchall.domain` are set, we
encapsulate the FROM only if the domain of the email is not the same as
the domain of the catchall parameter.

Otherwise we do not encapsulate the email (same behavior as before this
commit).

Task 2367946
See odoo/odoo/pull/61853

closes odoo/odoo#70980

X-original-commit: 08a561505b92d23c4c4ec4094b0cee209ece8753
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-05-18 13:35:11 +00:00
Ivan Yelizariev a4424c9058 [REM] delete UFO comment
Apparently, it was added by mistake 7 years ago in https://github.com/odoo/odoo/commit/e9d047e6119d8065a8d31a7d3b374630ab90c40d

closes odoo/odoo#70892

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-05-17 15:12:32 +00:00
Raphael Collet c6f1e4f0d7 [IMP] core: save memory of empty dicts
Use `frozendict` for dicts that often remain empty on models, like
_inherits and _depends, and also make `frozendict` more compact in
memory.

On a registry with 296 modules, this saves 635 kilobytes of memory,
which is about 6% of the registry's memory footprint.

closes odoo/odoo#70402

Related: odoo/enterprise#18142
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-05-10 14:29:43 +00:00
Raphael Collet 02b8c687e5 [IMP] core: squeeze registry.field_depends
On a registry with 296 modules, this saves 1 megabytes of memory, which
is about 8% of the registry's memory footprint.
2021-05-10 14:29:07 +00:00
Raphael Collet f7c9cb2b60 [IMP] core: share fields by not duplicating them on model class
Also speed up the basic setup of fields that are always duplicated
top-level (on the model's registry class).  This saves time and memory,
as we also discard field.args and field._base_fields on toplevel fields
(those values are no longer useful after setup).
2021-05-03 12:33:29 +00:00
Xavier Morel 3786e1dcb0 [FIX] core, mail: mark HTML manipulation results as markup-safe.
* The result of `plaintext2html` is fully controlled and
  markup-safe (the first thing we do is escape the input).
* For `append_content_to_html`, we assume the inputs are HTML and the
  output is thus always properly HTML.

  Alternatively, we may want to `Markup("%s%s") % ...` and require the
  inputs to be properly marked? That seems like a good idea.
* In `_replace_local_links`, applying re.sub will strip out the markup
  mark, so store it and reapply it on output if necessary.

  That one is a big gnarly, because if the input to ustr is
  markup-safe bytes (e.g. qweb rendering output) then the output is a
  Markup object, but if the input is str then the output is str, so we
  need to check before and after unless... we update ustr to check for
  subclasses instead of exact type?
* In `_prepend_preview` the issue is similar to that of
  `append_content_to_html`, though in this case we should *not* trust
  the input, so we can flag the "parent document" as Markup and format
  the preview bit in.
* And since we're marking mail's jinja output as safe, do the same for
  web and iot.

  Sadly there doesn't seem to be any hook for doing that at the
  environment level of jinja, so every `Template.render` site has to
  be marked.
2021-04-29 05:34:20 +00:00
Xavier Morel 01875541b1 [CHG] core, web: deprecate t-raw
Add a big fat warning when the qweb compiler finds a `t-raw`.

`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).

Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.

Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.

`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.

Also mark a bunch of APIs as markup-safe by default

* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
  by the trip through the database) and if the field is unsanitised
  the injection is very much intentional, probably. Note: this
  includes automatically decoding bytes as a number of default values
  & computes yield bytes, which Markup will happily accept... by
  repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
  non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
  the input is markup-safe, this means we should not need to escape
  values fed to `nl2br`, but it doesn't hurt either.

Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.

Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).

However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.

For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).

Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.

`t-out`
=======

`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.

Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.

Attributes handling
===================

There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.

This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.

This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.

A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.

The most visible instance of this was the `snippet_options` template,
specifically:

    <t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
    <div id="so_content_addition"
        t-att-data-selector="so_content_addition_selector"
        t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
        data-drop-in=".content, nav"/>

Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.

The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).

That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).

Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.

Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.

fixup! [CHG] core, web: deprecate t-raw
2021-04-29 05:34:19 +00:00
Nicolas Lempereur 76741baf40 [FIX] mail.py: sanitize do not remove file icon
Scenario:

- add a file in mass mailing editor => an icon is added linking the file
- save
- edit => the icon is replaced by a generic icon
- save
- send mail => no icon is shown in email received

The system is using eg. `data-mimetype="pdf"` to show the icon, before
12.0 this worked but in saas-12.3 the system uses mailing.mailing
body_arch's field that is sanitizing attributes and remove it on save,
so the icon is only working in received email if you save one and only
one time after adding file inside the mass mailing.

opw-2474053 (ticket for similar issue in 14.0)

closes odoo/odoo#70057

X-original-commit: 7e14515b84d8081557517df7e9dc7d8c0d17f2be
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
2021-04-28 17:13:12 +00:00
Naglis Jonaitis e59b032f06 [FIX] core: fix <act_window> deprecation warning
Without the f-string literal the XML ID is not printed.

closes odoo/odoo#69419

X-original-commit: d0cbe52c111af923f0df5d0d231b164a150bf2db
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-04-16 15:57:07 +00:00
Francois (fge) 27da86a138 [FIX] base: Catastrophic Backtracking in js_transpiler
This commit is to fix two problems when transpiling an @odoo-module file.

1. When a file exports an object with a long name, the regex EXPORT_FROM_RE will cause catastrophic backtracking. The regex will take a long time to resolve.
Example:
```js
    export {a, aReallyVeryLongNameElement};
```

2.  When a file exports an object without ending the line with ';'. The regex EXPORT_OBJECT_RE will remove spaces and line breaks that follow this kind of export.
Example:
Input:
```js
    export {a}

    xxx
```

Output:
```js
    Object.assign(__export, {a})xxx
```

Output after this commit:
```js
    Object.assign(__export, {a})

    xxx
```

closes odoo/odoo#68868

Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2021-04-15 13:40:02 +00:00
Tymoteusz Motylewski 31369ff4d3 [IMP] tools: display full path in error message
Show full path to not found addons folder to ease debugging.

Before:
error: option --addons-path: no such directory: '../non-existing'

After:
error: option --addons-path: no such directory: '/home/user/Odoo/non-existing'

This message is local to developer and never displayed to the enduser

closes odoo/odoo#68959

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-04-09 07:36:48 +00:00
Xavier-Do 4044e46861 [IMP] core: make compute order deterministic
CRM install query count can vary from one execution to another, leading
to difficulties when analysing performances evolution.
The main reason for this is that some compute methods were called in
different order. Even if compute order shouldn't have any effect on the
final result, making it well defined will help finding other causes of
non-determinism.

The initial observation was that sorting Environment.fields_to_compute
leads to a fixed number of query when installing crm.

The main cause of non-determinisim is the usage of `set` impacting
Field.compute_value and BaseModel._modified_triggers.
Transforming all these `set` to `OrderedSet` solves the problem.

The query count is now deterministic when installing a database from
scratch, but not when updating a database with -i crm.

OrderedSet is also slightly optimised by using a dict instead of an

closes odoo/odoo#68692

Ordereddict: dict order is deterministic since python3.6
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-04-02 15:29:12 +00:00
Tymoteusz Motylewski f1ea45c402 [IMP] doc: add missing command line options
- fixes some typos
- adds command parameters which are present when running odo-bin --help
- updates some descriptions, e.g. regarding configuration file and data-dir

closes odoo/odoo#67778

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-04-01 15:48:47 +00:00
nounoubensebia ad602bba0c [FIX] calendar_event, tools: appointment time mismatch
add mail_tz to calendar_attendee model in order to make it easier to change the
timezone displayed in the mail template so that we can use it to make the
appointment time shown on the email reminder sent to the attendees consistent
with the time shown on the website page.

Change the invitation mail template and use the mail_tz field to display time
field instead of using the partner's timezone in order to make the time shown
in the invitation consistent with the time shown on the website page.

Remove the get_interval method in the calendar_event model and use standard
formatting tools instead, as this is more conveniant than having a custom
method for formatting dates, For this reason the format_time function located
in tools/misc.py has been modified to be capable of handling timezones in
order to be able to display time in the correct timezone, furthermore, this
function has been added to the rendering context provided in the
mail_render_mixin file to be used in email templates.

see: https://github.com/odoo/enterprise/pull/16204

Task-2451154

closes odoo/odoo#65729

Related: odoo/enterprise#16204
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-03-30 13:17:07 +00:00
Nicolas Lempereur 5ef48dbdff [FIX] translate.py: CSV export correct module name
Correct small typo that the module when exporting translations as CSV
file might have been the wrong module because we used variable from
previous loop.

This did not seem to cause any issue since in this given case in import
we got the module from the part before . in XML ID ({module}.{name})
that was right.

found when working on opw-2439029

closes odoo/odoo#68297

X-original-commit: 62e7b161d39e5f5f6751cf87826fa6f7b729f83d
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
2021-03-24 11:54:02 +00:00
Nicolas Lempereur 5be7af0494 [FIX] translate.py: import model from CSV correctly
When importing transtlation from CSV, we:

- cut at `:` character to get the model
- used a model for types other than model and model_terms

This caused error that do not happen in PO import because:

- the model is before the `,` character
- the imd_model column is constrained to 64 characters, and a name that
  is not a model could be over that size.

With this changeset, the CSV import match better current PO import.

opw-2439029

closes odoo/odoo#68266

X-original-commit: cff91c6acbb7ef6a884d4dc4170574b83ff6e4cb
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
2021-03-23 18:18:21 +00:00
Christophe Simonis 995d5dfe82 [FIX] tools.file_path: correctly handle the addons/ prefix
The tests passed because the path with the prefix is valid at root_path.

closes odoo/odoo#68257

Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2021-03-23 16:32:10 +00:00
Fabien PinckaersandRaphael Collet 085e972e4a [IMP] core: clean up and speed up XML/HTML translation
The former algorithm was recreating a full etree from scratch, while
parsing the content to translate.  Instead, we parse the etree and
update only the content that to be translated.

Before:
| In [2]: %timeit views.invalidate_cache(); views.mapped('arch_db')
| 578 ms ± 7.83 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

After:
| In [2]: %timeit views.invalidate_cache(); views.mapped('arch_db')
| 208 ms ± 2.79 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Reading a view is 2.63x faster: from 504 ms, to 191 ms (sum of all the
1345 views, when installing website_sale).  Large views, like web pages
goes up to 3x-4x faster; small views are ~2x faster.

As views are cached, it only impact the loading to put in cache, but all
XML/HTML fields get the same performance gain.

closes odoo/odoo#68182

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2021-03-23 15:35:47 +00:00
Francois (fge) 9c804aace3 [FIX] base: js_transpiler import legacy module
In a @odoo-module js file, importing a legacy module that did not have
a name respecting the "addon.name" format was not supported.

This commit will support all keys that do not start with a "." or "@".
This commit will also improve some regexes allowing to extract path.

Legacy module name supported:
    import X from "some/path"
    import X from "addon.name"

Legacy module name not supported:
(relative path and @module/path are only used to import @odoo-module js
files)
    import X from "@some/path"
    import X from "./some/path"
    import X from "../../some/path"

closes odoo/odoo#68152

Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2021-03-23 14:15:02 +00:00
wan 2fcfa2cc18 [FIX] tools: support malformed more malformed pdf
We do not want to fail ever if loading the attachment fails.
We still log a warning in every case though.

Some malformed PDFs lead to seeking the wrong bytes while looking for
some parts, raising a ValueError

closes odoo/odoo#68171

X-original-commit: f4cc0579a3e0ee9c117c7dd0f6090edc6e94384d
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: William André (wan) <wan@odoo.com>
2021-03-19 19:41:02 +00:00
Julien Castiaux fb4cf35368 [FIX] base: View error reporting can lack a context
A context is attached to the ValidationError object when elements in the
view arch are broken. This context helps to locate the error in the
source file. When the view processing fails outside of the arch
evaluation, such context is missing.

closes odoo/odoo#68157

Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
2021-03-19 16:13:17 +00:00
Stéphane Bidoul 13b40bff2c [FIX] base: allow using another postgres schema than public
Before 1721ec1363 and fdc4ef97c9, Odoo did work fine when the
user had another default schema than 'public'.
This commit restores this behaviour by searching
for existing objects in the user's current schema,
which is the first in the schema search path and
the one used when no schema is specified when
creating objects.

closes odoo/odoo#68144

X-original-commit: 223781b34afacd1c0c5674d395cece6d472b048c
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2021-03-19 12:42:16 +00:00
Olivier Dony 49429f986a [IMP] tools: simplify and modernize file_open()
- Remove legacy arguments (`subdir`, `pathinfo`) of `file_open()`,
  they were not used anymore and made the code more complicated

- Drop the long-deprecated support for files inside zip archives,
  considering that zipped modules were discontinued a long time ago:
  278ed718e9

- Remove the redundant `_file_open()` method, that should never have been
  called directly anyway

- Add the possibility to filter allowed file extensions, in order to
  avoid opening up access to arbitrary addons files, such as .py files,
  depending on the context of use

- Split up the verification of the file path, to make it accessible
  without actually opening the file, as a separate `file_path()` function.

closes odoo/odoo#68043

Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2021-03-19 11:40:52 +00:00
oco-odoo 56f8c1c048 [FIX] base: tools: float_split_str: avoid traceback when rounding to 0 places and add some test cases
closes odoo/odoo#67970

X-original-commit: 42d409f180381316c97abd0c0ef5a062656e6122
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: oco-odoo <oco-odoo@users.noreply.github.com>
2021-03-16 15:09:51 +00:00
Tymoteusz Motylewski df4015f5f5 [IMP] tools: remove unneded dot from CLI group header
Internationalisation options. :
is now
Internationalisation options:
2021-03-11 18:06:21 +00:00
Martin Trigaux 0fda70aa8e [FIX] tools: properly format error message
To reproduce:
1. take a translatable record with an external id
2. delete the record but keep the ir.model.data (sql)
3. export the translations of the module linked to the orphan
   ir.model.data
-> Error while formating the message

This commit fixes three issues:
- %d instead of %s to construct the string
- wrong order to identify the missing records
- display the external ids instead of the ids (which may not be very
  helpful to debug)

Courtesy of Yannick Brant

closes odoo/odoo#67530

X-original-commit: a2a45539ca4056eb3b3d5d6bafbfb63c10255843
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-03-09 12:11:11 +00:00
Christophe Monniez 97579f9c13 [FIX] packaging: remove dead code that prevent rpm build
Since 9c2ce1f12c2, more files are included in packaging.  The rpm build
failed because shebang lines of packaged files must not use
`/usr/bin/env` [0] [1] [2].

As it seems that those old files are not of any use anymore, this commit
removes them.

[0] https://docs.fedoraproject.org/en-US/packaging-guidelines/#_shebang_lines
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Multiple_Python_Runtimes
[2] https://pagure.io/packaging-committee/issue/738

closes odoo/odoo#67294

X-original-commit: 54b6a7a97e574249c22f16ea2756989e4bd4fac3
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2021-03-04 18:39:14 +00:00
fja-odoo 9ea276ae2f [IMP] website_form, *: allow form default input value
* = web_editor

Add support for default values on website form

Compatible fields are: textarea, text, number, date, datetime, unique
checkox, multiple checkboxes, radio buttons, select.

Part of: https://github.com/odoo/odoo/pull/47949
task-2011583
2021-02-26 14:40:49 +00:00
Francois (fge) 7b53bb4b6f [FIX] base: @odoo-module alias ending with **/
When you define an @odoo-module with an alias without any space between
the alias and the **/, the alias contains the **/.
The purpose of this commit is to no longer add the **/ to the alias.

Example:
/** @odoo-module alias=web.base**/

Before:
  odoo.define(`web.base**/`, function(require) {
After:
  odoo.define(`web.base`, function(require) {

closes odoo/odoo#66929

Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
2021-02-26 16:06:43 +00:00
Francois (fge) 4627c224ef [IMP] base: add sourcemap support for CSS files.
Improve the development experience in debug=assets mode by reducing the
number of requests to the server. We are adapting the solution used for
the JS files to the CSS files. This solution consists of no longer
sending all the files separately, but sending only the bundles
associated with their sourcemap. This allows us to keep the same
debugging experience while drastically reducing the number of requests
to the server.

Benchmark:
saas 14.2                   917 requests    domcontentloaded after 3.76s
master (bundling du js)     299 requests    domcontentloaded after 2.03s
branch (bundling js+css)    36  requests    domcontentloaded after 1.01s

Task id : 2463840

closes odoo/odoo#66169

Related: odoo/design-themes#453
Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
2021-02-18 08:51:02 +00:00