First step to stream QWeb templates: removing the two post
processing operations applied on rendered templates. This
should slightly speed up the rendering of every page.
1/ Don't remove empty lines after rendering, but fix the root
cause of: view inheritancies and QWeb compilation that don't
add extra empty lines.
2/ handle page break in the two reports that uses it, rather
than processing every view produced.
closesodoo/odoo#82244
Related: odoo/enterprise#23328
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
All textual content generated by opening the tag must be flushed before
inserting the default content
closesodoo/odoo#81700
X-original-commit: effeba10787388ee0a3d153c984fab5731fc9b1a
Signed-off-by: Thibault Francois <tfr@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
For email design in the context of Microsoft Outlook, we want to keep
some magic Microsoft comments (Outlook conditional comment), which -
until this commit - were skipped by QWeb. These allow us to change the
rendering exclusively for Outlook so as to overcome some of its
limitations. This commit introduces a qweb rendering option
(`preserve_comments`) for when - like in mass mailing and digest - we
want to keep comments.
Part-of: odoo/odoo#80621
Bug
===
The arguments of the lambda function might be the same as other
variables in the QWeb. In that case the template rendering will crash.
To avoid this issue, we encapsulate the name of the argument to be sure
it's not the same as other variable in the QWeb engine (so the name of
the argument can no longer make the QWeb rendering crash)
Task-2674716
Closes#78392.
Forward-port of 96b09b127028e5c9a83c89acce86e1b6df904c6f
This commit fixes the case where you have some text after a comment.
Until know, we miss it. Now we render the tail part.
```
<t>
<!-- HIDE Text 1 -->
Text 1
<p>SHOW Text 2</p>
</t>
```
After the fix, Text 1 is correctly rendered
This commit fixes#76628closesodoo/odoo#78782
X-original-commit: 566360b07b4ff6c7290f264d9064400df2b06ca5
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
By activating the profiler debugger, the two qweb options is added. The
qweb templates that need to be rendered are compiled into a new function
to add instructions for saving data.
`Add qweb directive context`
It's a sub-option of "Record sql" or "Record traces", add some context on
thread at current call stack level. This context stored by collector
beside stack and is used by Speedscope to add a level to the stack with
this qweb directive information.
```
directive=t-call='website.layout', xpath=/t/t
t_call_content
directive=t-foreach='5' t-as="'a', xpath=/t/t/div/div
directive=t-esc='website.search([])', xpath=/t/t/div/div/t
execute
```
`Record qweb`
Add profiling data used by ProfilingQwebView widget. In the `ir.profile`
form view, the widget display the duration and number of sql of every qweb
directives with the xml of templates. Every xml is recorded to be
consulted even if the user change the xml templates.
```xml
<t t-call="website.layout">
<div>
<div t-foreach="5" t-as="a">
<t t-esc="website.search([])"/> <!-- will display 5 separate requests -->
</div>
</div>
</t>
```
closesodoo/odoo#74712
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This is the same behavior as javascript. Which had been changed during
the refactoring of qweb. t-esc asked for escaping whatever the content.
However, a lot of templates still use t-esc. It is easier to go back to
the previous functionality.
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes
closesodoo/odoo#68299
Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
* Remove AST in favor of pure Pyhon. This should make it easier for
developers to understand and create new directives because they do not
need to know AST.
* Remove `t-call-options` as it has been merged into `t-options` for more
consistency. Support for t-call-options is retained.
* Use generators for lists. This increases performances as the rendering
can be sent directly without having to wait for the creation of the
entire list.
* Optimize expressions runtime computation by pre-computing the static
parts.
Example:
'<' + 'div' + '>' + '<' + dynamic_value + '>'
Now compiles as:
'<div><' + dynamic_value + '>'
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
The way QWeb handles text nodes and node tails makes it generate
white spaces copied from the XML indentation even before the first
useful character
Before this commit blank lines appeared at the beginning of generated
documents as well as inside the document content
After this commit no blank lines appear at the beginning of generated
documents and blank lines inside document are removed
task-2366756
https://github.com/odoo/odoo/pull/60850closesodoo/odoo#60850
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
This restricts the attributes of a BaseModel instance to `env`, `_ids`
and `_prefetch_ids`. This way, one can only assign fields on a record;
other assignments are programming errors.
This also reduces the memory footprint of records from 168 to 64 bytes
(-62%), and makes their instanciation faster.
closesodoo/odoo#51075
Related: odoo/enterprise#10529
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
render, render_template, load, activity_schedule_with_view,
get_website_pages should all be private:
It should not be possible to render an aribtrary template only with
its name or id
Still need to render some qweb views from js so the method
render_template is kept public.
This explains why the website editor still need read access on
ir.ui.view as we want to allow any snippet to be rendered.
* from collections import <ABC> is deprecated, unclear why the
deprecation warning didn't appear before (possibly only appears in
3.7/3.8?) either way `collections.abc` should be 3.3+ so switch
everything to it.
* add some more ignores on third-party packages deprecation
warnings (meh)
* while at it, mitigate generation of non-breaking space on some
versions of Babel (in the french locale used by our tests anyway)
closesodoo/odoo#47581
Related: odoo/enterprise#9214
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Have a template of that form:
```xml
<t t-if=""/>
<!-- Comment -->
<t t-elif=""/>
```
Before this commit, both python and JS crashed because any type of node
between branching directive was forbidden
After this commit, Comments are just ignored and removed, while actual
template nodes are still forbidden between branching directives
There is no crash anymore
closesodoo/odoo#45899
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
Before this commit, when a t-call was calling an unexisting template, a QWeb
exception would be raised with the callee and not the caller as template.
That would be an issue since there was clean way to retrieve the view that did
the wrong t-call, for instance to be able to repair that view.
Now, in such a case the caller view will be returned.
That will avoid crapy code to retrieve the caller, eg searching on every Qweb
view archs.
Coming from #32009 (task-1943001)
Before this commit, when doing `<t t-raw="0"/>` to print the t-called
content, it was printing "[]" if the content was left empty.
closesodoo/odoo#31668
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Odoo no longer supports python 2, thus some of these helpers can and
have been replaced by python 3 built-ins, therefore there is no need for
them to stay defined.
The removed helpers are:
* izip, imap and ifilter
* unichr, text_type
* implements_to_string, implements_iterator
* string_types, integer_types
* to_native
The python 2 shims have also been removed, and only the python 3 helpers
have been kept, because they can still be usable (i.e. accepting
both bytes and str for functions that can only accept one of the two)
[REM] pyjsparser: remove PY3 shims
They're no longer necessary as Odoo doesn't officially support python 2
anymore.
closesodoo/odoo#28519
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
QWeb's rendering currently *requires* that when loading a document
instead of rendering the document itself it will render a sub-node
with a properly labeled t-name. If a view is invoked by id, ir.qweb
will look for a child with a t-name and fix up that t-name such that
the base qweb object will find and render it.
This feature mostly exists for JS compatibility (because a single
document can contain multiple cross-linkable templates, and it was
useful for cross-implementation tests though I'm not sue they're even
checked anymore), and the <template> tag kinda took care of it,
however with a more proper qweb view it becomes more of a liability /
pain in the ass.
Note: turns out there are bits of javascript which read server
templates (???) and need to be altered to correctly handle the lack of
<templates> wrapping.
When compiling the qweb template, if an error occurs, an error message is
displayed with the corresponding failed node.
This messsage was not always correct because the node is modified in place
during the compilation proccess.
The template is now re-read before displaying the error.
This change greatly facilitates inheritance and xpath and avoids having
to rewrite each time the entire dictionary of t-options.
e.g.
<span t-esc="5" t-options="{'widget': 'char'}" t-options-widget="'float'" t-options-precision="4"/>
the t-options-xxx overwrite the dict. The rendering became:
<span>5.0000</span>
Fabien Pinckaers