Replace all the calls to get_resource_path to the better file_path or
directly use file_open when not needed
Doing both a get_resource_path and file_open means checking twice that
the file exists.
Doing a simple path concatenation before a file_open is safe.
If given to another method (e.g. etree.parse), calling file_path is
the prefered method.
Note that get_resource_path used to return False when the file does
not exists while file_path/file_open raises a FileNotFoundException
closesodoo/odoo#135607
Related: odoo/upgrade#5187
Related: odoo/enterprise#47475
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Since [1] qweb library is not used anymore, rendering this test useless.
1 : 4b0a951af6closesodoo/odoo#136610
Signed-off-by: Raphael Collet <rco@odoo.com>
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.
Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.
The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.
Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.
The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.
We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.
Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.
Note that this poc is inspired from the long term cache but not all
use case where applie yet.
Part-of: odoo/odoo#119813
Issue: when use t-if to display text and use mutli line, the white-space
are removed.
```
<div>
<t t-out="nb"/>
<t t-if="nb == 0 or nb == 1">product</t>
<t t-else="">products</t>
</div>
```
is rendered:
```
<div>
3products
</div>
```
closesodoo/odoo#114986
X-original-commit: 3150a3769ff4d762c0e52dab31a70b59292c1bcc
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Warnings show up when using a recent pylint. It's only a fraction of
what e.g. pycharm flags as "local variable might be referenced before
assignment" but seems a good idea to fix anyway in prevision of
possibly eventually updating the reference pylint.
closesodoo/odoo#107968
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Since odoo/odoo#97347 we support default values for `t-field`
directives.
However this causes some issues in the case where we have: no value, no
default value AND force_display is true.
This is because the indentation for one of the lines in the compiled
code was not correct. (full explanation here: https://github.com/odoo/odoo/pull/97347#issuecomment-1222430659)
closesodoo/odoo#98714
X-original-commit: 2b605d3ccd55dc4f5b1ad44439411542e5b04b31
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: William Braeckman (wbr) <wbr@odoo.com>
Since 5913b7265e it was not possible
anymore to use default value with `t-field`, this commit restore the
previous behavior.
closesodoo/odoo#98453
X-original-commit: 12dc2353f572d097fc57ec723a11651a8471fdf5
Signed-off-by: Rémy Voet <ryv@odoo.com>
The methods to generate the attributes are called all the time in order
to reset the attribute dictionary. The profiler is modified so as not to
display directives in the log that do not exist on the tag.
Issue: attributes could be generated by directives and not be used (for
example on <t>). These attributes could end up unwittingly on the next
node.
closesodoo/odoo#93216
Issue: opw-2859447
X-original-commit: 38724f106c0747aef475991c894e9ee93c1db967
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Christophe Matthieu (chm) <chm@odoo.com>
The `t-cache` directive allows you to keep the rendered result
of a template part. The supplied key must be a tuple. This tuple
can contain recordset in this case the zone will be invalidated
each time the write_date of these records changes.
The `t-nocache` directive makes it possible to force rendering
of a part even if it is in a `t-cache`. The values available in
the `t-nocache` are the one provided when calling the template
(and therefore ignores any t-set that could have been done).
Part-of: odoo/odoo#88276
Before this commit, when a view contained a ProcessingInstruction node
`<?XXX YYY?>`, if this was rendered via qweb, it was leading to very
strange results in the form of
```
<<cyfunction ProcessingInstruction at 0x7f538c4902b0>>YYY</<cyfunction ProcessingInstruction at 0x7f538c4902b0>>
```
After this commit, it will lead to "<?XXX YYY?>" being rendered which
will automatically be parsed into an HTML comment if reaching a browser:
```
<!--?XXX YYY?-->
```
Note: this will be rendered only if preserve_comments is set to true.
This was added following [1] which actually led to an issue where some
ProcessingInstruction nodes were rendered by mistake. That was fixed
with [2]. With this commit, that kind of mistake will silently fail
(which may not be good) but will lead to no harm and allows to better
debug such cases (by actually allowing rendering what was in the view).
[1]: https://github.com/odoo/odoo/commit/f67832a3ae0d9a3b5b53129132762e6bc1aed874
[2]: https://github.com/odoo/odoo/commit/de348b3f818adcf2068deb629e612c246a6a4374closesodoo/odoo#92213
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.
The report rendering and call `ir.qweb` instead of `ir.ui.view`.
Part-of: odoo/odoo#85110
The duplication was used to have a cleaner html page source code by
aligning the <script t-foreach/> for example. However, this behavior
seems difficult to understand and use. Additionally the behavior can be
done with a <t> tag and placing the script inside.
From now on, the space "trick" is only done for <t> tags, so the
template fragment
<article t-foreach="[0, 1, 2]" t-as="value" t-esc="value"/>
<t t-foreach="[0, 1, 2]" t-as="value">
<article t-esc="value"/>
</t>
renders as
<article>0</article><article>1</article><article>2</article>
<article>0</article>
<article>1</article>
<article>2</article>
closesodoo/odoo#86695
X-original-commit: 8f817790d3f36455c66647e79df40b7173ac498a
Signed-off-by: Raphael Collet <rco@odoo.com>
During the merge of qweb/ir.qweb at commit 9ce5bc8881,
the behavior of qweb was preserved instead of the one from ir.qweb.
Rendering inline templates should not be allowed as it is not needed and
include security risks.
Also, minimizing the number of possible types for template arguments
simplifies the API. An error message allows the developer to know how to
adapt his code.
closesodoo/odoo#86404
X-original-commit: 31f3152a1328ec8e2dd28f15433a2cc3ccc80f81
Signed-off-by: Olivier Dony <odo@odoo.com>
QWeb is the primary templating engine used by Odoo. It is an XML
templating engine and used mostly to generate XML, HTML fragments and
pages.
To create new XML template, please see :doc:`QWeb Templates documentation
<https://www.odoo.com/documentation/15.0/developer/reference/frontend/qweb.html>`
In **input** you have an XML template giving the corresponding input
etree. Each etree input nodes are used to generate a python function.
This fonction is called and will give the XML **output**.
The ``_compile`` method is responsible to generate the function from the
etree, that function is a python generator that yield one output line at a
time. This generator is consumed by ``_render``. The generated function is
orm cached.
In the graphic below you can see theresume of the call of the methods
performed in the IrQweb class.
Odoo
┗━► _render (returns MarkupSafe)
┗━► _compile (returns function) ◄━━━━━━━━━┓
┗━► _compile_node (returns code string array) ◄━━━━━━━┓ ┃
┃ (add technical directives: t-inner-content, t-tag) ┃ ┃
┣━► _directives_eval_order (defined directive order) ┃ ┃
┃ ┃ ┃
┣━► _compile_directives (recursive) ◄━━━━┓ ┃ ┃
┃ ┣━► _compile_directive ┃ ┃ ┃
┃ ┃ ┗━► t-if ━━► _compile_directive_if ━┫ ┃ ┃
┃ ┃ ┗━► t-foreach ━━► _compile_directive_foreach ━┫ ┃ ┃
┃ ┃ ┗━► t-* ━━► ... ━┛ ┃ ┃
┃ ┃ ┗━► t-inner-content ━━► _compile_directive_inner_content ◄━━━━┓ ━┛ ┃
┃ ┃ ┗━► t-tag ━━► _compile_directive_tag ━┫ ┃
┃ ┃ ┗━► t-call ━━► _compile_directive_call ━┫ ━━━┛
┃ ┃ ┗━► t-out ━━► _compile_directive_out ◄━┓ ━┫
┃ ┃ ┗━► t-field ━━► _compile_directive_field ━┛ ┃
┃ ┃ ┃
┗━━┻━► _compile_static_node ━┛
Part-of: odoo/odoo#81024
Purpose
=======
Clarify that qweb should be used as <ir.qweb> when the latter is
available. <ir.qweb> version is a an optimized version of QWeb rendering
and using it is better for performance and daily usage. Moreover it used
Odoo model instead of standard python class, that's why we do not want
people to use QWeb instead of <ir.qweb>.
Task-2709589
closesodoo/odoo#82781
X-original-commit: b08f475e34ccd6e0417374953c11728efad07402
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
First step to stream QWeb templates: removing the two post
processing operations applied on rendered templates. This
should slightly speed up the rendering of every page.
1/ Don't remove empty lines after rendering, but fix the root
cause of: view inheritancies and QWeb compilation that don't
add extra empty lines.
2/ handle page break in the two reports that uses it, rather
than processing every view produced.
closesodoo/odoo#82244
Related: odoo/enterprise#23328
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
All textual content generated by opening the tag must be flushed before
inserting the default content
closesodoo/odoo#81700
X-original-commit: effeba10787388ee0a3d153c984fab5731fc9b1a
Signed-off-by: Thibault Francois <tfr@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
Bug
===
The arguments of the lambda function might be the same as other
variables in the QWeb. In that case the template rendering will crash.
To avoid this issue, we encapsulate the name of the argument to be sure
it's not the same as other variable in the QWeb engine (so the name of
the argument can no longer make the QWeb rendering crash)
Task-2674716
Closes#78392.
Forward-port of 96b09b127028e5c9a83c89acce86e1b6df904c6f
This commit fixes the case where you have some text after a comment.
Until know, we miss it. Now we render the tail part.
```
<t>
<!-- HIDE Text 1 -->
Text 1
<p>SHOW Text 2</p>
</t>
```
After the fix, Text 1 is correctly rendered
This commit fixes#76628closesodoo/odoo#78782
X-original-commit: 566360b07b4ff6c7290f264d9064400df2b06ca5
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
This is the same behavior as javascript. Which had been changed during
the refactoring of qweb. t-esc asked for escaping whatever the content.
However, a lot of templates still use t-esc. It is easier to go back to
the previous functionality.
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes
closesodoo/odoo#68299
Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
* Remove AST in favor of pure Pyhon. This should make it easier for
developers to understand and create new directives because they do not
need to know AST.
* Remove `t-call-options` as it has been merged into `t-options` for more
consistency. Support for t-call-options is retained.
* Use generators for lists. This increases performances as the rendering
can be sent directly without having to wait for the creation of the
entire list.
* Optimize expressions runtime computation by pre-computing the static
parts.
Example:
'<' + 'div' + '>' + '<' + dynamic_value + '>'
Now compiles as:
'<div><' + dynamic_value + '>'
* QWeb bodies should be markup-safe so `0` should always be
markup-safe.
* `head` is qweb-rendered so the same.
* The `json` pseudo-module in qweb templates is `json.scriptsafe`,
which should be markup-safe.
`markupsafe.escape` always escapes single and double quotes, and
escapes them to their numeric values rather than symbolic
According to pallets/jinja@f35e28154f,
this is for compatibility with HTML 3.2: the only named entities in
the HTML 3.2 DTD are `amp`, `gt`, and `lt`.
Update tests to match.
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
As this backport already exists in saas-14.2 and master, only the
`lxml.html.Element` part is kept in this commit which is a safer way to
build the widget.
closesodoo/odoo#68483
X-original-commit: c891d61802018a14944c22d01316872427a40284
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When printing barcodes for a large amount of records, wkhtmltopdf will
make the same amount of http requests to the server in order to retrieve
barcodes. This can lead to performance issues or wkhtml to crash.
Fortunately, an already existsing qweb widget exists that include
barcodes images as inline base64.
With this commit, the web widget is a bit improved to accept attributes
for the generated image tag. Each option dictionary key starting with
`img_` will be converted into a tag attribute with the corresponding
value.
e.g.: `'img_alt': 'Barcode'` will result in `<img alt="Barcode"...`
The `alt` img attribute now defaults to `'Barcode %s' % value` with the
barcode value.
Also, the `quiet` reportlab option is also avalaible in the widget
options, as well as the `mask` option.
Finally, if the `symbology` option is not given, the widget defaults to
`Code128`. If `symbology` is set to `auto`, the action report will try
to guess the barcode type, based on the len of the value.
The way QWeb handles text nodes and node tails makes it generate
white spaces copied from the XML indentation even before the first
useful character
Before this commit blank lines appeared at the beginning of generated
documents as well as inside the document content
After this commit no blank lines appear at the beginning of generated
documents and blank lines inside document are removed
task-2366756
https://github.com/odoo/odoo/pull/60850closesodoo/odoo#60850
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
render, render_template, load, activity_schedule_with_view,
get_website_pages should all be private:
It should not be possible to render an aribtrary template only with
its name or id
Still need to render some qweb views from js so the method
render_template is kept public.
This explains why the website editor still need read access on
ir.ui.view as we want to allow any snippet to be rendered.
load_lang was a kind of hybrid method trying to active or creating a
language if not found. This was error prone.
Instead rely on two methods with clear purpose:
ResLang._create_lang(lang, lang_name=None)
- create a new res.lang entry using the locale of the server
return the res.lang record to match the API of _activate_lang
ResLang._active_lang(code)
- activate the given code lang
Most of the time, _active_lang is what is expected
tools.trans_load_data and IrTranslation._load_module_terms no longer
activate the language if not active.
Loading the translations should be explicit on an activated language,
it is too error prone to silently activate/create a language if not
found.
Remove lang_name from trans_load_data as no longer needed.
get_installed and _lang_get_id are both ormcached and correctly check
the context
Retrieving a res.lang from a code is a frequent action that can be
achieved with _lang_get (cf previous commit).
Using _lang_get ensure the active_test in the context is correct and
is not poluted with another context propagation issue.
odoo/odoo#35490 discussion is an example of bad context propagation
closesodoo/odoo#35504
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
* add a json_scriptsafe module, which re-exports the stdlib's json but
provides a script-safe `dumps` function, because of the way it works
the script-safe version is fine / safe to use outside script
contexts, so there's no need to provide different functions to qweb
contexts
* make the json.dumps in qweb contexts <script>-safe (should be
rendered using t-raw)
* update callsites to properly dump json:
- dump in template, not in the python code
- dump the entire structure at once, not bits and bobs inside a JSON
literal
Task 1999158
closesodoo/odoo#33682
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this commit, the t-lang on a t-call was not
propagated into the context's of the callee template
It resulted into mismatch translation and formatting in the final
rendered template
i.e. having a t-esc with a qweb widget float would take the wrong float format
After this commit, the t-lang parameter alters the context of the template
and t-esc with widgets are translated
OPW 1939989
closesodoo/odoo#31291
This change greatly facilitates inheritance and xpath and avoids having
to rewrite each time the entire dictionary of t-options.
e.g.
<span t-esc="5" t-options="{'widget': 'char'}" t-options-widget="'float'" t-options-precision="4"/>
the t-options-xxx overwrite the dict. The rendering became:
<span>5.0000</span>
Before this commit, if a SyntaxError or any other exception occurs in a
qweb attribute, and the path to the node contains at least one
namespaced tag, e.g.:
<mynode:node xmlns:mynode="http://example.com/mynode">
<field t-att-value="'This is a TypeError' + []"/>
</mynode:node>
the following exception is raised, instead of the actual one:
lxml.etree.XPathEvalError: Undefined namespace prefix
After this commit, if the path to the failing node contains a namespaced
tag, the node is not retrieved, to avoid the above exception, So this
one will be raised:
Error to render compiling AST
TypeError: Can't convert 'list' object to str implicitly
Template: module.template_name
Path: /templates/mynode:node/field
In addition, a test case was included, to ensure this works as expected
Closes#24824
opw-1853232
Martin Trigaux