Prior to this commit, lazy-loaded bundles weren't pregenerated,
potentially causing non-deterministic test failures when the
generation process took too long. Plus, each Python test loading these
bundles necessitated their regeneration.
With this commit, the lazy-loaded bundles are now included in the
pregenerated bundles.
task-3493014
runbot-24842
closesodoo/odoo#134464
Related: odoo/enterprise#46985
Signed-off-by: David Monjoie (dmo) <dmo@odoo.com>
before this commit, if profiling is enabled in the db,
and on trying to validate a sale order, a traceback is
shown
* enable profiling
* confirm a quotation
traceback:
Failed to render QWeb template : <div style="margin: 0px;
padding: 0px;">
<p style="margin: 0px; padding: 0px; font-size: 13px;">
introduced in: https://github.com/odoo/odoo/commit/016f26a9315c693bdeb894725898c2cf725d8989
here the options['ref'] is coming as the email template
body and it is failing on try to do int of options['ref']
after this commit, no traceback wont be shown on
confirming sale order, when profiling is enabled
closesodoo/odoo#138357
X-original-commit: 01548aff72b031389a7563948bb12bc3abb07bfc
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
The breakpoint() builtin was added in [Python 3.7] ([PEP 553]). It is
conveniently everything-agnostic, can be configured (including
disabled) via an envvar (`PYTHONBREAKPOINT`) and
code (`sys.breakpointhook`), and defaults to pdb (`pdb.set_trace`).
This is more flexible as it allows using arbitrary callables without
special casing, which mostly allows using less common
debuggers (e.g. IDE debug servers / hooks).
As using an empty string for the debugger name was not allowed
previously, co-opt that to invoke `breakpoint`. And deprecate the old
style.
[Python 3.7]: https://docs.python.org/3/whatsnew/3.7.html#pep-553-built-in-breakpoint
[PEP 553]: https://peps.python.org/pep-0553/closesodoo/odoo#134842
Related: odoo/documentation#5800
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
These flags should not be part of Odoo codebase. They have been
implemented internally.
closesodoo/odoo#130947
X-original-commit: 182e4475cef2f6d160381d58726f452169b738e3
Signed-off-by: Vranckx Florian (flvr) <flvr@odoo.com>
Signed-off-by: Adrien Widart (awt) <awt@odoo.com>
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.
Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.
The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.
Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.
The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.
We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.
Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.
Note that this poc is inspired from the long term cache but not all
use case where applie yet.
Part-of: odoo/odoo#119813
`libwebp` cannot be used to perform resize operations on webp images on
the server.
This commit uploads all resizes of webp images whenever a webp image is
uploaded:
- resized to any smaller size in 1024, 512, 256 & 128,
- each size converted to jpg as well to be usable in `wkhtmltopdf`.
When an Image field requires `_imageProcessing` to resize a webp image,
it retrieves the pre-resized image instead of running through the
Pillow toolbox.
Pre-converted images are stored as `ir.attachment`s with:
- `res_model` = ir.attachment
- `res_id` = id of transformed attachment
i.e. id of original size webp for resized images
id of webp for jpeg-converted images
- `description` = "format: image/jpeg" for jpeg conversions,
"resize: [size]" for resized images.
task-2774352
Part-of: odoo/odoo#85494
Wkhtmltopdf is unable to render WebP images.
This commit intercepts the requests for the image data during PDF
reports rendering to replace them by the converted image that was
uploaded with it.
task-2774352
Part-of: odoo/odoo#85494
If a user is not present in the request, he is in no group at all and
can not access any model, including the one available for public
users.
Avoid ambiguity by using sudo or add a user specifically.
Part-of: odoo/odoo#125216
1. move cache to _get_asset_paths
The `_get_asset_content` cache has many cache key that are related to a
posprocessing of the `_get_asset_paths` result, the heavy part of this
method. Moving the cache to _get_asset_content will have the benefit
to create less duplicates entries in the ormcache as well as less cache
miss.
To simplify even further, the css and js parameters are removed since
they only filter the output of get_paths, the heavy part of globing the
file will be done before that. Anyway, they are both true when called
from _get_asset_content, and the only other call, in
`_get_related_bundle` don't really need to filter them since it is not
a critical part regarding performance, and the funtional result will
stay the same.
The initial orm cache key was using `_get_template_cache_keys`, a little
overkill and possibly creating duplicates entries again. The only
context key needed is website_id for `_get_related_assets`.
Note that it is not really enough, the orm cache key should actually
contain `request.session.get('force_website_id')` as well has
`request.httprequest.host`. This will be addressed latter since a nicer
solution would be to have website_id as a unique parameter computed
earlier.
2. better _get_asset_paths cache key
The orm cache key was simplified in previous point but there is still
one concern, the website_id depends on more parameters than that:
- request.session.get('force_website_id')
- request.httprequest.host
- existing websites
The idea here is to call `get_current_website` instead of using all
parameters that could define the webiste.
In the same spirit of `_get_template_cache_keys` `_assets_path_params`
can be overriden to give extra params that are usefull to list assets
path. Those params are computed before entering the method
`_get_asset_paths`. This may latter put at a higher level latter, in
get_asset_node, to simplify the _generate_asset_nodes_cache key.
3. better assets_node caches key
The main purpose of this part is to improve ormcache containing assets
nodes. The ormcache key contains
- to much context key
- missing session/host/env info
- unwanted boolean options.
- keys leading to the same cache value
The main goal being to reduce the size of the cache keys, decrease the
number of cache entries and improve the cache hit.
This will also make the behaviour more coherent and hopefully less bug
prone because of mismatch in parameters.
The main reason of the orm cache is the slowness of the validation of
the assets. This includes:
- listing files (dedicated orm cache)
- computing version
The cache key was depending on
- `debug`
The only relevant value for debug is "contains assets"
We dont need to differ between debug='', debug='1', debug='test',
and 'debug=assets', 'debug=tests,assets', ...
- `defer_load`, `lazy_load`, `media`
Those values are only useful to generate html node, a leightweight
operations that does not really needs to be in cache. `media` was also
used in the generation but it looks useless if we have the media on the
node. THIS NEEDS TO BE VALIDATED but in any case, since media is not
used to generate the url, it doesn't make sence to use it in the
generation.
The main idea to remove them from the ormcache key is simply to generate
the nodes outide the ormcached values.
-`async_load`
This one is similar to `defer_load` and `lazy_load` but it looks like
it wasn't used anymore. This was simply removed
- context.get('lang')
The only information needed is the direction, rtl or ltr. This means
en and fr languages, despite sharing the same css assets, will duplicate
the ormcache entries.
-`_get_template_cache_keys`
Only the lang and webiste where really relevant in this flow. Other
keys are actually useless in this flow.
Some information used in the generation where not in the orm cache key
- `self.env.user.lang` if there is no lang in the context
- `request.session.get('force_website_id')`
- `request.httprequest.host`
- ...
The proposed solution is to:
- extract any informùation needed from thecontext, request, environment
before entering the ormcache, reduce it to the minimal possible set of
values needed
```
rtl = self.env['res.lang']._lang_get_direction(self.env.context.get('lang') or self.env.user.lang) == 'rtl'
assets_params = self.env['ir.asset']._get_assets_params() # website_id
debug_assets = debug and 'assets' in debug
```
and remove a leightweight part of the logic
```
def _get_asset_nodes(self, bundle, css=True, js=True, debug=False, defer_load=False, lazy_load=False, media=None):
links = self._get_asset_links(bundle, css=css, js=js, debug=debug)
return self._links_to_nodes(links, defer_load=defer_load, lazy_load=lazy_load, media=media)
```
Where _get_asset_links is the cached part, and _links_to_nodes is the
lightweight part generating the nodes based on the `defer_load`, ....
Additionnal notes:
- data-asset-version and data-asset-bundle are removed from the node
since they don't seem to be used anymore since 65d70acdbf
- async_load is removed since there is no occurence of this in the code.
- a small hack is still needed to pass javascript content instead of
links, this is only to manage css compile error and will hopefully be
removed in the future.
- a context key is still in use to generate the bundle, the
`commit_assetsbundle` but it has no impact on content and will hopefully
be removed in the future.
4. Add test for ormcache hit/miss
In this context, hit/miss is about having the same cache key for the
same result. This test demonstrates the current state, were entries are
create in the ormcache only if the key is really different and will lead
to a different result.
5. remove cache invalidation
This cache invalidation is quite agressive since everytime an
assetbundle is updated, all workers will clear their cache.
The concerned cache by this clear_cache is `_generate_asset_nodes_cache`
throug `_get_asset_nodes`.
The cache is ignored, both in dev=xml and debug=assets.
This clear cache was made conditionnal in 553ea82f81 but this does
not solve an issue we can have in production.
Lets imagine a clean solution
- all sources are updated
- all workers are restarted.
The orm caches are all empty, but since the sources
changed, all bundles will be recomputed. This means that every bundle
updated in database with save_attachement will invalidate the cache of
all workers. Rendering a pdf report of any kind using a specific bundle
will invalidate all cache. Starting a debug=assets for the first time
will invalidate all cache, even if the cache is not used in this case.
But for a regenerated bundle we would expect the ormcache to be:
- empty (did not generate the same bundle yet)
- have the same value (concurrent generation of the same bundle)
Having a different value would mean that the bundle was generated with
another version of the sources. In this case it is maybe even better not
to invalidate the cache since it could lead to an invalidation war
between two workers.
The only case where invalidating this cache is useful is when a bundle
changes, Usually if an ir_asset is created, modified, ...
There is still another rare but possible possibility to have a 404 if
the transaction is rollbacked after populating the assets node cache.
In this case, we only need to clear the cache locally in case of
rollback.
Part-of: odoo/odoo#121376
Currently when a QWebException raise Exception which will be caught by sentry
and causes unnecessary traffic.
So, we stop catching QWebException which are supposed to be generated by User's
Mistakes.
sentry-4137969310
closesodoo/odoo#121841
X-original-commit: 2859143c9b2950578385b66126178f27db750863
Signed-off-by: Adrien Widart (awt) <awt@odoo.com>
Signed-off-by: Smit Nagar (smna) <smna@odoo.com>
One of the most costly part of a page loading when the ormcache is cold
is computing the assets node, the unique identifier of an attachment to
validate whether the existing attachment is still valid with the current
version of the static files.
This operation needs to glob assets path in the filesystem,
get the modification date, check attachments, ...
Right now this task is not really optimized and can take some time
because of an excessive number of glob on the filesystem, unnecessary
exists to define absolute path, double computation of file list and
modified times when getting js and css bundle separately, ...
A list of modifications mainly discussed in the pr message are made
with this commit to speedup things.
- split css and js unique
- prepare api for an in memory glob
- change api to propagate absolute path and meta information through
`ir.asset._get_paths`-> _get_asset_paths -> `_get_asset_content` ->
`AssetsBundle`
closesodoo/odoo#121159
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The assets bundle system allows to generate `<script>` elements whose
resource is meant to be lazy loaded. In that case, a "data-src" is used
instead of "src" directly as attribute. This is valid HTML code: "src"
is not required. However... this lazy loading system also automatically
added `defer="defer"` which makes the "src" attribute required, thus
failing W3C validation.
Now, we only add "data-src" and the `defer="defer"` part is added on the
client-side, once the "data-src" is switched to "src". Note that this
"defer" attribute may not be needed in this case at all, but it cannot
hurt.
Part-of: odoo/odoo#120311
Before this commit an ir_assets generated automaticaly by the web editor
will generate an url ending with ...custom.addon.bundle_name.ext
After this commit the url will start with /_custom/addon.bundle_name/...
This will make it easier to spot at immediately if it is a custom asset
and thus it is useless to apply the glob. Actually, it will fail on the
/_custom when trying to glob, making it faster.
This is mainly useful to clarify and debug but in a case with mainly
customised ir_asset for one bundle, it may have an impact on speed.
An upgrade script was created for this change.
closesodoo/odoo#120699
Related: odoo/upgrade#4636
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Issue: when use t-if to display text and use mutli line, the white-space
are removed.
```
<div>
<t t-out="nb"/>
<t t-if="nb == 0 or nb == 1">product</t>
<t t-else="">products</t>
</div>
```
is rendered:
```
<div>
3products
</div>
```
closesodoo/odoo#114986
X-original-commit: 3150a3769ff4d762c0e52dab31a70b59292c1bcc
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
When python expression is evaluated in odoo form an action or qweb, we
are checking the opcodes generated by the evaluation of this code. We do
such a verification, because the code from actions and templates can be
written by someone having not access to the server and we don't want to
let them perform actions out of the scope of their database.
In python 3.11, some opcodes from previous versions of Python have been
renamed, grouped or sepcified. There are also new ones that have been
introduce.
In this PR, we are whitelisting the new ones that are needed by odoo to
properly work in this version of Python.
Part-of: odoo/odoo#112450
XML files are now declared in python module manifests. During the qweb
't-call-asset' directive, assetbundle will fetch the declared xml files,
apply the inheritance (t-inherit) and create a javascript service (for
eg: 'web.assets_backend.bundle.xml') which is added at the end of the
*.js mimifier file.
When the debug mode is activated, comments are added in the template
indicating which file the template comes from as well as the
inheritances applied to it.
****
JavaScript:
assets.js (module @web/core/assets) takes care of loading libraries,
javascripts and styles.
`loadJS(url)` (loads the javascript and returns a resolved promise when
the templates are also loaded via the '*.bundle.xml' service)
`loadCSS(url)` (loads the style a resolved promise when the file is
loaded)
`loadXML(xml, app=assets.defaultApp)` (load template into
application/owl, used by the `*.bundle.xml` services)
`getBundle(bundleName)` (get the bundle descriptor)
`loadBundle(desc)` (load the files and bundle from a descriptor)
templates (XML element content all owl templates)
A new `ready(serviceName)` method on boot.js lets you know when a
service is loaded are the require.
The xmlDependencies attribute no longer exists.
Python:
The xmls taken into account by assetbundle.py, applying `t-inherit`
inheritances and adding an `name_of_the_bundle.bundle.xml` service in
the generated JavaScript file.
****
Every manifest changes is into the next commit, except 'web_tour' in
this current commit as example.
Part-of: odoo/odoo#95500
Visible_menu check per_read model per model leading to a lot of queries
We can get all readable fields with one query
Computation time goes from ~400 total to less than ~50 total
The different would be even bigger on a server with a distant database.
(higher ping)
Part-of: odoo/odoo#99176
During tests on runbot, the main reason tours are slow to start is
because the first loading of "/web" need to generate assets bundles.
Generation can take up to 5 seconds time the number of tour.
Before this, the first requests to /web takes around 7 seconds on
runbot and the next ones less than 1 second.
After this commit, the first request to /web takes around 1.5 seconds
Note that the main difficulty is to choose when to generate the assets.
(With optimisations from following commits) the generation time is
arround 30 seconds (21 css + 12 js) the first time, for all modules.
If the attachements exists the generation time is arround 3 seconds
(2.5 js + 0.5 css) mainly because of globs to find usefull files.
In practice for runbot the ideal would be to generate them at
the end of the install so that it is shared for all post_install builds.
Doint it at the end of an install is not wanted in all cases, saas
pregenerated templates and upgrade may avoid doing that.
Doing it in a special runbot step (subcommand) is not possible for niglty
execution (no split, in one go). This needs to be in the code.
It is also not practical for devs wanting to have the same behaviour on
a local machine.
A solution to make the test conditionnal at install is to base the
condion on an existing test_module. test_assetsbundle is a good
candidate here. On runbot, the at_install step (befor split) will
automatically generate assets with this solution.
Assets are also generated before post_install tests. This should be fast
if they already exists and ensure that assets are up to date after
modifying sources locally or after downloading a database on runbot.
It should be fast enough for small database and may be even faster
in the future for small diffs.
Part-of: odoo/odoo#99176
Since 5913b7265e it was not possible
anymore to use default value with `t-field`, this commit restore the
previous behavior.
closesodoo/odoo#98453
X-original-commit: 12dc2353f572d097fc57ec723a11651a8471fdf5
Signed-off-by: Rémy Voet <ryv@odoo.com>
`content = self._compile_to_str(content)` is done twice in the compiled
method of qweb. Remove it once.
closesodoo/odoo#95755
Signed-off-by: Julien Castiaux <juc@odoo.com>
*: mass_mailing, web, web_editor, website_blog, website_event,
website_event_meet, website_livechat, website_sale, website_twitter,
base
This commit does 3 things:
- Adapt existing code so that the editor can be run outside of its
editing element. Prior to this commit, it was expected that the
editor would be attached to the element it was currently editing.
This needs to change however as the editable is now inside an
iframe for the website edition. mass_mailing already had a similar
approach but the editor was started within the iframe.
With these changes, the editor is alongside the iframe, editing
the content that's inside it. This allows multiple features
such as reloading the iframe while keeping the editor visible and
resizing the iframe for a mobile preview.
- Adds logic to the wysiwyg_adapter so that it can undertake
the duty of the previous widget sytem. The wysiwyg existed not only
in the legacy widget system, but most importantly in the frontend.
One of the duties of this adapter is to send events inside the
iframe when it is necessary to reach the frontend public widget
(i.e. widgets_start_request)
- Removes existing SCSS that is no longer used.
See merge commit for more information.
task-2687506
The methods to generate the attributes are called all the time in order
to reset the attribute dictionary. The profiler is modified so as not to
display directives in the log that do not exist on the tag.
Issue: attributes could be generated by directives and not be used (for
example on <t>). These attributes could end up unwittingly on the next
node.
closesodoo/odoo#93216
Issue: opw-2859447
X-original-commit: 38724f106c0747aef475991c894e9ee93c1db967
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Christophe Matthieu (chm) <chm@odoo.com>
The `t-cache` directive allows you to keep the rendered result
of a template part. The supplied key must be a tuple. This tuple
can contain recordset in this case the zone will be invalidated
each time the write_date of these records changes.
The `t-nocache` directive makes it possible to force rendering
of a part even if it is in a `t-cache`. The values available in
the `t-nocache` are the one provided when calling the template
(and therefore ignores any t-set that could have been done).
Part-of: odoo/odoo#88276
Before this commit, when a view contained a ProcessingInstruction node
`<?XXX YYY?>`, if this was rendered via qweb, it was leading to very
strange results in the form of
```
<<cyfunction ProcessingInstruction at 0x7f538c4902b0>>YYY</<cyfunction ProcessingInstruction at 0x7f538c4902b0>>
```
After this commit, it will lead to "<?XXX YYY?>" being rendered which
will automatically be parsed into an HTML comment if reaching a browser:
```
<!--?XXX YYY?-->
```
Note: this will be rendered only if preserve_comments is set to true.
This was added following [1] which actually led to an issue where some
ProcessingInstruction nodes were rendered by mistake. That was fixed
with [2]. With this commit, that kind of mistake will silently fail
(which may not be good) but will lead to no harm and allows to better
debug such cases (by actually allowing rendering what was in the view).
[1]: https://github.com/odoo/odoo/commit/f67832a3ae0d9a3b5b53129132762e6bc1aed874
[2]: https://github.com/odoo/odoo/commit/de348b3f818adcf2068deb629e612c246a6a4374closesodoo/odoo#92213
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.
e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`
The goal of this revision is to change that so it sends the list of all fields
only once.
In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
- `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
- `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`
The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
As it no longer contains the fields,
the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
which is a dict with as key the model name and as values
the model fields description. It contains the fields description
for all models implied in the view:
the model of the main view and the model of all one2many and many2many fields.
With this change, the fields description will only be sent once by model
implied in the view.
In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.
- one2many and many2many fields views are passed directly in the main view
architecture rather than being put in the `views` key
of the field description.
This is actually easier to treat by the web client,
and this will allow in a future work to cache an entire view in one block
of text rather than having to combine multiple cached blocks of text
to return one view.
- one2many and many2many fields which do not have directly embedded views
have their views directly injected in the architecture,
so the web client doesn't have to do RPC calls to `load_views`
for each one2many and many2many fields not having embedded views.
For instance, this allow to reduce the number of RPC calls to `load_views`
from 8 to 1 when loading the form of `product.product`.
Currently, this behavior is limited to 1 level deep but we consider making it
go all the way down in future works. We did not do it for the moment because
in certain cases it rises the processing time and the size (bytes) too much.
e.g. the sale.order view can be 5 levels deep,
meaning you can reach 4 dialogs on top the main view.
```
sale.order form > order_line > sale.order.line form > invoice_lines >
account.move.line form > asset_ids > account.asset form >
depreciation_move_ids > account.move form.
```
This will also benefit in future works to cache an entire view in one block
of text rather to having to combine multiple cached block of text
to get one view.
- `fields_view_get` becomes `get_view`.
As it no longer returns the fields description,
keeping the `fields` in the name `fields_view_get` no longer makes sense.
Hence removing `fields` from the method name, it becomes `view_get`.
As it gets renamed anyway, we take the opportunity to rename it `get_view`,
which is more in line with the general getter/setter guidelines
in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
This is not mandatory, there is no technical reason to rename `load_views` as
it practically sends the same info as before,
the view architectures and their fields description. Just in another way.
We just take the opportunity of this pull request to suggest a cleaner API:
`_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
`_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
in `_get_view` and `get_view`.
The rationale is that submenu was already no longer used (deprecated)
and the mobile options is introduced.
The mobile options is necessary to tell the server to send the mobile views
for x2many fields (kanban instead of tree).
Instead of adding a new argument each time we add a new option to
`fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
view information. Now, `get_view` returns a tuple with the view architecture
as an `etree` node, and the view as a browse record. The rationale is that all
overrides of `_fields_view_get` were about modifying the arch only
(e.g. changing the address format/re-organizing the address related field
nodes of the partner according to the company country).
To do so, all these overrides were doing `etree.fromstring` to parse the arch
which was sent in text to convert it to an `etree`,
then operations were done on the `etree`,
and then `etree.tostring` was called to convert back the arch to string.
With this change of signature to send the arch as an `etree`,
all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
has been performed in `get_view`:
- `fields` is removed, as explained above,
- `view_id` is renamed `id`,
- `name` is removed, it was unused by the web client,
- `type` is removed, it was unused by the web client,
- `field_parent` is removed, it was unused by the web client,
- `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
(now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
`fields_view_get`, `_fields_view_get` and `load_views` are provided,
with deprecation warnings in them.
- The web client could cache the model fields description
(as it already caches the views),
so it doesn't need to fetch them again if it asks for another view of a model
for which he already has the fields description.
If we do so, `get_views` could return only the list of models used by
the views, without the fields description as of now,
and the web client would then call `fields_get` independently only for
the models for which it doesn't have yet the fields description.
This would avoid the server to return the fields description
and to call `fields_get`, which is costly, for each `get_views`,
therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
This is already done for qweb views, it's not done for back-end views.
Therefore the postprocessing of the views is performed for each `get_views`,
which is costly, while the view architecture doesn't change for users
belonging to the same groups, according to the groups implied by the view.
This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.
Part-of: odoo/odoo#87522
Install auth_signup, go to /web/login, 500 Internal Server Error.
auth_signup extends the /web/login template and in this extension calls
`keep_query()` which has been wrongly moved from base to http_routing in
commit 880954ebfc. Here, we restored `keep_query()` in the base module
but moved in ir_qweb.
closesodoo/odoo#87491
Related: odoo/enterprise#25754
Signed-off-by: Julien Castiaux <juc@odoo.com>
This flat structure makes caching easier if odoo adds shared caches for
example. In addition, the use of idempotent names makes it possible to
compare methods during development.
closesodoo/odoo#85110
Related: odoo/design-themes#554
Related: odoo/enterprise#24622
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.
The report rendering and call `ir.qweb` instead of `ir.ui.view`.
Part-of: odoo/odoo#85110
The duplication was used to have a cleaner html page source code by
aligning the <script t-foreach/> for example. However, this behavior
seems difficult to understand and use. Additionally the behavior can be
done with a <t> tag and placing the script inside.
From now on, the space "trick" is only done for <t> tags, so the
template fragment
<article t-foreach="[0, 1, 2]" t-as="value" t-esc="value"/>
<t t-foreach="[0, 1, 2]" t-as="value">
<article t-esc="value"/>
</t>
renders as
<article>0</article><article>1</article><article>2</article>
<article>0</article>
<article>1</article>
<article>2</article>
closesodoo/odoo#86695
X-original-commit: 8f817790d3f36455c66647e79df40b7173ac498a
Signed-off-by: Raphael Collet <rco@odoo.com>
During the merge of qweb/ir.qweb at commit 9ce5bc8881,
the behavior of qweb was preserved instead of the one from ir.qweb.
Rendering inline templates should not be allowed as it is not needed and
include security risks.
Also, minimizing the number of possible types for template arguments
simplifies the API. An error message allows the developer to know how to
adapt his code.
closesodoo/odoo#86404
X-original-commit: 31f3152a1328ec8e2dd28f15433a2cc3ccc80f81
Signed-off-by: Olivier Dony <odo@odoo.com>
QWeb is the primary templating engine used by Odoo. It is an XML
templating engine and used mostly to generate XML, HTML fragments and
pages.
To create new XML template, please see :doc:`QWeb Templates documentation
<https://www.odoo.com/documentation/15.0/developer/reference/frontend/qweb.html>`
In **input** you have an XML template giving the corresponding input
etree. Each etree input nodes are used to generate a python function.
This fonction is called and will give the XML **output**.
The ``_compile`` method is responsible to generate the function from the
etree, that function is a python generator that yield one output line at a
time. This generator is consumed by ``_render``. The generated function is
orm cached.
In the graphic below you can see theresume of the call of the methods
performed in the IrQweb class.
Odoo
┗━► _render (returns MarkupSafe)
┗━► _compile (returns function) ◄━━━━━━━━━┓
┗━► _compile_node (returns code string array) ◄━━━━━━━┓ ┃
┃ (add technical directives: t-inner-content, t-tag) ┃ ┃
┣━► _directives_eval_order (defined directive order) ┃ ┃
┃ ┃ ┃
┣━► _compile_directives (recursive) ◄━━━━┓ ┃ ┃
┃ ┣━► _compile_directive ┃ ┃ ┃
┃ ┃ ┗━► t-if ━━► _compile_directive_if ━┫ ┃ ┃
┃ ┃ ┗━► t-foreach ━━► _compile_directive_foreach ━┫ ┃ ┃
┃ ┃ ┗━► t-* ━━► ... ━┛ ┃ ┃
┃ ┃ ┗━► t-inner-content ━━► _compile_directive_inner_content ◄━━━━┓ ━┛ ┃
┃ ┃ ┗━► t-tag ━━► _compile_directive_tag ━┫ ┃
┃ ┃ ┗━► t-call ━━► _compile_directive_call ━┫ ━━━┛
┃ ┃ ┗━► t-out ━━► _compile_directive_out ◄━┓ ━┫
┃ ┃ ┗━► t-field ━━► _compile_directive_field ━┛ ┃
┃ ┃ ┃
┗━━┻━► _compile_static_node ━┛
Part-of: odoo/odoo#81024
Followup of #82838 and #82844
By default, every subclass of BaseModel is registered in a mapping for
the automatic discovery of model classes by registries. Prevent
registration of the mockup class used for rendering the database manager
templates.
closesodoo/odoo#83626
X-original-commit: 3bce74b20f8c15befee453382e3ab4697bee6960
Signed-off-by: Raphael Collet <rco@odoo.com>
Fixup for #82838, which could cause the following crash when creating a
new database from the DB manager UI, but not from the command-line:
```
Traceback (most recent call last):
File "/home/odoo/15.0/odoo/service/db.py", line 63, in _initialize_db
registry = odoo.modules.registry.Registry.new(db_name, demo, None, update_module=True)
File "/home/odoo/15.0/odoo/modules/registry.py", line 87, in new
odoo.modules.load_modules(registry, force_demo, status, update_module)
File "/home/odoo/15.0/odoo/modules/loading.py", line 415, in load_modules
loaded_modules, processed_modules = load_module_graph(
File "/home/odoo/15.0/odoo/modules/loading.py", line 199, in load_module_graph
registry.init_models(cr, model_names, {'module': package.name}, new_install)
File "/home/odoo/15.0/odoo/modules/registry.py", line 427, in init_models
model._auto_init()
File "/home/odoo/15.0/odoo/models.py", line 2795, in _auto_init
raise_on_invalid_object_name(self._name)
File "/home/odoo/15.0/odoo/models.py", line 104, in raise_on_invalid_object_name
raise ValueError(msg)
ValueError: The _name attribute MockIrQWeb is not valid.
```
Not visible on runbot as it does not use the DB manager for creating
databases.
Fixes#82843closesodoo/odoo#82853
X-original-commit: c0da3aa4812dc7be4f7ce94d771e44d093d4f612
Signed-off-by: Olivier Dony <odo@odoo.com>
Due to #82724 and 8ec7739dd72fa0976697f7738b225dbb9efa7c97,
the qweb renderer can't be used directly, however there is no way to use
the proper ir.qweb one without a database.
This is necessary for the /web/database/ routes (db manager) which work
without a database, and require qweb rendering since the abandon of
jinja rendering in v15.
This patch builds on the preparation work of the qweb/ir_qweb merge in #81024
which introduces a limited helper `render()` method in ir_qweb so that
it can be used statically without a database.
Fixes#82835closesodoo/odoo#82841
X-original-commit: 3014e922b48696fb9aba4b477ebe1878dd01bb69
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
Purpose
=======
Clarify that qweb should be used as <ir.qweb> when the latter is
available. <ir.qweb> version is a an optimized version of QWeb rendering
and using it is better for performance and daily usage. Moreover it used
Odoo model instead of standard python class, that's why we do not want
people to use QWeb instead of <ir.qweb>.
Task-2709589
closesodoo/odoo#82781
X-original-commit: b08f475e34ccd6e0417374953c11728efad07402
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
First step to stream QWeb templates: removing the two post
processing operations applied on rendered templates. This
should slightly speed up the rendering of every page.
1/ Don't remove empty lines after rendering, but fix the root
cause of: view inheritancies and QWeb compilation that don't
add extra empty lines.
2/ handle page break in the two reports that uses it, rather
than processing every view produced.
closesodoo/odoo#82244
Related: odoo/enterprise#23328
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
In the payment_adyen module, files of adyen are added as part of the
assets_frontend bundle. That bundle is lazy loaded on the website... but
those external files were not, as not added within the bundle itself.
This commit fixes that, properly lazy loading "remaining" files
out of a lazy loaded assets bundle.
closesodoo/odoo#77877
X-original-commit: 8dd71bdc42c8d4ab3373a2a0601c93d1687aabca
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
Nicolas Bayet