pengyuthon/odoo_source
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[IMP] base: better ormcache management

Browse Source 
1. move cache to _get_asset_paths

The `_get_asset_content` cache has many cache key that are related to a
posprocessing of the `_get_asset_paths` result, the heavy part of this
method. Moving the cache to _get_asset_content will have the benefit
to create less duplicates entries in the ormcache as well as less cache
miss.

To simplify even further, the css and js parameters are removed since
they only filter the output of get_paths, the heavy part of globing the
file will be done before that. Anyway, they are both true when called
from _get_asset_content, and the only other call, in
`_get_related_bundle` don't really need to filter them since it is not
a critical part regarding performance, and the funtional result will
stay the same.

The initial orm cache key was using `_get_template_cache_keys`, a little
overkill and possibly creating duplicates entries again. The only
context key needed is website_id for `_get_related_assets`.

Note that it is not really enough, the orm cache key should actually
contain `request.session.get('force_website_id')` as well has
`request.httprequest.host`. This will be addressed latter since a nicer
solution would be to have website_id as a unique parameter computed
earlier.

2. better _get_asset_paths cache key

The orm cache key was simplified in previous point but there is still
one concern, the website_id depends on more parameters than that:
- request.session.get('force_website_id')
- request.httprequest.host
- existing websites

The idea here is to call `get_current_website` instead of using all
parameters that could define the webiste.

In the same spirit of `_get_template_cache_keys` `_assets_path_params`
can be overriden to give extra params that are usefull to list assets
path. Those params are computed before entering the method
`_get_asset_paths`. This may latter put at a higher level latter, in
get_asset_node, to simplify the _generate_asset_nodes_cache key.

3. better assets_node caches key

The main purpose of this part is to improve ormcache containing assets
nodes. The ormcache key contains
- to much context key
- missing session/host/env info
- unwanted boolean options.
- keys leading to the same cache value

The main goal being to reduce the size of the cache keys, decrease the
number of cache entries and improve the cache hit.
This will also make the behaviour more coherent and hopefully less bug
prone because of mismatch in parameters.

The main reason of the orm cache is the slowness of the validation of
the assets. This includes:
- listing files (dedicated orm cache)
- computing version

The cache key was depending on
- `debug`
The only relevant value for debug is "contains assets"
We dont need to differ between debug='', debug='1', debug='test',
and 'debug=assets', 'debug=tests,assets', ...
- `defer_load`, `lazy_load`, `media`
Those values are only useful to generate html node, a leightweight
operations that does not really needs to be in cache. `media` was also
used in the generation but it looks useless if we have the media on the
node. THIS NEEDS TO BE VALIDATED but in any case, since media is not
used to generate the url, it doesn't make sence to use it in the
generation.
The main idea to remove them from the ormcache key is simply to generate
the nodes outide the ormcached values.
-`async_load`
This one is similar to `defer_load` and `lazy_load` but it looks like
it wasn't used anymore. This was simply removed
- context.get('lang')
The only information needed is the direction, rtl or ltr. This means
en and fr languages, despite sharing the same css assets, will duplicate
the ormcache entries.
-`_get_template_cache_keys`
Only the lang and webiste where really relevant in this flow. Other
keys are actually useless in this flow.

Some information used in the generation where not in the orm cache key
- `self.env.user.lang` if there is no lang in the context
- `request.session.get('force_website_id')`
- `request.httprequest.host`
- ...

The proposed solution is to:
- extract any informùation needed from thecontext, request, environment
before entering the ormcache, reduce it to the minimal possible set of
values needed
```
    rtl = self.env['res.lang']._lang_get_direction(self.env.context.get('lang') or self.env.user.lang) == 'rtl'
    assets_params = self.env['ir.asset']._get_assets_params()  # website_id
    debug_assets = debug and 'assets' in debug
```

and remove a leightweight part of the logic

```
    def _get_asset_nodes(self, bundle, css=True, js=True, debug=False, defer_load=False, lazy_load=False, media=None):
        links = self._get_asset_links(bundle, css=css, js=js, debug=debug)
        return self._links_to_nodes(links, defer_load=defer_load, lazy_load=lazy_load, media=media)
```

Where _get_asset_links is the cached part, and _links_to_nodes is the
lightweight part generating the nodes based on the `defer_load`, ....

Additionnal notes:
- data-asset-version and data-asset-bundle are removed from the node
since they don't seem to be used anymore since 65d70acdbf
- async_load is removed since there is no occurence of this in the code.
- a small hack is still needed to pass javascript content instead of
links, this is only to manage css compile error and will hopefully be
removed in the future.
- a context key is still in use to generate the bundle, the
`commit_assetsbundle` but it has no impact on content and will hopefully
be removed in the future.

4. Add test for ormcache hit/miss

In this context, hit/miss is about having the same cache key for the
same result. This test demonstrates the current state, were entries are
create in the ormcache only if the key is really different and will lead
to a different result.

5. remove cache invalidation

This cache invalidation is quite agressive since everytime an
assetbundle is updated, all workers will clear their cache.

The concerned cache by this clear_cache is `_generate_asset_nodes_cache`
throug `_get_asset_nodes`.

The cache is ignored, both in dev=xml and debug=assets.

This clear cache was made conditionnal in 553ea82f81 but this does
not solve an issue we can have in production.

Lets imagine a clean solution
- all sources are updated
- all workers are restarted.

The orm caches are all empty, but since the sources
changed, all bundles will be recomputed. This means that every bundle
updated in database with save_attachement will invalidate the cache of
all workers. Rendering a pdf report of any kind using a specific bundle
will invalidate all cache. Starting a debug=assets for the first time
will invalidate all cache, even if the cache is not used in this case.

But for a regenerated bundle we would expect the ormcache to be:
- empty (did not generate the same bundle yet)
- have the same value (concurrent generation of the same bundle)

Having a different value would mean that the bundle was generated with
another version of the sources. In this case it is maybe even better not
to invalidate the cache since it could lead to an invalidation war
between two workers.

The only case where invalidating this cache is useful is when a bundle
changes, Usually if an ir_asset is created, modified, ...

There is still another rare but possible possibility to have a 404 if
the transaction is rollbacked after populating the assets node cache.
In this case, we only need to clear the cache locally in case of
rollback.

Part-of: odoo/odoo#121376
This commit is contained in:
Xavier-Do
2023-06-10 11:14:11 +02:00
parent e6973df266
commit 6d5d234f15
16 changed files with 445 additions and 383 deletions
Download Patch File Download Diff File Expand all files Collapse all files
addons/bus/controllers/websocket.py
+3 -2
View File
@@ -64,8 +64,9 @@ class WebsocketController(Controller):
prevent new worker versions to be loaded from the browser cache.
"""
bundle = 'bus.websocket_worker_assets'
files, _ = request.env["ir.qweb"]._get_asset_content(bundle)
asset = AssetsBundle(bundle, files)
assets_params = request.env["ir.asset"]._get_assets_params()
files, _ = request.env["ir.qweb"]._get_asset_content(bundle, assets_params=assets_params)
asset = AssetsBundle(bundle, files, assets_params=assets_params)
stream = request.env['ir.binary']._get_stream_from(asset.js(
is_minified="assets" not in request.session.debug
))
addons/test_website/tests/test_qweb.py
+2 -2
View File
@@ -65,11 +65,11 @@ class TestQweb(TransactionCaseWithUserDemo):
<head>
<link type="text/css" rel="stylesheet" href="http://test.external.link/style1.css"/>
<link type="text/css" rel="stylesheet" href="http://test.external.link/style2.css"/>
<link type="text/css" rel="stylesheet" href="http://test.cdn%(css)s" data-asset-bundle="%(asset_xmlid)s" data-asset-version="%(asset_version_css)s"/>
<link type="text/css" rel="stylesheet" href="http://test.cdn%(css)s"/>
<meta/>
<script type="text/javascript" src="http://test.external.link/javascript1.js"></script>
<script type="text/javascript" src="http://test.external.link/javascript2.js"></script>
<script type="text/javascript" src="http://test.cdn%(js)s" data-asset-bundle="%(asset_xmlid)s" data-asset-version="%(asset_version_js)s" onerror="__odooAssetError=1"></script>
<script type="text/javascript" src="http://test.cdn%(js)s" onerror="__odooAssetError=1"></script>
</head>
<body>
<img src="http://test.external.link/img.png" loading="lazy"/>
addons/web/tests/test_assets.py
+1 -1
View File
@@ -34,7 +34,7 @@ class TestAssetsGenerateTimeCommon(odoo.tests.TransactionCase):
start_t = time.time()
css = assets_type == 'css'
js = assets_type == 'js'
self.env['ir.qweb']._generate_asset_nodes(bundle, css=css, js=js)
self.env['ir.qweb']._generate_asset_links(bundle, css=css, js=js)
yield (f'{bundle}.{assets_type}', time.time() - start_t)
except ValueError:
_logger.info('Error detected while generating bundle %r %s', bundle, assets_type)
addons/web/tests/test_assets_xml.py
+1 -7
View File
@@ -63,19 +63,13 @@ class TestStaticInheritanceCommon(odoo.tests.TransactionCase):
def renderBundle(self, debug=False):
files = []
for url in self.template_files:
atype = 'text/xml'
if '.js' in url:
atype = 'text/javascript'
files.append({
'atype': atype,
'url': url,
'filename': url,
'content': None,
'media': None,
})
asset = AssetsBundle('web.test_bundle', files, env=self.env, css=False, js=True, debug='assets' if debug else '')
# to_node return the files descriptions and generate attachments.
asset.to_node(css=False, js=False)
asset = AssetsBundle('web.test_bundle', files, env=self.env, css=False, js=True, debug_assets=debug)
content = asset.xml(show_inherit_info=debug)
return f'<templates xml:space="preserve">\n{content}\n</templates>'
addons/web/tests/test_js.py
+1 -1
View File
@@ -45,7 +45,7 @@ class WebSuite(odoo.tests.HttpCase):
for asset in assets:
filename = asset['filename']
if not filename or asset['atype'] != 'text/javascript':
if not filename or not filename.endswith('.js'):
continue
with open(filename, 'rb') as fp:
if RE_ONLY.search(fp.read().decode('utf-8')):
addons/web_editor/controllers/main.py
+6 -5
View File
@@ -448,9 +448,9 @@ class Web_Editor(http.Controller):
AssetsUtils = request.env['web_editor.assets']
files_data_by_bundle = []
resources_type_info = {'t_call_assets_attribute': 't-js', 'mimetype': 'text/javascript'}
t_call_assets_attribute = 't-js'
if file_type == 'scss':
resources_type_info = {'t_call_assets_attribute': 't-css', 'mimetype': 'text/scss'}
t_call_assets_attribute = 't-css'
# Compile regex outside of the loop
# This will used to exclude library scss files from the result
@@ -460,7 +460,7 @@ class Web_Editor(http.Controller):
url_infos = dict()
for v in views:
for asset_call_node in etree.fromstring(v["arch"]).xpath("//t[@t-call-assets]"):
attr = asset_call_node.get(resources_type_info['t_call_assets_attribute'])
attr = asset_call_node.get(t_call_assets_attribute)
if attr and not json.loads(attr.lower()):
continue
asset_name = asset_call_node.get("t-call-assets")
@@ -468,7 +468,7 @@ class Web_Editor(http.Controller):
# Loop through bundle files to search for file info
files_data = []
for file_info in request.env["ir.qweb"]._get_asset_content(asset_name)[0]:
if file_info["atype"] != resources_type_info['mimetype']:
if file_info["url"].rpartition('.')[2] != file_type:
continue
url = file_info["url"]
@@ -608,7 +608,8 @@ class Web_Editor(http.Controller):
if not bundle_css:
bundle = 'web.assets_frontend'
files, _ = request.env["ir.qweb"]._get_asset_content(bundle)
asset = AssetsBundle(bundle, files)
assets_params = request.env["ir.asset"]._get_assets_params()
asset = AssetsBundle(bundle, files, assets_params=assets_params)
bundle_css = asset.css().index_content
color_search = re.search(r'(?i)--%s:\s+(%s|%s)' % (css_color_value, regex_hex, regex_rgba), bundle_css)
if not color_search:
addons/website/models/ir_asset.py
+19 -13
View File
@@ -10,35 +10,41 @@ class IrAsset(models.Model):
key = fields.Char(copy=False) # used to resolve multiple assets in a multi-website environment
website_id = fields.Many2one('website', ondelete='cascade')
def _get_related_assets(self, domain):
website = self.env['website'].get_current_website(fallback=False)
if website:
domain += website.website_domain()
assets = super()._get_related_assets(domain)
return assets.filter_duplicate()
def _get_assets_params(self):
params = super()._get_assets_params()
params['website_id'] = self.env['website'].get_current_website(fallback=False).id
return params
def _get_active_addons_list(self):
def _get_related_assets(self, domain, website_id=None, **params):
if website_id:
domain += self.env['website'].website_domain(website_id)
assets = super()._get_related_assets(domain, **params)
return assets.filter_duplicate(website_id)
def _get_active_addons_list(self, website_id=None, **params):
"""Overridden to discard inactive themes."""
addons_list = super()._get_active_addons_list()
website = self.env['website'].get_current_website(fallback=False)
addons_list = super()._get_active_addons_list(**params)
if not website:
if not website_id:
return addons_list
IrModule = self.env['ir.module.module'].sudo()
# discard all theme modules except website.theme_id
themes = IrModule.search(IrModule.get_themes_domain()) - website.theme_id
themes = IrModule.search(IrModule.get_themes_domain()) - self.env["website"].browse(website_id).theme_id
to_remove = set(themes.mapped('name'))
return [name for name in addons_list if name not in to_remove]
def filter_duplicate(self):
def filter_duplicate(self, website_id=None):
""" Filter current recordset only keeping the most suitable asset per distinct name.
Every non-accessible asset will be removed from the set:
* In non website context, every asset with a website will be removed
* In a website context, every asset from another website
"""
current_website = self.env['website'].get_current_website(fallback=False)
if website_id is not None:
current_website = self.env['website'].browse(website_id)
else:
current_website = self.env['website'].get_current_website(fallback=False)
if not current_website:
return self.filtered(lambda asset: not asset.website_id)
addons/website/models/ir_qweb.py
+9 -10
View File
@@ -20,14 +20,14 @@ re_background_image = re.compile(r"(background-image\s*:\s*url\(\s*['\"]?\s*)([^
class AssetsBundleMultiWebsite(AssetsBundle):
def _get_asset_url_values(self, id, unique, extra, name, sep, extension):
website_id = self.env.context.get('website_id')
website_id = self.assets_params.get('website_id')
website_id_path = website_id and ('%s/' % website_id) or ''
extra = website_id_path + extra
res = super(AssetsBundleMultiWebsite, self)._get_asset_url_values(id, unique, extra, name, sep, extension)
return res
def get_debug_asset_url(self, extra='', name='%', extension='%'):
website_id = self.env.context.get('website_id')
website_id = self.assets_params.get('website_id')
website_id_path = website_id and ('%s/' % website_id) or ''
extra = website_id_path + extra
return super(AssetsBundleMultiWebsite, self).get_debug_asset_url(extra, name, extension)
@@ -106,8 +106,8 @@ class IrQWeb(models.AbstractModel):
return irQweb
def _get_asset_bundle(self, xmlid, files, env=None, css=True, js=True, debug=False):
return AssetsBundleMultiWebsite(xmlid, files, env=env, css=css, js=js, debug=debug)
def _get_asset_bundle(self, xmlid, files=None, env=None, css=True, js=True, debug_assets=False, rtl=False, assets_params=None):
return AssetsBundleMultiWebsite(xmlid, files, env=env, css=css, js=js, debug_assets=debug_assets, rtl=rtl, assets_params=assets_params)
def _post_processing_att(self, tagName, atts):
if atts.get('data-no-post-process'):
@@ -161,14 +161,13 @@ class IrQWeb(models.AbstractModel):
# version part combine with the initial extra (rtl) should be enough to ensure they are identical.
# we dont expect to have any pregenerated rtl/website attachment so we don't manage assets with extra
nodes = super()._pregenerate_assets_bundles()
links = super()._pregenerate_assets_bundles()
website = self.env['website'].search([], order='id', limit=1)
if not website:
return nodes
return links
nb_created = 0
for node in nodes:
bundle_info = node[1]
bundle_url = bundle_info.get('src', '') or bundle_info.get('href', '')
for link in links:
bundle_url = link[0]
if bundle_url.startswith('/web/assets/'):
# example: "/web/assets/2152-ee56665/web.assets_frontend_lazy.min.js"
_, _, _, id_unique, name = bundle_url.split('/')
@@ -189,4 +188,4 @@ class IrQWeb(models.AbstractModel):
if nb_created:
_logger.runbot('%s bundle(s) were copied for website %s', nb_created, website.id)
return nodes
return links
addons/website/tests/test_ir_asset.py
+2 -2
View File
@@ -39,12 +39,12 @@ class TestIrAsset(odoo.tests.HttpCase):
'active': False,
})
files = IrAsset.with_context(website_id=website_1.id)._get_asset_paths('test_bundle.irasset', css=True)
files = IrAsset._get_asset_paths('test_bundle.irasset', {'website_id': website_1.id})
self.assertEqual(len(files), 2, "There should be two assets in the specific website.")
self.assertEqual(files[0][0], '/website/test/base0.css', "First asset should be the same as the base one.")
self.assertEqual(files[1][0], '/website/test/specific1.css', "Second asset should be the specific one.")
files = IrAsset.with_context(website_id=website_2.id)._get_asset_paths('test_bundle.irasset', css=True)
files = IrAsset._get_asset_paths('test_bundle.irasset', {'website_id': website_2.id})
self.assertEqual(len(files), 3, "All three assets should be in the unmodified website.")
self.assertEqual(files[0][0], '/website/test/base0.css', "First asset should be the base one.")
self.assertEqual(files[1][0], '/website/test/base1.css', "Second asset should be the base one.")
addons/website/tests/test_performance.py
+1 -1
View File
@@ -146,7 +146,7 @@ class TestWebsitePerformancePost(UtilPerf):
@mute_logger('odoo.http')
def test_50_perf_sql_web_assets(self):
# assets route /web/assets/..
self.env['ir.qweb']._generate_asset_nodes('web.assets_frontend_lazy', css=False, js=True)
self.env['ir.qweb']._generate_asset_links('web.assets_frontend_lazy', css=False, js=True)
assets_url = self.env['ir.attachment'].search([('url', '=like', '/web/assets/%/web.assets_frontend_lazy%.js')], limit=1).url
self.assertEqual(self._get_url_hot_query(assets_url), 4)
self.assertEqual(self._get_url_hot_query(assets_url, cache=False), 4)
addons/website/tests/test_ui.py
+16 -1
View File
@@ -222,7 +222,9 @@ class TestUi(odoo.tests.HttpCase):
self.start_tour("/", 'website_navbar_menu')
def test_05_specific_website_editor(self):
asset_bundle_xmlid = 'website.assets_wysiwyg'
website_default = self.env['website'].search([], limit=1)
new_website = self.env['website'].create({'name': 'New Website'})
code = b"document.body.dataset.hello = 'world';"
@@ -231,7 +233,7 @@ class TestUi(odoo.tests.HttpCase):
'mimetype': 'text/javascript',
'datas': base64.b64encode(code),
})
custom_url = '/web/content/%s/%s' % (attach.id, attach.name)
custom_url = '/_custom/web/content/%s/%s' % (attach.id, attach.name)
attach.url = custom_url
self.env['ir.asset'].create({
@@ -240,6 +242,19 @@ class TestUi(odoo.tests.HttpCase):
'path': custom_url,
'website_id': new_website.id,
})
files, _ = self.env['ir.qweb']._get_asset_content(asset_bundle_xmlid, assets_params={'website_id': website_default.id})
self.assertNotIn(custom_url, [f['url'] for f in files])
base_website_bundle = self.env['ir.qweb']._get_asset_bundle(asset_bundle_xmlid, files, env=self.env, assets_params={'website_id': website_default.id})
base_website_css_version = base_website_bundle.get_version('css')
base_website_js_version = base_website_bundle.get_version('js')
files, _ = self.env['ir.qweb']._get_asset_content('website.assets_wysiwyg', assets_params={'website_id': new_website.id})
self.assertIn(custom_url, [f['url'] for f in files])
new_website_bundle_modified = self.env['ir.qweb']._get_asset_bundle(asset_bundle_xmlid, files, env=self.env, assets_params={'website_id': new_website.id})
self.assertEqual(new_website_bundle_modified.get_version('css'), base_website_css_version)
self.assertNotEqual(new_website_bundle_modified.get_version('js'), base_website_js_version, "js version for new website should now have been changed")
url_params = url_encode({'path': '/@/'})
self.start_tour(f'/website/force/{website_default.id}?{url_params}', "generic_website_editor", login='admin')
self.start_tour(f'/website/force/{new_website.id}?{url_params}', "specific_website_editor", login='admin')
odoo/addons/base/models/assetsbundle.py
+52 -123
View File
@@ -57,7 +57,7 @@ class AssetsBundle(object):
TRACKED_BUNDLES = ['web.assets_common', 'web.assets_backend']
def __init__(self, name, files, env=None, css=True, js=True, debug=None):
def __init__(self, name, files, env=None, css=True, js=True, debug_assets=False, rtl=False, assets_params=None):
"""
:param name: bundle name
:param files: files to be added to the bundle
@@ -71,16 +71,16 @@ class AssetsBundle(object):
self.stylesheets = []
self.css_errors = []
self.files = files
self.user_direction = self.env['res.lang']._lang_get(
self.env.context.get('lang') or self.env.user.lang
).direction
self.rtl = rtl
self.assets_params = assets_params or {}
self.has_css = css
self.has_js = js
self._checksum_cache = {}
self.is_debug_assets = debug and 'assets' in debug
self.is_debug_assets = debug_assets
# asset-wide html "media" attribute
for f in files:
extension = f['url'].rpartition('.')[2]
params = {
'url': f['url'],
'filename': f['filename'],
@@ -89,45 +89,37 @@ class AssetsBundle(object):
}
if css:
css_params = {
'media': f['media'],
'direction': self.user_direction,
'rtl': self.rtl,
}
if f['atype'] == 'text/sass':
if extension == 'sass':
self.stylesheets.append(SassStylesheetAsset(self, **params, **css_params))
elif f['atype'] == 'text/scss':
elif extension == 'scss':
self.stylesheets.append(ScssStylesheetAsset(self, **params, **css_params))
elif f['atype'] == 'text/less':
elif extension == 'less':
self.stylesheets.append(LessStylesheetAsset(self, **params, **css_params))
elif f['atype'] == 'text/css':
elif extension == 'css':
self.stylesheets.append(StylesheetAsset(self, **params, **css_params))
if js:
if f['atype'] == 'text/javascript':
if extension == 'js':
self.javascripts.append(JavascriptAsset(self, **params))
elif f['atype'] == 'text/xml':
elif extension == 'xml':
self.templates.append(XMLAsset(self, **params))
def to_node(self, css=True, js=True, async_load=False, defer_load=False, lazy_load=False):
def get_links(self):
"""
:returns [(tagName, attributes, content)] if the tag is auto close
:returns a list of tuple. a tuple can be (url, None) or (None, inlineContent)
"""
response = []
if css and self.stylesheets:
if self.has_css and self.stylesheets:
css_attachments = self.css(is_minified=not self.is_debug_assets) or []
for attachment in css_attachments:
if self.is_debug_assets:
href = self.get_debug_asset_url(extra='rtl/' if self.user_direction == 'rtl' else '',
href = self.get_debug_asset_url(extra='rtl/' if self.rtl else '',
name=css_attachments.name,
extension='')
else:
href = attachment.url
attr = dict([
["type", "text/css"],
["rel", "stylesheet"],
["href", href],
['data-asset-bundle', self.name],
['data-asset-version', self.get_version('css')],
])
response.append(("link", attr, None))
response.append((href, None))
if self.css_errors:
msg = '\n'.join(self.css_errors).replace('"', '\\"').replace('\n', '\\n')
js_error = """
@@ -139,24 +131,13 @@ class AssetsBundle(object):
}
});
""" % msg
response.append(JavascriptAsset(self, inline=js_error).to_node())
response.append(StylesheetAsset(self, url="/web/static/lib/bootstrap/dist/css/bootstrap.css").to_node())
response.append((None, js_error))
response.append(('/web/static/lib/bootstrap/dist/css/bootstrap.css', None))
if js and self.javascripts:
if self.has_js and self.javascripts:
js_attachment = self.js(is_minified=not self.is_debug_assets)
src = self.get_debug_asset_url(name=js_attachment.name, extension='') if self.is_debug_assets else js_attachment[0].url
attr = dict([
["async", "async" if async_load else None],
# lazy_load will add defer in JS otherwise this is not W3C valid
# (defer is probably not needed there anyways)
["defer", "defer" if defer_load else None],
["type", "text/javascript"],
["data-src" if lazy_load else "src", src],
['data-asset-bundle', self.name],
['data-asset-version', self.get_version('js')],
['onerror', '__odooAssetError=1']
])
response.append(("script", attr, None))
response.append((src, None))
return response
@@ -192,6 +173,7 @@ class AssetsBundle(object):
'name': name,
'sep': sep,
'extension': extension,
'params': self.assets_params,
}
def get_asset_url(self, id='%', unique='%', extra='', name='%', sep="%", extension='%'):
@@ -229,10 +211,10 @@ class AssetsBundle(object):
ira = self.env['ir.attachment']
is_css = extension in ['css', 'min.css', 'css.map']
url = self.get_asset_url(
extra='%s' % ('rtl/' if is_css and self.user_direction == 'rtl' else ''),
extra='%s' % ('rtl/' if is_css and self.rtl else ''),
name=self.name,
sep='',
extension='.%s' % extension
sep='.',
extension=extension,
)
domain = [
@@ -244,8 +226,7 @@ class AssetsBundle(object):
if attachments:
self._unlink_attachments(attachments)
# force bundle invalidation on other workers
self.env['ir.qweb'].clear_caches()
# clear_cache was removed
return True
@@ -268,10 +249,10 @@ class AssetsBundle(object):
url_pattern = self.get_asset_url(
unique=unique,
extra='%s' % ('rtl/' if is_css and self.user_direction == 'rtl' else ''), # not sure about css.map
extra='%s' % ('rtl/' if is_css and self.rtl else ''), # not sure about css.map
name=self.name,
sep='',
extension='.%s' % extension
sep='.',
extension=extension,
)
self.env.cr.execute("""
SELECT max(id)
@@ -285,6 +266,14 @@ class AssetsBundle(object):
attachment_ids = [r[0] for r in self.env.cr.fetchall()]
return self.env['ir.attachment'].sudo().browse(attachment_ids)
def add_post_rollback(self):
"""
In some rare cases it is possible that an attachment is created during a transaction, added to the ormcache
but the transaction is rollbacked, leading to 404 when getting the attachments.
This postrollback hook will help fix this issue by clearing the cache if it isn't commited.
"""
self.env.cr.postrollback.add(self.env.registry._Registry__cache.clear)
def save_attachment(self, extension, content):
"""Record the given bundle in an ir.attachment and delete
all other ir.attachments referring to this bundle (with the same name and extension).
@@ -318,19 +307,17 @@ class AssetsBundle(object):
'public': True,
'raw': content.encode('utf8'),
}
self.add_post_rollback()
attachment = ira.with_user(SUPERUSER_ID).create(values)
url = self.get_asset_url(
id=attachment.id,
unique=self.get_version('css' if is_css else 'js'),
extra='%s' % ('rtl/' if extension in ['css', 'min.css'] and self.user_direction == 'rtl' else ''),
name=fname,
sep='', # included in fname
extension=''
extra='%s' % ('rtl/' if extension in ['css', 'min.css'] and self.rtl else ''),
name=self.name,
sep='.', # included in fname
extension=extension,
)
values = {
'url': url,
}
attachment.write(values)
attachment.url = url
if self.env.context.get('commit_assetsbundle') is True:
self.env.cr.commit()
@@ -588,7 +575,7 @@ class AssetsBundle(object):
sourcemap_attachment = self.get_attachments('css.map') \
or self.save_attachment('css.map', '')
debug_asset_url = self.get_debug_asset_url(name=self.name,
extra='rtl/' if self.user_direction == 'rtl' else '')
extra='rtl/' if self.rtl else '')
generator = SourceMapGenerator(
source_root="/".join(
[".." for i in range(0, len(debug_asset_url.split("/")) - 2)]
@@ -634,7 +621,7 @@ class AssetsBundle(object):
compiled += self.compile_css(assets[0].compile, source)
# We want to run rtlcss on normal css, so merge it in compiled
if self.user_direction == 'rtl':
if self.rtl:
stylesheet_assets = [asset for asset in self.stylesheets if not isinstance(asset, (SassStylesheetAsset, ScssStylesheetAsset, LessStylesheetAsset))]
compiled += '\n'.join([asset.get_source() for asset in stylesheet_assets])
compiled = self.run_rtlcss(compiled)
@@ -798,9 +785,6 @@ class WebAsset(object):
except ValueError:
raise AssetNotFound("Could not find %s" % self.name)
def to_node(self):
raise NotImplementedError()
@property
def last_modified(self):
if self._last_modified is None:
@@ -882,21 +866,6 @@ class JavascriptAsset(WebAsset):
except AssetError as e:
return u"console.error(%s);" % json.dumps(to_text(e))
def to_node(self):
if self.url:
return ("script", dict([
["type", "text/javascript"],
["src", self.html_url],
['data-asset-bundle', self.bundle.name],
['data-asset-version', self.bundle_version],
]), None)
else:
return ("script", dict([
["type", "text/javascript"],
["charset", "utf-8"],
['data-asset-bundle', self.bundle.name],
['data-asset-version', self.bundle_version],
]), self.with_header())
def with_header(self, content=None, minimal=True):
if minimal:
@@ -927,7 +896,7 @@ class XMLAsset(WebAsset):
try:
content = super()._fetch_content()
except AssetError as e:
return f'<error data-asset-bundle={self.bundle.name!r} data-asset-version={self.bundle_version!r}>{json.dumps(to_text(e))}</error>'
return u"console.error(%s);" % json.dumps(to_text(e))
parser = etree.XMLParser(ns_clean=True, recover=True, remove_comments=True)
root = etree.parse(io.BytesIO(content.encode('utf-8')), parser=parser).getroot()
@@ -939,17 +908,6 @@ class XMLAsset(WebAsset):
def bundle_version(self):
return self.bundle.get_version('js')
def to_node(self):
attributes = {
'async': 'async',
'defer': 'defer',
'type': 'text/xml',
'data-src': self.html_url,
'data-asset-bundle': self.bundle.name,
'data-asset-version': self.bundle_version,
}
return ("script", attributes, None)
def with_header(self, content=None):
if content is None:
content = self.content
@@ -981,31 +939,22 @@ class StylesheetAsset(WebAsset):
rx_sourceMap = re.compile(r'(/\*# sourceMappingURL=.*)', re.U)
rx_charset = re.compile(r'(@charset "[^"]+";)', re.U)
def __init__(self, *args, media=None, direction=None, **kw):
self.media = media
self.direction = direction
def __init__(self, *args, rtl=False, **kw):
self.rtl = rtl
super().__init__(*args, **kw)
if self.direction == 'rtl' and self.url:
if self.rtl and self.url:
self.html_url_args = self.url.rsplit('.', 1)
self.html_url_format = '%%s/%s/%s.%%s' % ('rtl', self.bundle.name)
self.html_url_args = tuple(self.html_url_args)
@property
def content(self):
content = super().content
if self.media:
content = '@media %s { %s }' % (self.media, content) # this is not good!
return content
@property
def bundle_version(self):
return self.bundle.get_version('css')
@func.lazy_property
def unique_descriptor(self):
direction = self.direction or ''
media = self.media or ''
return f'{self.url or self.inline},{self.last_modified},{direction},{media}' # note that media shouldn't be used in unique since it isn't in extra
direction = (self.rtl and 'rtl') or 'ltr'
return f'{self.url or self.inline},{self.last_modified},{direction}'
def _fetch_content(self):
try:
@@ -1047,26 +996,6 @@ class StylesheetAsset(WebAsset):
content = re.sub(r' *([{}]) *', r'\1', content)
return self.with_header(content)
def to_node(self):
if self.url:
attr = dict([
["type", "text/css"],
["rel", "stylesheet"],
["href", self.html_url],
["media", escape(to_text(self.media)) if self.media else None],
['data-asset-bundle', self.bundle.name],
['data-asset-version', self.bundle_version],
])
return ("link", attr, None)
else:
attr = dict([
["type", "text/css"],
["media", escape(to_text(self.media)) if self.media else None],
['data-asset-bundle', self.bundle.name],
['data-asset-version', self.bundle_version],
])
return ("style", attr, self.with_header())
class PreprocessedCSS(StylesheetAsset):
rx_import = None
@@ -1074,7 +1003,7 @@ class PreprocessedCSS(StylesheetAsset):
def __init__(self, *args, **kw):
super().__init__(*args, **kw)
self.html_url_args = tuple(self.url.rsplit('/', 1))
self.html_url_format = '%%s/%s%s/%%s.css' % ('rtl/' if self.direction == 'rtl' else '', self.bundle.name)
self.html_url_format = '%%s/%s%s/%%s.css' % ('rtl/' if self.rtl else '', self.bundle.name)
def get_command(self):
raise NotImplementedError
odoo/addons/base/models/ir_asset.py
+34 -40
View File
@@ -8,8 +8,7 @@ import odoo
import odoo.modules.module # get_manifest, don't from-import it
from odoo import api, fields, models, tools
from odoo.tools import misc
from odoo.tools.constants import SCRIPT_EXTENSIONS, STYLE_EXTENSIONS, TEMPLATE_EXTENSIONS, ASSET_EXTENSIONS, EXTERNAL_ASSET
from odoo.tools.constants import ASSET_EXTENSIONS, EXTERNAL_ASSET
_logger = getLogger(__name__)
@@ -91,7 +90,18 @@ class IrAsset(models.Model):
active = fields.Boolean(string='active', default=True)
sequence = fields.Integer(string="Sequence", default=DEFAULT_SEQUENCE, required=True)
def _get_asset_paths(self, bundle, css=False, js=False):
def _get_assets_params(self):
"""
This method can be overriden to add param _get_asset_paths call.
Those params will be part of the orm cache key
"""
return {}
@tools.conditional(
'xml' not in tools.config['dev_mode'],
tools.ormcache('bundle', 'tuple(sorted(assets_params.items()))'),
)
def _get_asset_paths(self, bundle, assets_params):
"""
Fetches all asset file paths from a given list of addons matching a
certain bundle. The returned list is composed of tuples containing the
@@ -109,27 +119,21 @@ class IrAsset(models.Model):
records matching the bundle are also applied to the current list.
:param bundle: name of the bundle from which to fetch the file paths
:param css: boolean: whether or not to include style files
:param js: boolean: whether or not to include script files and template
files
:param assets_params: parameters needed by overrides, mainly website_id
see _get_assets_params
:returns: the list of tuples (path, addon, bundle)
"""
installed = self._get_installed_addons_list()
addons = self._get_active_addons_list()
addons = self._get_active_addons_list(**assets_params)
asset_paths = AssetPaths()
exts = []
if js:
exts += SCRIPT_EXTENSIONS
exts += TEMPLATE_EXTENSIONS
if css:
exts += STYLE_EXTENSIONS
addons = self._topological_sort(tuple(addons))
self._fill_asset_paths(bundle, addons, installed, exts, asset_paths, [])
self._fill_asset_paths(bundle, asset_paths, [], addons, installed, **assets_params)
return asset_paths.list
def _fill_asset_paths(self, bundle, addons, installed, exts, asset_paths, seen):
def _fill_asset_paths(self, bundle, asset_paths, seen, addons, installed, **assets_params):
"""
Fills the given AssetPaths instance by applying the operations found in
the matching bundle of the given addons manifests.
@@ -150,22 +154,22 @@ class IrAsset(models.Model):
# of the CURRENT bundle.
bundle_start_index = len(asset_paths.list)
assets = self._get_related_assets([('bundle', '=', bundle)]).filtered('active')
assets = self._get_related_assets([('bundle', '=', bundle)], **assets_params).filtered('active')
# 1. Process the first sequence of 'ir.asset' records
for asset in assets.filtered(lambda a: a.sequence < DEFAULT_SEQUENCE):
self._process_path(bundle, asset.directive, asset.target, asset.path, addons, installed, exts, asset_paths, seen, bundle_start_index)
self._process_path(bundle, asset.directive, asset.target, asset.path, asset_paths, seen, addons, installed, bundle_start_index, **assets_params)
# 2. Process all addons' manifests.
for addon in addons:
for command in odoo.modules.module.get_manifest(addon)['assets'].get(bundle, ()):
directive, target, path_def = self._process_command(command)
self._process_path(bundle, directive, target, path_def, addons, installed, exts, asset_paths, seen, bundle_start_index)
self._process_path(bundle, directive, target, path_def, asset_paths, seen, addons, installed, bundle_start_index, **assets_params)
# 3. Process the rest of 'ir.asset' records
for asset in assets.filtered(lambda a: a.sequence >= DEFAULT_SEQUENCE):
self._process_path(bundle, asset.directive, asset.target, asset.path, addons, installed, exts, asset_paths, seen, bundle_start_index)
self._process_path(bundle, asset.directive, asset.target, asset.path, asset_paths, seen, addons, installed, bundle_start_index, **assets_params)
def _process_path(self, bundle, directive, target, path_def, addons, installed, exts, asset_paths, seen, bundle_start_index):
def _process_path(self, bundle, directive, target, path_def, asset_paths, seen, addons, installed, bundle_start_index, **assets_params):
"""
This sub function is meant to take a directive and a set of
arguments and apply them to the current asset_paths list
@@ -180,18 +184,17 @@ class IrAsset(models.Model):
"""
if directive == INCLUDE_DIRECTIVE:
# recursively call this function for each INCLUDE_DIRECTIVE directive.
self._fill_asset_paths(path_def, addons, installed, exts, asset_paths, seen + [bundle])
self._fill_asset_paths(path_def, asset_paths, seen + [bundle], addons, installed, **assets_params)
return
if can_aggregate(path_def):
paths = self._get_paths(path_def, installed, exts)
paths = self._get_paths(path_def, installed)
else:
paths = [(path_def, EXTERNAL_ASSET, -1)] # external urls
# retrieve target index when it applies
if directive in DIRECTIVES_WITH_TARGET:
target_paths = self._get_paths(target, installed, exts)
if not target_paths and target.rpartition('.')[2] not in exts:
target_paths = self._get_paths(target, installed)
if not target_paths and target.rpartition('.')[2] not in ASSET_EXTENSIONS:
# nothing to do: the extension of the target is wrong
return
if target_paths:
@@ -237,14 +240,10 @@ class IrAsset(models.Model):
:root_bundle: string: bundle from which to initiate the search.
:returns: the first matching bundle or None
"""
ext = target_path_def.split('.')[-1]
installed = self._get_installed_addons_list()
target_path, _full_path, _modified = self._get_paths(target_path_def, installed)[0]
css = ext in STYLE_EXTENSIONS
js = ext in SCRIPT_EXTENSIONS or ext in TEMPLATE_EXTENSIONS
asset_paths = self._get_asset_paths(root_bundle, css=css, js=js)
assets_params = self._get_assets_params()
asset_paths = self._get_asset_paths(root_bundle, assets_params)
for path, _full_path, bundle, _modified in asset_paths:
if path == target_path:
@@ -280,7 +279,7 @@ class IrAsset(models.Model):
return misc.topological_sort({manif['name']: manif['depends'] for manif in manifs})
@api.model
@tools.ormcache_context(keys='install_module')
@tools.ormcache()
def _get_installed_addons_list(self):
"""
Returns the list of all installed addons.
@@ -288,10 +287,9 @@ class IrAsset(models.Model):
"""
# Main source: the current registry list
# Second source of modules: server wide modules
# Third source: the currently loading module from the context (similar to ir_ui_view)
return self.env.registry._init_modules.union(odoo.conf.server_wide_modules or []).union(self.env.context.get('install_module', []))
return self.env.registry._init_modules.union(odoo.conf.server_wide_modules or [])
def _get_paths(self, path_def, installed, extensions=None):
def _get_paths(self, path_def, installed):
"""
Returns a list of tuple (path, full_path, modified) matching a given glob (path_def).
The glob can only occur in the static direcory of an installed addon.
@@ -352,11 +350,7 @@ class IrAsset(models.Model):
msg += " It may be due to security reasons."
_logger.warning(msg)
# Paths are filtered on the extensions (if any).
return [
path
for path in paths
if not extensions or path[0].rsplit('.', maxsplit=1)[-1] in extensions
]
return paths
def _process_command(self, command):
"""Parses a given command to return its directive, target and path definition."""
odoo/addons/base/models/ir_qweb.py
+92 -69
View File
@@ -2154,7 +2154,7 @@ class IrQWeb(models.AbstractModel):
xmlid = el.attrib.pop('t-call-assets')
css = self._compile_bool(el.attrib.pop('t-css', True))
js = self._compile_bool(el.attrib.pop('t-js', True))
async_load = self._compile_bool(el.attrib.pop('async_load', False))
# async_load support was removed
defer_load = self._compile_bool(el.attrib.pop('defer_load', False))
lazy_load = self._compile_bool(el.attrib.pop('lazy_load', False))
media = el.attrib.pop('media', False)
@@ -2164,7 +2164,6 @@ class IrQWeb(models.AbstractModel):
css={css},
js={js},
debug=values.get("debug"),
async_load={async_load},
defer_load={defer_load},
lazy_load={lazy_load},
media={media!r},
@@ -2412,15 +2411,28 @@ class IrQWeb(models.AbstractModel):
return (attributes, content, inherit_branding)
def _get_asset_nodes(self, bundle, css=True, js=True, debug=False, async_load=False, defer_load=False, lazy_load=False, media=None):
def _get_asset_nodes(self, bundle, css=True, js=True, debug=False, defer_load=False, lazy_load=False, media=None):
"""Generates asset nodes.
If debug=assets, the assets will be regenerated when a file which composes them has been modified.
Else, the assets will be generated only once and then stored in cache.
"""
if debug and 'assets' in debug:
return self._generate_asset_nodes(bundle, css, js, debug, async_load, defer_load, lazy_load, media)
media = css and media or None
links = self._get_asset_links(bundle, css=css, js=js, debug=debug)
return self._links_to_nodes(links, defer_load=defer_load, lazy_load=lazy_load, media=media)
def _get_asset_links(self, bundle, css=True, js=True, debug=None):
"""Generates asset nodes.
If debug=assets, the assets will be regenerated when a file which composes them has been modified.
Else, the assets will be generated only once and then stored in cache.
"""
rtl = self.env['res.lang']._lang_get_direction(self.env.context.get('lang') or self.env.user.lang) == 'rtl'
assets_params = self.env['ir.asset']._get_assets_params() # website_id
debug_assets = debug and 'assets' in debug
if debug_assets:
return self._generate_asset_links(bundle, css=css, js=js, debug_assets=True, assets_params=assets_params, rtl=rtl)
else:
return self._generate_asset_nodes_cache(bundle, css, js, debug, async_load, defer_load, lazy_load, media)
return self._generate_asset_links_cache(bundle, css=css, js=js, assets_params=assets_params, rtl=rtl)
# qweb cache feature
@@ -2472,80 +2484,91 @@ class IrQWeb(models.AbstractModel):
# in non-xml-debug mode we want assets to be cached forever, and the admin can force a cache clear
# by restarting the server after updating the source code (or using the "Clear server cache" in debug tools)
'xml' not in tools.config['dev_mode'],
tools.ormcache('bundle', 'css', 'js', 'debug', 'async_load', 'defer_load', 'lazy_load', 'media', 'tuple(self.env.context.get(k) for k in self._get_template_cache_keys())'),
tools.ormcache('bundle', 'css', 'js', 'tuple(sorted(assets_params.items()))', 'rtl'),
)
def _generate_asset_nodes_cache(self, bundle, css=True, js=True, debug=False, async_load=False, defer_load=False, lazy_load=False, media=None):
return self._generate_asset_nodes(bundle, css, js, debug, async_load, defer_load, lazy_load, media)
def _generate_asset_links_cache(self, bundle, css=True, js=True, assets_params=None, rtl=False):
return self._generate_asset_links(bundle, css, js, False, assets_params, rtl)
@tools.ormcache('bundle', 'defer_load', 'lazy_load', 'media', 'tuple(self.env.context.get(k) for k in self._get_template_cache_keys())')
def _get_asset_content(self, bundle, defer_load=False, lazy_load=False, media=None):
asset_paths = self.env['ir.asset']._get_asset_paths(bundle=bundle, css=True, js=True)
def _get_asset_content(self, bundle, assets_params=None):
if assets_params is None:
assets_params = self.env['ir.asset']._get_assets_params() # website_id
asset_paths = self.env['ir.asset']._get_asset_paths(bundle=bundle, assets_params=assets_params)
files = []
remains = []
external_asset = []
for path, full_path, _bundle, last_modified in asset_paths:
ext = path.rpartition('.')[2]
is_js = ext in SCRIPT_EXTENSIONS
is_xml = ext in TEMPLATE_EXTENSIONS
is_css = ext in STYLE_EXTENSIONS
if not is_js and not is_xml and not is_css:
continue
mimetype = None
if is_js:
mimetype = 'text/javascript'
elif is_css:
mimetype = f'text/{ext}'
elif is_xml:
mimetype = 'text/xml'
if full_path is not EXTERNAL_ASSET:
files.append({
'atype': mimetype,
'url': path,
'filename': full_path,
'content': '',
'media': media,
'last_modified': last_modified,
})
else:
if is_js:
tag = 'script'
attributes = {
"type": mimetype,
}
attributes["data-src" if lazy_load else "src"] = path
if defer_load:
# lazy_load will add defer in JS otherwise this is not
# W3C valid (defer is probably not needed there anyways)
attributes["defer"] = "defer"
elif is_css:
tag = 'link'
attributes = {
"type": mimetype,
"rel": "stylesheet",
"href": path,
'media': media,
}
elif is_xml:
tag = 'script'
attributes = {
"type": mimetype,
"async": "async",
"rel": "prefetch",
"data-src": path,
}
remains.append((tag, attributes, None))
external_asset.append((path, None))
return (files, external_asset)
return (files, remains)
def _get_asset_bundle(self, bundle_name, files=None, env=None, css=True, js=True, debug_assets=False, rtl=False, assets_params=None):
return AssetsBundle(bundle_name, files, env=env, css=css, js=js, debug_assets=debug_assets, rtl=rtl, assets_params=assets_params)
def _get_asset_bundle(self, bundle_name, files, env=None, css=True, js=True, debug=False):
return AssetsBundle(bundle_name, files, env=env, css=css, js=js, debug=debug)
def _links_to_nodes(self, paths, defer_load=False, lazy_load=False, media=None):
return [self._link_to_node(path, defer_load=defer_load, lazy_load=lazy_load, media=media) for path in paths]
def _generate_asset_nodes(self, bundle, css=True, js=True, debug=False, async_load=False, defer_load=False, lazy_load=False, media=None):
files, remains = self._get_asset_content(bundle, defer_load=defer_load, lazy_load=lazy_load, media=css and media or None)
asset = self._get_asset_bundle(bundle, files, env=self.env, css=css, js=js, debug=debug)
remains = [node for node in remains if (css and node[0] == 'link') or (js and node[0] == 'script')]
return remains + asset.to_node(css=css, js=js, async_load=async_load, defer_load=defer_load, lazy_load=lazy_load)
def _link_to_node(self, descriptor, defer_load=False, lazy_load=False, media=None):
path, js_content = descriptor # small hack to manage css compilation error, to transform into a full css solution or saved js error
ext = path.rsplit('.', maxsplit=1)[-1] if path else 'js'
is_js = ext in SCRIPT_EXTENSIONS
is_xml = ext in TEMPLATE_EXTENSIONS
is_css = ext in STYLE_EXTENSIONS
if not is_js and not is_xml and not is_css:
return
if is_js:
is_asset_bundle = path and path.startswith('/web/assets/')
attributes = {
'type': 'text/javascript',
}
if (defer_load or lazy_load):
attributes['defer'] = 'defer'
if path:
if lazy_load:
attributes['data-src'] = path
else:
attributes['src'] = path
if is_asset_bundle:
attributes['onerror'] = "__odooAssetError=1"
if js_content:
attributes['charset'] = 'utf-8'
return ('script', attributes, js_content)
if is_css:
attributes = {
'type': f'text/{ext}', # we don't really expect to have anything else than pure css here
'rel': 'stylesheet',
'href': path,
'media': media,
}
return ('link', attributes, None)
if is_xml:
attributes = {
'type': 'text/xml',
'async': 'async',
'rel': 'prefetch',
'data-src': path,
}
return ('script', attributes, None)
def _generate_asset_links(self, bundle, css=True, js=True, debug_assets=False, assets_params=None, rtl=False):
aggregable_files, external_assets = self._get_asset_content(bundle, assets_params=assets_params)
external_assets = [node for node in external_assets if (css and node[0].rpartition('.')[2] in STYLE_EXTENSIONS) or (js and node[0].rpartition('.')[2] in SCRIPT_EXTENSIONS)]
asset_bundle = self._get_asset_bundle(bundle, aggregable_files, env=self.env, css=css, js=js, debug_assets=debug_assets, rtl=rtl, assets_params=assets_params)
bundle_links = asset_bundle.get_links()
return external_assets + bundle_links
def _get_asset_link_urls(self, bundle, debug=False):
asset_nodes = self._get_asset_nodes(bundle, js=False, debug=debug)
@@ -2575,16 +2598,16 @@ class IrQWeb(models.AbstractModel):
js_bundles.add(asset)
if css:
css_bundles.add(asset)
nodes = []
links = []
start = time.time()
for bundle in sorted(js_bundles):
nodes += self._generate_asset_nodes(bundle, css=False, js=True)
links += self._generate_asset_links(bundle, css=False, js=True)
_logger.info('JS Assets bundles generated in %s seconds', time.time()-start)
start = time.time()
for bundle in sorted(css_bundles):
nodes += self._generate_asset_nodes(bundle, css=True, js=False)
links += self._generate_asset_links(bundle, css=True, js=False)
_logger.info('CSS Assets bundles generated in %s seconds', time.time()-start)
return nodes
return links
def render(template_name, values, load, **options):
odoo/addons/test_assetsbundle/tests/test_assetsbundle.py
+205 -106
View File
@@ -122,47 +122,57 @@ class FileTouchable(AddonManifestPatched):
self.touches[filepath] = touch_time or time.time()
return patch('os.path.getmtime', lambda filename: self.touches.get(filename) or GETMTINE(filename))
@tagged('-at_install', 'post_install')
class Test404(HttpCase):
def get_assets_node_cache(self):
return [e for e in self.env.registry._Registry__cache.d if e[0] == 'ir.qweb' and '_generate_' in str(e[1])] #_generate_asset_nodes_cache
def test_rollback(self):
"""
In some rare case, attachments can be generatted, added to the cache but the transaction is rollbacked leading to 404
this actually tests the add_post_rollback behaviour
"""
self.env['ir.attachment'].search([('url', '=like', '/web/assets/%web.assets_frontend%')]).unlink()
self.env.registry._clear_cache()
self.url_open('/some_404_routes')
self.assertFalse(self.get_assets_node_cache(), 'orm_cache should be emptied by rollback')
self.assertFalse(self.env['ir.attachment'].search([('url', '=like', '/web/assets/%web.assets_frontend%%')]), "attachments should have been rollbacked")
self.url_open('/')
self.assertTrue(self.env['ir.attachment'].search([('url', '=like', '/web/assets/%web.assets_frontend%%')]), "attachments should have been generated as expected")
self.assertTrue(self.get_assets_node_cache(), 'orm_cache should remain populated')
self.env.registry._clear_cache()
class TestJavascriptAssetsBundle(FileTouchable):
@classmethod
def setUpClass(cls):
super().setUpClass()
# this is mainly to avoid tests breaking when executed after pre-generate
cls.maxDiff = 10000
cls.env['ir.attachment'].search([('url', '=like', '/web/assets/%test_assetsbundle%')]).unlink()
def setUp(self):
super(TestJavascriptAssetsBundle, self).setUp()
self.jsbundle_name = 'test_assetsbundle.bundle1'
self.cssbundle_name = 'test_assetsbundle.bundle2'
self.env['res.lang']._activate_lang('ar_SY')
def _get_asset(self, bundle, env=None, debug=''):
env = (env or self.env)
files, _ = env['ir.qweb']._get_asset_content(bundle)
return AssetsBundle(bundle, files, env=env, debug=debug)
def _get_asset(self, bundle, rtl=False, debug_assets=False):
files, _ = self.env['ir.qweb']._get_asset_content(bundle)
return AssetsBundle(bundle, files, env=self.env, debug_assets=debug_assets, rtl=rtl)
def _any_ira_for_bundle(self, extension, lang=None):
def _any_ira_for_bundle(self, extension, rtl=False):
""" Returns all ir.attachments associated to a bundle, regardless of the verion.
"""
user_direction = self.env['res.lang']._lang_get(lang or self.env.user.lang).direction
bundle = self.jsbundle_name if extension in ['js', 'min.js'] else self.cssbundle_name
rtl = 'rtl/' if extension in ['css', 'min.css'] and user_direction == 'rtl' else ''
rtl = 'rtl/' if rtl and extension in ['css', 'min.css'] else ''
url = f'/web/assets/%-%/{rtl}{bundle}.{extension}'
domain = [('url', '=like', url)]
return self.env['ir.attachment'].search(domain)
def _node_to_list(self, nodes):
res = []
for index, (tagName, t_attrs, content) in enumerate(nodes):
for name, value in t_attrs.items():
res.append(value)
return res
def test_01_generation(self):
""" Checks that a bundle creates an ir.attachment record when its `js` method is called
for the first time and this ir.attachment is different depending on `is_minified` param.
"""
self.bundle = self._get_asset(self.jsbundle_name, env=self.env)
self.bundle = self._get_asset(self.jsbundle_name)
# there shouldn't be any minified attachment associated to this bundle
self.assertEqual(len(self._any_ira_for_bundle('min.js')), 0,
@@ -226,13 +236,13 @@ class TestJavascriptAssetsBundle(FileTouchable):
def test_03_date_invalidation(self):
""" Checks that a bundle is invalidated when one of its assets' modification date is changed.
"""
bundle0 = self._get_asset(self.jsbundle_name, debug="assets")
bundle0 = self._get_asset(self.jsbundle_name, debug_assets=True)
bundle0.js()
last_modified0 = bundle0.get_checksum('js')
version0 = bundle0.get_version('js')
path = get_resource_path('test_assetsbundle', 'static', 'src', 'js', 'test_jsfile1.js')
bundle1 = self._get_asset(self.jsbundle_name, debug="assets")
bundle1 = self._get_asset(self.jsbundle_name, debug_assets=True)
with self._touch(path):
bundle1.js()
@@ -284,10 +294,9 @@ class TestJavascriptAssetsBundle(FileTouchable):
and create a minified ir.attachment.
"""
debug_bundle = self._get_asset(self.jsbundle_name)
nodes = debug_bundle.to_node()
content = self._node_to_list(nodes)
content = debug_bundle.get_links()
# there should be a minified file
self.assertEqual(content[3].count('test_assetsbundle.bundle1.min.js'), 1)
self.assertIn('test_assetsbundle.bundle1.min.js', content[0][0])
# there should be one minified assets created in normal mode
self.assertEqual(len(self._any_ira_for_bundle('min.js')), 1,
@@ -297,35 +306,14 @@ class TestJavascriptAssetsBundle(FileTouchable):
self.assertEqual(len(self._any_ira_for_bundle('js')), 0,
"there shouldn't be any non-minified assets created in normal mode")
def test_06_debug(self):
""" Checks that a bundle rendered in debug 1 mode outputs non-minified assets
and create an non-minified ir.attachment.
"""
debug_bundle = self._get_asset(self.jsbundle_name, debug='1')
nodes = debug_bundle.to_node()
content = self._node_to_list(nodes)
# there should be a minified file
self.assertEqual(content[3].count('test_assetsbundle.bundle1.min.js'), 1,
"there should be one minified assets created in debug mode")
# there should be one minified assets created in debug mode
self.assertEqual(len(self._any_ira_for_bundle('min.js')), 1,
"there should be one minified assets created in debug mode")
# there shouldn't be any non-minified assets created in debug mode
self.assertEqual(len(self._any_ira_for_bundle('js')), 0,
"there shouldn't be any non-minified assets created in debug mode")
def test_07_debug_assets(self):
""" Checks that a bundle rendered in debug assets mode outputs non-minified assets
and create an non-minified ir.attachment at the .
"""
debug_bundle = self._get_asset(self.jsbundle_name, debug='assets')
nodes = debug_bundle.to_node()
content = self._node_to_list(nodes)
# there should be a non-minified file (not .min.js)
self.assertEqual(content[3].count('test_assetsbundle.bundle1.js'), 1,
"there should be one non-minified assets created in debug assets mode")
debug_bundle = self._get_asset(self.jsbundle_name, debug_assets=True)
content = debug_bundle.get_links()
# there should be a minified file
self.assertIn('test_assetsbundle.bundle1.js', content[0][0], "there should be one non-minified assets created in debug assets mode")
# there shouldn't be any minified assets created in debug mode
self.assertEqual(len(self._any_ira_for_bundle('min.js')), 0,
@@ -398,11 +386,10 @@ class TestJavascriptAssetsBundle(FileTouchable):
def test_12_css_debug(self):
""" Check that a bundle in debug mode outputs non-minified assets.
"""
debug_bundle = self._get_asset(self.cssbundle_name, debug='assets')
nodes = debug_bundle.to_node()
content = self._node_to_list(nodes)
# find back one of the original asset file
self.assertIn('/web/assets/debug/test_assetsbundle.bundle2.css', content)
debug_bundle = self._get_asset(self.cssbundle_name, debug_assets=True)
content = debug_bundle.get_links()
# there should be a minified file
self.assertEqual(content[0][0], '/web/assets/debug/test_assetsbundle.bundle2.css')
# there should be one css asset created in debug mode
self.assertEqual(len(self._any_ira_for_bundle('css')), 1,
@@ -423,9 +410,8 @@ class TestJavascriptAssetsBundle(FileTouchable):
self.assertEqual(ira0.store_fname, ira1.store_fname)
# the ir.attachment records should be deduplicated in the bundle's content
nodes = bundle0.to_node()
content = self._node_to_list(nodes)
self.assertEqual(content[2].count('test_assetsbundle.bundle2.min.css'), 1)
content = bundle0.get_links()
self.assertIn('test_assetsbundle.bundle2.min.css', content[0][0])
# Language direction specific tests
@@ -433,17 +419,17 @@ class TestJavascriptAssetsBundle(FileTouchable):
""" Checks that a bundle creates an ir.attachment record when its `css` method is called
for the first time for language with different direction and separate bundle is created for rtl direction.
"""
self.bundle = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}))
self.bundle = self._get_asset(self.cssbundle_name, rtl=True)
# there shouldn't be any attachment associated to this bundle
self.assertEqual(len(self._any_ira_for_bundle('min.css', lang='ar_SY')), 0)
self.assertEqual(len(self._any_ira_for_bundle('min.css', rtl=True)), 0)
self.assertEqual(len(self.bundle.get_attachments('min.css')), 0)
# trigger the first generation and, thus, the first save in database
self.bundle.css()
# there should be one attachment associated to this bundle
self.assertEqual(len(self._any_ira_for_bundle('min.css', lang='ar_SY')), 1)
self.assertEqual(len(self._any_ira_for_bundle('min.css', rtl=True)), 1)
self.assertEqual(len(self.bundle.get_attachments('min.css')), 1)
def test_16_ltr_and_rtl_css_access(self):
@@ -473,23 +459,22 @@ class TestJavascriptAssetsBundle(FileTouchable):
self.assertEqual(ltr_version0, ltr_version1)
self.assertEqual(ltr_date0, ltr_date1)
# Assets access for ar_SY language
rtl_bundle0 = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}))
rtl_bundle0 = self._get_asset(self.cssbundle_name, rtl=True)
rtl_bundle0.css()
self.assertEqual(len(self._any_ira_for_bundle('min.css', lang='ar_SY')), 1)
self.assertEqual(len(self._any_ira_for_bundle('min.css', rtl=True)), 1)
rtl_version0 = rtl_bundle0.get_version('css')
rtl_ira0 = self._any_ira_for_bundle('min.css', lang='ar_SY')
rtl_ira0 = self._any_ira_for_bundle('min.css', rtl=True)
rtl_date0 = rtl_ira0.create_date
rtl_bundle1 = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}))
rtl_bundle1 = self._get_asset(self.cssbundle_name, rtl=True)
rtl_bundle1.css()
self.assertEqual(len(self._any_ira_for_bundle('min.css', lang='ar_SY')), 1)
self.assertEqual(len(self._any_ira_for_bundle('min.css', rtl=True)), 1)
rtl_version1 = rtl_bundle1.get_version('css')
rtl_ira1 = self._any_ira_for_bundle('min.css', lang='ar_SY')
rtl_ira1 = self._any_ira_for_bundle('min.css', rtl=True)
rtl_date1 = rtl_ira1.create_date
self.assertEqual(rtl_version0, rtl_version1)
@@ -507,14 +492,12 @@ class TestJavascriptAssetsBundle(FileTouchable):
def test_17_css_bundle_date_invalidation(self):
""" Checks that both css bundles are invalidated when one of its assets' modification date is changed
"""
# Assets access for en_US language
ltr_bundle0 = self._get_asset(self.cssbundle_name, debug='assets')
ltr_bundle0 = self._get_asset(self.cssbundle_name, debug_assets=True)
ltr_bundle0.css()
ltr_last_modified0 = ltr_bundle0.get_checksum('css')
ltr_version0 = ltr_bundle0.get_version('css')
# Assets access for ar_SY language
rtl_bundle0 = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}), debug='assets')
rtl_bundle0 = self._get_asset(self.cssbundle_name, rtl=True, debug_assets=True)
rtl_bundle0.css()
rtl_last_modified0 = rtl_bundle0.get_checksum('css')
rtl_version0 = rtl_bundle0.get_version('css')
@@ -522,7 +505,7 @@ class TestJavascriptAssetsBundle(FileTouchable):
# Touch test_cssfile1.css
# Note: No lang specific context given while calling _get_asset so it will load assets for en_US
path = get_resource_path('test_assetsbundle', 'static', 'src', 'css', 'test_cssfile1.css')
ltr_bundle1 = self._get_asset(self.cssbundle_name, debug='assets')
ltr_bundle1 = self._get_asset(self.cssbundle_name, debug_assets=True)
with self._touch(path):
ltr_bundle1.css()
@@ -532,12 +515,12 @@ class TestJavascriptAssetsBundle(FileTouchable):
self.assertNotEqual(ltr_last_modified0, ltr_last_modified1)
self.assertNotEqual(ltr_version0, ltr_version1)
rtl_bundle1 = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}), debug='assets')
rtl_bundle1 = self._get_asset(self.cssbundle_name, rtl=True, debug_assets=True)
rtl_bundle1.css()
rtl_last_modified1 = rtl_bundle1.get_checksum('css')
rtl_version1 = rtl_bundle1.get_version('css')
rtl_ira1 = self._any_ira_for_bundle('min.css', lang='ar_SY')
rtl_ira1 = self._any_ira_for_bundle('min.css', rtl=True)
self.assertNotEqual(rtl_last_modified0, rtl_last_modified1)
self.assertNotEqual(rtl_version0, rtl_version1)
@@ -560,7 +543,7 @@ class TestJavascriptAssetsBundle(FileTouchable):
ltr_files0 = ltr_bundle0.files
ltr_version0 = ltr_bundle0.get_version('css')
rtl_bundle0 = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}))
rtl_bundle0 = self._get_asset(self.cssbundle_name, rtl=True)
rtl_bundle0.css()
rtl_files0 = rtl_bundle0.files
rtl_version0 = rtl_bundle0.get_version('css')
@@ -585,11 +568,11 @@ class TestJavascriptAssetsBundle(FileTouchable):
self.assertNotEqual(ltr_files0, ltr_files1)
self.assertNotEqual(ltr_version0, ltr_version1)
rtl_bundle1 = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}))
rtl_bundle1 = self._get_asset(self.cssbundle_name, rtl=True)
rtl_bundle1.css()
rtl_files1 = rtl_bundle1.files
rtl_version1 = rtl_bundle1.get_version('css')
rtl_ira1 = self._any_ira_for_bundle('min.css', lang='ar_SY')
rtl_ira1 = self._any_ira_for_bundle('min.css', rtl=True)
self.assertNotEqual(rtl_files0, rtl_files1)
self.assertNotEqual(rtl_version0, rtl_version1)
@@ -606,12 +589,11 @@ class TestJavascriptAssetsBundle(FileTouchable):
def test_19_css_in_debug_assets(self):
""" Checks that a bundle rendered in debug mode(assets) with right to left language direction stores css files in assets bundle.
"""
debug_bundle = self._get_asset(self.cssbundle_name, env=self.env(context={'lang': 'ar_SY'}), debug='assets')
nodes = debug_bundle.to_node()
content = self._node_to_list(nodes)
debug_bundle = self._get_asset(self.cssbundle_name, rtl=True, debug_assets=True)
content = debug_bundle.get_links()
# there should be an css assets bundle in /debug/rtl if user's lang direction is rtl and debug=assets
self.assertIn('/web/assets/debug/rtl/{0}.css'.format(self.cssbundle_name), content,
self.assertEqual('/web/assets/debug/rtl/{0}.css'.format(self.cssbundle_name), content[0][0],
"there should be an css assets bundle in /debug/rtl if user's lang direction is rtl and debug=assets")
# there should be an css assets bundle created in /rtl if user's lang direction is rtl and debug=assets
@@ -626,29 +608,20 @@ class TestJavascriptAssetsBundle(FileTouchable):
attachments = self.env['ir.attachment'].search([('url', '=like', '/web/assets/%-%/test_assetsbundle.bundle4.%')])
self.assertEqual(len(attachments), 2)
asset_data_css = etree.HTML(html).xpath('//*[@data-asset-bundle]')[0]
asset_data_js = etree.HTML(html).xpath('//*[@data-asset-bundle]')[1]
format_data = {
"js": attachments[0].url,
"css": attachments[1].url,
"asset_bundle_css": asset_data_css.attrib.get('data-asset-bundle'),
"asset_version_css": asset_data_css.attrib.get('data-asset-version'),
"asset_bundle_js": asset_data_js.attrib.get('data-asset-bundle'),
"asset_version_js": asset_data_js.attrib.get('data-asset-version'),
}
self.assertEqual(html.strip(), ("""<!DOCTYPE html>
self.assertEqual(str(html.strip()), ("""<!DOCTYPE html>
<html>
<head>
<link type="text/css" rel="stylesheet" href="http://test.external.link/style1.css"/>
<link type="text/css" rel="stylesheet" href="http://test.external.link/style2.css"/>
<link type="text/css" rel="stylesheet" href="%(css)s" data-asset-bundle="%(asset_bundle_css)s" data-asset-version="%(asset_version_css)s"/>
<link type="text/css" rel="stylesheet" href="%(css)s"/>
<meta/>
<script type="text/javascript" src="http://test.external.link/javascript1.js"></script>
<script type="text/javascript" src="http://test.external.link/javascript2.js"></script>
<script type="text/javascript" src="%(js)s" data-asset-bundle="%(asset_bundle_js)s" data-asset-version="%(asset_version_js)s" onerror="__odooAssetError=1"></script>
<script type="text/javascript" src="%(js)s" onerror="__odooAssetError=1"></script>
</head>
<body>
</body>
@@ -659,27 +632,20 @@ class TestJavascriptAssetsBundle(FileTouchable):
attachments = self.env['ir.attachment'].search([('url', '=like', '%/test_assetsbundle.bundle4.js')])
self.assertEqual(len(attachments), 1)
asset_data_css = etree.HTML(html).xpath('//*[@data-asset-bundle]')[0]
asset_data_js = etree.HTML(html).xpath('//*[@data-asset-bundle]')[1]
format_data = {
"asset_bundle_css": asset_data_css.attrib.get('data-asset-bundle'),
"asset_version_css": asset_data_css.attrib.get('data-asset-version'),
"asset_bundle_js": asset_data_js.attrib.get('data-asset-bundle'),
"asset_version_js": asset_data_js.attrib.get('data-asset-version'),
"css": '/web/assets/debug/test_assetsbundle.bundle4.css',
"js": '/web/assets/debug/test_assetsbundle.bundle4.js',
}
self.assertEqual(html.strip(), ("""<!DOCTYPE html>
self.assertEqual(str(html.strip()), ("""<!DOCTYPE html>
<html>
<head>
<link type="text/css" rel="stylesheet" href="http://test.external.link/style1.css"/>
<link type="text/css" rel="stylesheet" href="http://test.external.link/style2.css"/>
<link type="text/css" rel="stylesheet" href="%(css)s" data-asset-bundle="%(asset_bundle_css)s" data-asset-version="%(asset_version_css)s"/>
<link type="text/css" rel="stylesheet" href="%(css)s"/>
<meta/>
<script type="text/javascript" src="http://test.external.link/javascript1.js"></script>
<script type="text/javascript" src="http://test.external.link/javascript2.js"></script>
<script type="text/javascript" src="%(js)s" data-asset-bundle="%(asset_bundle_js)s" data-asset-version="%(asset_version_js)s" onerror="__odooAssetError=1"></script>
<script type="text/javascript" src="%(js)s" onerror="__odooAssetError=1"></script>
</head>
<body>
</body>
@@ -777,11 +743,11 @@ class TestAssetsBundleWithIRAMock(FileTouchable):
def _get_asset(self):
with patch.object(type(self.env['ir.asset']), '_get_installed_addons_list', Mock(return_value=self.installed_modules)):
files, _ = self.env['ir.qweb']._get_asset_content(self.stylebundle_name)
return AssetsBundle(self.stylebundle_name, files, env=self.env, debug='assets')
return AssetsBundle(self.stylebundle_name, files, env=self.env, debug_assets=True)
def _bundle(self, asset, should_create, should_unlink, reason=''):
self.counter.clear()
asset.to_node()
asset.get_links()
if should_create:
self.assertEqual(self.counter['create'], 2, f'An attachment should have been created {reason}')
else:
@@ -1805,7 +1771,7 @@ class TestAssetsManifest(AddonManifestPatched):
'path': '/test_assetsbundle/%s' % path_to_dummy,
})
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec')
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec', {})
self.assertEqual(files, [('/test_assetsbundle/../../tests/dummy.xml', None, 'test_assetsbundle.irassetsec', None)])
# TODO, validate this behaviour
# the idea is that if the second element is False (not None) it will be added to the assetbundle, but considered in any case as an attachment url)
@@ -1822,7 +1788,7 @@ class TestAssetsManifest(AddonManifestPatched):
'bundle': 'test_assetsbundle.irassetsec',
'path': '%s' % path_to_dummy,
})
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec')
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec', {})
self.assertEqual(files, [('../../tests/dummy.xml', None, 'test_assetsbundle.irassetsec', None)])
def test_33(self):
@@ -1871,7 +1837,7 @@ class TestAssetsManifest(AddonManifestPatched):
'bundle': 'test_assetsbundle.irassetsec',
'path': '/test_assetsbundle/data/ir_asset.xml',
})
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec')
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec', {})
self.assertEqual(files, [('/test_assetsbundle/data/ir_asset.xml', None, 'test_assetsbundle.irassetsec', None)])
def test_36(self):
@@ -1880,7 +1846,7 @@ class TestAssetsManifest(AddonManifestPatched):
'bundle': 'test_assetsbundle.irassetsec',
'path': '/test_assetsbundle/static/accessible.xml',
})
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec')
files = self.env['ir.asset']._get_asset_paths('test_assetsbundle.irassetsec', {})
modified = files[0][3]
base_path = __file__.replace('/tests/test_assetsbundle.py', '')
@@ -1925,3 +1891,136 @@ class TestAssetsManifest(AddonManifestPatched):
.my_div.subdiv{color: blue;}
"""
)
@tagged('-at_install', 'post_install')
class AssetsNodeOrmCacheUsage(TransactionCase):
def cache_keys(self):
keys = self.env.registry._Registry__cache.d
asset_keys = [key for key in keys if key[0] == 'ir.asset' and '_get_asset_paths' in str(key[1])] # ignore topological sort entry
qweb_keys = [key for key in keys if key[0] == 'ir.qweb']
return asset_keys, qweb_keys
def test_assets_node_orm_cache_usage_debug(self):
self.env.registry._clear_cache()
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 0)
self.assertEqual(len(qweb_keys), 0)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', debug='tests')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', debug='1')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
# in debug=assets, the ormcache is not used for _generate_asset_links_cache
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', debug='assets')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
def test_assets_node_orm_cache_usage_file_type(self):
self.env.registry._clear_cache()
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 0)
self.assertEqual(len(qweb_keys), 0)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', js=True, css=False)
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', js=False, css=True)
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 2)
# NOTE: this result is not really desired but this is the current behaviour. In practice, we usually only generate one of them.
# This could be enforced or avoided
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', js=True, css=True)
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 3)
def test_assets_node_orm_cache_usage_lang(self):
self.env.registry._clear_cache()
self.env['res.lang']._activate_lang('ar_SY')
self.env['res.lang']._activate_lang('fr_FR')
self.env['res.lang']._activate_lang('en_US')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 0)
self.assertEqual(len(qweb_keys), 0)
self.env['ir.qweb'].with_context(lang='fr_FR')._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb'].with_context(lang='en_US')._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb'].with_context(lang='ar_SY')._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 2) # a second cache entry is created for rtl
def test_assets_node_orm_cache_usage_website(self):
if self.env['ir.module.module'].search([('name', '=', 'website'), ('state', '=', 'uninstalled')]):
return # only makes sence if website is installed
self.env.registry._clear_cache()
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 0)
self.assertEqual(len(qweb_keys), 0)
self.env['ir.qweb'].with_context(website_id=None)._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb'].with_context(website_id=1)._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 2) # the content may be different for different websites, even if it is not always the case
self.assertEqual(len(qweb_keys), 2)
def test_assets_node_orm_cache_usage_node_flags(self):
self.env.registry._clear_cache()
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 0)
self.assertEqual(len(qweb_keys), 0)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1)
self.assertEqual(len(qweb_keys), 1)
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', media='print')
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1, "media shouldn't create another entry")
self.assertEqual(len(qweb_keys), 1, "media shouldn't create another entry")
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', defer_load=True)
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1, "defer_load shouldn't create another entry")
self.assertEqual(len(qweb_keys), 1, "defer_load shouldn't create another entry")
self.env['ir.qweb']._get_asset_nodes('web.assets_backend', lazy_load=True)
asset_keys, qweb_keys = self.cache_keys()
self.assertEqual(len(asset_keys), 1, "lazy_load shouldn't create another entry")
self.assertEqual(len(qweb_keys), 1, "lazy_load shouldn't create another entry")
odoo/tests/common.py
+1
View File
@@ -653,6 +653,7 @@ class BaseCase(case.TestCase, metaclass=MetaCase):
when normalizing both archs. Takes either "xml" or "html"
:type parser: str
"""
self.maxDiff = 10000
if original:
original = _normalize_arch_for_assert(original, parser)
if expected: