This test can sometimes fail randomly
FAIL: TestProfiling.test_sync_recorder
Traceback (most recent call last):
File "/data/build/odoo/odoo/addons/base/tests/test_profiler.py", line 440, in test_sync_recorder
self.assertEqual(stacks_methods, [
AssertionError: Lists differ: [['a'[114 chars]], ['__exit__', '_remove'], ['__exit__'], ['__exit__', 'stop']] != [['a'[114 chars]], ['__exit__', 'stop']]
First differing element 11:
['__exit__', '_remove']
['__exit__', 'stop']
First list contains 2 additional elements.
First extra element 12:
['__exit__']
[['a'],
['a', 'b'],
['a'],
['a', 'c'],
['a', 'c', 'd'],
['a', 'c'],
['a', 'c', 'd'],
['a', 'c'],
['a'],
[],
['__exit__'],
- ['__exit__', '_remove'],
- ['__exit__'],
['__exit__', 'stop']]
Since we don't care about the last lines, just remove them from the
assertion.
closesodoo/odoo#163016
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The test test_tz_legacy will fail if the taget does not exist on the
operating system. This is breaking in some versions of the tz-data
package. Don't make this test fail if the target is missing.
closesodoo/odoo#161341
X-original-commit: 276eb0192fdddb736453857c18bf9f0cccecb4a3
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The main purpose of this test is to ensure that line return are removed,
but all other special character are kept once the file is saved.
Unfortunately, werkzeug versions have different strategies on how to
quote the filename, removing less special character in latest versions
like after https://github.com/pallets/werkzeug/commit/babfc93b3834bcbb22163442a9af70141bcc5a81
This also changes after the changes that removed werkzeug urls methods
replacing them by urllib, making the behaviour different again.
This commit makes the test more robust by checking that the filename
correspond to the expeted one once unquoted, not comparing the quoted
versions.
Part-of: odoo/odoo#160842
In ubuntu noble, some timezone where removed leading to errors when
trying to assign/access them.
This was partially fixed in the code by removing all references to old
timezones but one issue remains: if a database contains timezones that
are not defined in the os, the resolution will fail and break at runtime
This patches proposes to alter timezone to fallback on the new canonical
timezone if the timezone was removed.
This list was generated by checking all symlink in /usr/share/zoneinfo
in ubuntu 22.04 that disapeared in ubuntu 24.04
This solutions will work when moving a database from one server to
another, even without migration.
The all_timezone is not modified on purpose to avoid breaking existing
logic. This list may be used to define if a timezone is known by
postgress, define selection fiels, .... we don"t want to increase the
list in those case.
Some other logic using all_timezone may need to be updated but This
will be done in master.
Part-of: odoo/odoo#160842
Some of the non canononical timezones are not present in Ubuntu Noble,
it would be a better practice to only use canonical timezones in data
and tests.
Note that this is not a real fix for all cases since the database that
ran on Ubuntu Jammy and are moved to an ubuntu Noble server will have
the issue with timezones already in database.
One of the possible fix would be to manage that during upgrades, but
this isn't a verry flexible solution since upgrade are meant to manage
chyange of version, not change of server. If an old 17.0 versions needs
to be moved to a Noble server, this won't work.
Another solution would be to install package like tzdata-legacy that may
keep the old timezones but it is not the only think since TAI-10 are
also in this package. This solution is not ideal because non canonical
timezone will still be shown in the dropdown. We would need to filter
them.
A last solution would be to add the support for those old timezones by
monkeypatching the lib. This way, only new timezones would be shown but
non canonical one won't crash when used. This is not ideal either
because we may need to keep this for a while. But in combination with
the upgrade solution, it may work proprely.
Part-of: odoo/odoo#160842
In latest versions of werkzeug, the `werkzeug.urls` module has been
reduced to remove feature present in urllib.parse. This commit vendored
the old version to avoid breaking compatibility with older versions of
odoo on ubuntu Noble, without adapting the whole codebase. This
should/may be removed in stable by adaptaing eveything to urlib.
This version of the lib was minimalized to avoid redondunce with feature
still present in werkzeug.urls. Some unused features in odoo are still
vendored for ease but are not exposed on the werkzeug.urls for now.
Part-of: odoo/odoo#160842
Screencast are not always enable and a recent change broke the ffmpeg
call. This call was already broken in some ffmpeg versions.
This test will help to ensure this feature continues to work, and will
also test it in different ffmpeg versions during distro builds.
closesodoo/odoo#156738
X-original-commit: a3d6ef64e218683f7455f60d20860b62780be3b9
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Reduce the log size in case of failure of a screencast frames
Avoid a race condition while removing tree.
closesodoo/odoo#156341
X-original-commit: 97dc741929eb9497f9b44c110607254f0a873698
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This is breaking since 2024-01-01, disabling the test waiting for a
proper fix.
closesodoo/odoo#147846
X-original-commit: 56cdb8a4ad4a4650df2399305ed6210833b325b5
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
After the previous fix introducing a deepcopy, the pregenerate of the
assets became slower because of the many call to get_manifest (in loop,
recursively)
Since the manifest is immutable here and won't go outside of the call,
we can use the lower level version.
closesodoo/odoo#146602
X-original-commit: acb671f4d1b9c53c9e7d91414e847f26e8026ca3
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Before this commit the clean_assetbundle could unlink invalid
attachment, mostly when generating a no website assetbundle with
a different version than a website one, the website assetbundle will be
deleted.
Generating a new asset bundle attachment /web/assets/439-b4c80c3/1/web.assets_frontend.min.css (id:439)
Generating a new asset bundle attachment /web/assets/440-3723971/web.assets_frontend.min.css (id:440)
Deleting attachments [439] (matching /web/assets/%-%/web.assets_frontend.min.css) because it was replaced with /web/assets/%-3723971/%%%
The issue is that %-%/ will match 439-b4c80c3/1/ and not only
439-b4c80c3/
Note that it looks like this issue existed for a while but was invisible
because before 16.4 clean_attachment was invalidating the ormcache,
hiding the fact that a still valid asset was deleted and regenerated.
The proposed fix replaces the domain with %-_______/. The unique is
always 7 character long. Note that this change was already made in 17.0
when removing the id from the asset url so this doesn't need to be
completely forward-ported.
opw-3558552
closesodoo/odoo#145452
X-original-commit: 2ac466547e01bfd65415a53ae1efc9d45d9299fb
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The clean attachement was invalidation the cache before 16.4,
mainly because if an attachement is in cache of another worker and
deleted, this will cause a 404 when this worker serves a page needing
this attachment.
This was changed because an attachment should be unlink through
clean_attachments in two cases:
- the code source change on the server and a cold worker generates a
bundle
- an ir_asset was modified
In the first case, we consider that the server restarted (normally) and
all caches should be emptied.
In the second case, a specific invalidation is made.
But when something goes wrong, it is hard to debug, especially because
there is no information on when the new attachment was created, and the
previous one deleted.
This should solve the issue by helping to identify the cause of the
deletion.
closesodoo/odoo#143962
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
A custom script is modifying the output of
load_information_from_description_file to disable the
auto-install of modules during local testing. It was naively adapted for
v16.0 by replacing the corresponding methods. Since a lru cache was added
(nice optimization in most cases) this is an issue because running lint
test afterward will get the cached value with an incorrect autoinstall
value. It makes sens to avoid reading the file on the filesystem each
time, but making a deepcopy looks like an acceptable safeguard to avoid
hard to debug behaviors.
closesodoo/odoo#143628
X-original-commit: ad10ff4410ccf7c38f6154480484495bb57c53ff
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
If a .map is missing (outdated) the error message will report
'min' expected in extension in non debug mode
when the problem is actually that map are not generate through this
route.
If the attachment corresponding to a .map is not found, it
was most likely garbage collected.
Change the message to:
.map should have been generated through debug assets, (version
5eff983 most likely outdated)
closesodoo/odoo#141469
Signed-off-by: Jorge Pinna Puissant (jpp) <jpp@odoo.com>
Previously proposed formating was trying to normalize extra in one part
of the path. This means that no extra needed a placeholder.
The implementation was meant to be more generic and extendible since a
part of the logic has to be in website.
A suggestion was made to make it more restricted but explicite by
keeping the url simple in web/controllers/binary.py but adding a
controller in website to add this extra part.
The base extra direction is now in the extension, as the min part.
Initial urls:
/web/assets/{unique}/[{website_id}/][rtl/]{bundle_name}[.min].{extension}
New urls:
/web/assets/[{website_id}]/{unique}/{bundle_name}[.rtl][.min].{extension}
Managed by two routes:
/web/assets/<string:unique>/<string:filename>
/web/assets/<int:website_id>/<string:unique>/<string:filename>
Where filename is in the format {bundle_name}[.rtl][.min].{extension}
Multiple possibilities where proposed
- /web/assets/website/<int:website_id>/<string:unique>/<string:filename>
More explicit but prefixing by /website was considered
- /website/assets/<int:website_id>/<string:unique>/<string:filename>
This one is a litle painfull to match similar attachement, where
website is ignored.
- /website/<int:website_id>/assets/<string:unique>/<string:filename>
Almost accepted but subjective, and anyway two previous solution breaks
the cdn mecanism and would need a migration
- /web/assets/<int:website_id>/<string:unique>/<string:filename>
Almost accepted but subjective, and anyway two previous solution breaks
the cdn mecanism and would need a migration
This last solution was not ideal to match without unique
/web/assets/%/<string:filename> can match both
/web/assets/123456/<string:filename>
and
/web/assets/1/123456/<string:filename>
Anyway, matching without unique shouldn't be supported for al (even if
it is kind of supported with any right now) but it will work by changing
unique wildcard to a more specific one (_ * 7)
closesodoo/odoo#131353
Related: odoo/enterprise#47313
Related: odoo/design-themes#730
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The generation inside the rendering has some drawbacks:
- `commit_assetsbundle` is needed for reports rendering because the
template rendering may generate some assets that will be accessed by
another transaction before the transaction is committed. But this
solution is not ideal since the transaction is committed in the middle
of the request
- when the first rendered page is a 404, the assets are not committed
and the page is broken.
- when starting, deleting an attachment can create a concurrent update
error and the request is retried. This will occur once per attachment
and for all worker trying to access the same resource. The whole
transaction is rollbacked, even the previously created assets bundle.
- The cold page load is a slower since there is more work to do.
- Implementing a readonly request is difficult because it could be
transformed to read write and re-executed if the assets bundle does not
exist.
Generating assets when needed solves those issues. The concurrency
when deleting an assets could still occur but only once per bundle, and
in a smaller transaction. This could be solved with a lock now that we
have more control on the transaction. The commit_assetsbundle can be
removed and 404 page should have a correct layout. The cold page load
could be a little faster because the assets bundle can be generated in
parallel requests instead of sequentially when rendering the page.
Part-of: odoo/odoo#131353
This is a small improvement to reduce the dependency between attachment
and link generation. This will help to move the generation in the
web assets route in next commit.
Part-of: odoo/odoo#131353
The current solution returns two node, one for the css itself, another
for the script managing the error. This means that we must know if we
have a css error when generating the assets links, when rendering a
t-call-assets.
The proposed solution will solve this issue by managing the error
in the css file itself. A slight change will also save the error in the
attachment instead of choosing the previous one, meaning that the
error will be saved and the bundle won't be recomputed anymore.
This solution also removes the only case when a t-call-assets can return
content instead of a link, this will be cleaned in the next commit.
As a slight change, a warning will alway be displayed on the bottom of
the page, to manage the case where the javascript managing the error is
not in the page.
Part-of: odoo/odoo#131353
The main motivation is to be able to generate assets bundle outside
the t-call-assets call.
The need of an id in the url makes it mandatory to have an attachment
when adding the url in the page. Without this restriction, we can guess
the url without generating the assets.
This can also have other useful side effect:
There are corner case when a worked could have an invalid url in
cache because, if the transaction is rollbacked or if another request
generates the same attachment at the same time. This should be
partially solved by removing the id: The url remains valid even if the
attachment does not exist.
Note that the extra part of the url was made explicit, always there and
taking one / to remove complexity and ambiguity.
Note that an additional query appeared in .test_50_perf_sql_web_assets
because of the search, this but two of them were in _find_record. One of
them was an `exist`, not making much sense since we are not getting the
id from the attachment url anymore but from a search, and the other one
was prefetch of the "public field" since the call to _find_record does
not go in other cases (xmlid, website published, access token, ....). A
attachment of a asset is always public, and this part of the security
was moved to the search domain. The final result is one less query:
- one query to search
- one query to read the fields (_get_stream_from) (the prefetch could
actually be set to avoid prefetching everything)
Part-of: odoo/odoo#131353
The current used version of wkhtmltopdf manages is frozen to 0.12.5
because some features used by odoo are only available in this version
using a patched qt.
Anyway, it become difficult to keep this version up to date, especially
with Ubuntu Jammy and updates of the dependencies used by wkhtmltopdf.
More than that because of qt related issues, wkhtmltopdf maintenance
will end soon.
Adding a test to check some caracteristics of pdf reports may be usefull
to check that current version is still working, and eventually to find
an alternative solution.
This test checks that pdf reports contains the expected headers and
footers elements with the correct page number (2 record of 2 pages)
This also check that the pdf is in the expected A4 format.
Future test may check other formats, margins, layout,
page size/pagebreak combination, (table of content?), ...
This test may also check performances and existing limitations.
closesodoo/odoo#136626
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
The issue with the t-cache (discribeds with a test in previous commit)
is that the key/value can be anything. It can be based on other t-cache
(assets in the example) but could be anything else.
The proposed solution will clear the t-cache when ANY other cache is
cleared. This is similar to the behaviour when the t-cache was
introduced (single ormcache)
Also sligly improve logging to precise what was cleared
closesodoo/odoo#138647
X-original-commit: aa69fa9f8efed0da45c601e5d57ce252446e376f
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The issue that can happen is a t-cache covering a t-call-asset.
The asset_cache is invalidated but not the t-cache, meaning that the url
pointing to the asset will lead to a 404 once another request, without
t-cache, will regenerate the asset.
X-original-commit: fffc198914b8c885905e9dddd72b6490fcc4c120
Part-of: odoo/odoo#138647
Using freezetime to decorate a TestCase can have strange effects.
It looks like an override but monkeypatching the
setUpClass/tearDownClass. It started breaking between python 3.10.6 and
3.10.12. See pr message for more details.
closesodoo/odoo#137215
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When calling track_prepare, an important part of the logic is getting
description in fields_get. We actually don't need the description,
fields_get is mainly use here to check for groups, but the dictionnary
is immediately transformed to a set making values irrelevant.
The same optimisation is done in _message_track, usefull to avoid an
additionnal query in test_recurring_order_creation_perf, because of a
value not in cache.
closesodoo/odoo#134689
X-original-commit: a7e7f90531a21af99a4c95abb829f8f6de94f68a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Since 16.4, the number of file made pylint reach the default memory
limit from time to time. This commit will remove the limit for this test
as it was done for chrome. The next step would be to split the test
per set of module or maybe analyze the memory consuption of some custom
check.
closesodoo/odoo#131336
X-original-commit: 8acb8d9a8bf5e19b53528275a4ddd5d4a289472b
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Since #121376 a clear_cache was removed when unlinking an attachment
This clear_cache was not useful when restarting a server with new
sources, an other operation changing the content of an assets should
invalidate the cache manually. This is the case of ir.asset CUD
operations.
Unfortunately, a manual update was left missing in website, when
archiving ir.assets used for snippets.
This was discovered on runbot, with two workers, when the first workers
generates assets before the cron, and the second one after the cron.
The second worker unlinks attachments creating an inconsistency in the
cache of the first one.
This problem can be solved quickly by invalidating the assets cache in
the cron manually but this will be done in all cases. The proposed
solution will check the ir.asset that should change and only clear the
cache if the state changed.
Regarding performances, this should actually be a slight improvement in
query count since at the cost of one more select to prefetch the record
we can avoid multiple update, one per snippet. In most case no update at
all should be done, at most 2 can be done (one for archive, one for
unarchive).
Example with some assets to unarchive:
Before:
TOTAL ENTRIES: 277
SELECT: 158 (~0.16591858863830566s)
UNKWOW: 54 (~0.0202481746673584s)
UPDATE: 65 (~0.03202557563781738s)
After:
TOTAL ENTRIES: 214
SELECT: 159 (~0.1663439826965332s)
UNKWOW: 54 (~0.029229164123535156s)
UPDATE: 1 (~0.0002865791320800781s)
If the number of query is lowered, the python processing is slightly
higher. Locally the test time is similar, slightly longer since an
additional call to _disable_unused_snippets_assets was added to check
the cache invalidation
closesodoo/odoo#130973
X-original-commit: 3aeab51ff4379e6d76172614fd39f09d6c449ac6
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Some test may perform a request without being in test_mode
In this case, a check signaling could be called and since the sequence
was incremented the registry may be reloaded.
This is a simple fix to avoid this issue waiting for a stronger check.
closesodoo/odoo#129814
X-original-commit: 41bb17b3046e8f1605d6f893ca1807d77ed21173
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
During the nightly, it looks like some test in
`TestAccountEarlyPaymentDiscount` will reload the registry, restarting
all base tests, including `.test_add_field_valid`
This test cannot be executed on an existing database without providing
a `-i` for a strange reason.
If this is not a real issue, this simple assertion will avoid to get
stuck while running this test on an existing database.
closesodoo/odoo#129491
X-original-commit: 4c6e05cbb43d08c544d9d558f80b67ccb786d482
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
A clear_cache was added in _update_xmlids in pr 119813.
If it was needed in case of update, it is not useful when creating an
xmlid or when the value is unchanged.
The initial idea was to detect if an update or an insert was done, maybe
using create_date and write_date. Unfortunately the create_date is the
same as the write_date in the same transaction. This is unlikely but in
this case, we could update the cache in place.
The final behavior is to update the cache in all case, and notify other
workers only if a model was updated.
This will help to avoid invalidating the cache too mush during module
loading. Even if the impact on time is small, the increase in queries
was visible.
This solution would even be a slight improvement on previous query count
closesodoo/odoo#129029
Related: odoo/enterprise#44349
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
The _xmlid_lookup cache contains the id of the ir.model.data with is
unused, making cache pupulation and usage more complex than needed.
Removing this id in prevision of the next commit
Part-of: odoo/odoo#129029
The number of invalidation during the test is quite large and
the crons cal also spam this log sometimes.
Waiting for further cleanup, removing this log to avoid spaming runbot
logs and making them harder to read.
closesodoo/odoo#128921
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Splitting the cached revealed a missing cache invaldation.
The cache invalidation added a a query in test_related_fields
The query can be avoid by not updating the xmlid if it is not useful.
closesodoo/odoo#119813
Related: odoo/enterprise#42527
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The cron hr_presence will write hr_presence_last_compute_date on company
leading to an invalidation of the cache.
This commit makes the invalidation based on a list of field.
This may break some non tested behaviour, will be a good opportunity to
add test if any bugfixes is linked to this change.
Part-of: odoo/odoo#119813
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.
Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.
The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.
Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.
The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.
We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.
Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.
Note that this poc is inspired from the long term cache but not all
use case where applie yet.
Part-of: odoo/odoo#119813
A cache invalidation was triggered when displaying the settings page
because of a write on ir.property
return call_kw(request.env[model], method, args, kwargs)
File "/home/xdo/osrc/master/odoo/odoo/api.py", line 461, in call_kw
result = _call_kw_multi(method, model, args, kwargs)
File "/home/xdo/osrc/master/odoo/odoo/api.py", line 448, in _call_kw_multi
result = method(recs, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/models.py", line 6689, in onchange
record._onchange_eval(name, field_onchange[name], result)
File "/home/xdo/osrc/master/odoo/odoo/models.py", line 6403, in _onchange_eval
res = method(self)
File "/home/xdo/osrc/master/enterprise/sale_stock_renting/models/res_config_settings.py", line 17, in _onchange_padding_time
self.env['ir.property']._set_default("preparation_time", "product.template", self.padding_time)
File "/home/xdo/osrc/master/odoo/odoo/addons/base/models/ir_property.py", line 203, in _set_default
prop.write({'value': value})
This commit will avoid this by checking the current value.
Part-of: odoo/odoo#119813
This is a proposal to try to find a similar asset before generating one.
If get_attachments fails, the next step will be to generate the
attachments from scratch, a slow operations.
When creating a new website, all assetsbundle would actually be
similar to their version without website, but the url is different.
This can be visible because the first loading of /web is slow after
creating a new website: the website_id is forced in the session
and the assets_backend are regnerated, identical to the original ones.
This commit proposes to try to find an attachments with differents extra
but the same uniquifier when possible and copy it's content.
Note that just returning the other attachement url may work, but it
would be confusing to randomly have links to assets comming from another
website_id. This would also be a problem if the original attachment is
unlinked, forcing to recompute it for other websites.
Good to know, since the content is the same, no duplication of the
content should appear in the filestore, just an entry in the database.
Part-of: odoo/odoo#121376
1. move cache to _get_asset_paths
The `_get_asset_content` cache has many cache key that are related to a
posprocessing of the `_get_asset_paths` result, the heavy part of this
method. Moving the cache to _get_asset_content will have the benefit
to create less duplicates entries in the ormcache as well as less cache
miss.
To simplify even further, the css and js parameters are removed since
they only filter the output of get_paths, the heavy part of globing the
file will be done before that. Anyway, they are both true when called
from _get_asset_content, and the only other call, in
`_get_related_bundle` don't really need to filter them since it is not
a critical part regarding performance, and the funtional result will
stay the same.
The initial orm cache key was using `_get_template_cache_keys`, a little
overkill and possibly creating duplicates entries again. The only
context key needed is website_id for `_get_related_assets`.
Note that it is not really enough, the orm cache key should actually
contain `request.session.get('force_website_id')` as well has
`request.httprequest.host`. This will be addressed latter since a nicer
solution would be to have website_id as a unique parameter computed
earlier.
2. better _get_asset_paths cache key
The orm cache key was simplified in previous point but there is still
one concern, the website_id depends on more parameters than that:
- request.session.get('force_website_id')
- request.httprequest.host
- existing websites
The idea here is to call `get_current_website` instead of using all
parameters that could define the webiste.
In the same spirit of `_get_template_cache_keys` `_assets_path_params`
can be overriden to give extra params that are usefull to list assets
path. Those params are computed before entering the method
`_get_asset_paths`. This may latter put at a higher level latter, in
get_asset_node, to simplify the _generate_asset_nodes_cache key.
3. better assets_node caches key
The main purpose of this part is to improve ormcache containing assets
nodes. The ormcache key contains
- to much context key
- missing session/host/env info
- unwanted boolean options.
- keys leading to the same cache value
The main goal being to reduce the size of the cache keys, decrease the
number of cache entries and improve the cache hit.
This will also make the behaviour more coherent and hopefully less bug
prone because of mismatch in parameters.
The main reason of the orm cache is the slowness of the validation of
the assets. This includes:
- listing files (dedicated orm cache)
- computing version
The cache key was depending on
- `debug`
The only relevant value for debug is "contains assets"
We dont need to differ between debug='', debug='1', debug='test',
and 'debug=assets', 'debug=tests,assets', ...
- `defer_load`, `lazy_load`, `media`
Those values are only useful to generate html node, a leightweight
operations that does not really needs to be in cache. `media` was also
used in the generation but it looks useless if we have the media on the
node. THIS NEEDS TO BE VALIDATED but in any case, since media is not
used to generate the url, it doesn't make sence to use it in the
generation.
The main idea to remove them from the ormcache key is simply to generate
the nodes outide the ormcached values.
-`async_load`
This one is similar to `defer_load` and `lazy_load` but it looks like
it wasn't used anymore. This was simply removed
- context.get('lang')
The only information needed is the direction, rtl or ltr. This means
en and fr languages, despite sharing the same css assets, will duplicate
the ormcache entries.
-`_get_template_cache_keys`
Only the lang and webiste where really relevant in this flow. Other
keys are actually useless in this flow.
Some information used in the generation where not in the orm cache key
- `self.env.user.lang` if there is no lang in the context
- `request.session.get('force_website_id')`
- `request.httprequest.host`
- ...
The proposed solution is to:
- extract any informùation needed from thecontext, request, environment
before entering the ormcache, reduce it to the minimal possible set of
values needed
```
rtl = self.env['res.lang']._lang_get_direction(self.env.context.get('lang') or self.env.user.lang) == 'rtl'
assets_params = self.env['ir.asset']._get_assets_params() # website_id
debug_assets = debug and 'assets' in debug
```
and remove a leightweight part of the logic
```
def _get_asset_nodes(self, bundle, css=True, js=True, debug=False, defer_load=False, lazy_load=False, media=None):
links = self._get_asset_links(bundle, css=css, js=js, debug=debug)
return self._links_to_nodes(links, defer_load=defer_load, lazy_load=lazy_load, media=media)
```
Where _get_asset_links is the cached part, and _links_to_nodes is the
lightweight part generating the nodes based on the `defer_load`, ....
Additionnal notes:
- data-asset-version and data-asset-bundle are removed from the node
since they don't seem to be used anymore since 65d70acdbf
- async_load is removed since there is no occurence of this in the code.
- a small hack is still needed to pass javascript content instead of
links, this is only to manage css compile error and will hopefully be
removed in the future.
- a context key is still in use to generate the bundle, the
`commit_assetsbundle` but it has no impact on content and will hopefully
be removed in the future.
4. Add test for ormcache hit/miss
In this context, hit/miss is about having the same cache key for the
same result. This test demonstrates the current state, were entries are
create in the ormcache only if the key is really different and will lead
to a different result.
5. remove cache invalidation
This cache invalidation is quite agressive since everytime an
assetbundle is updated, all workers will clear their cache.
The concerned cache by this clear_cache is `_generate_asset_nodes_cache`
throug `_get_asset_nodes`.
The cache is ignored, both in dev=xml and debug=assets.
This clear cache was made conditionnal in 553ea82f81 but this does
not solve an issue we can have in production.
Lets imagine a clean solution
- all sources are updated
- all workers are restarted.
The orm caches are all empty, but since the sources
changed, all bundles will be recomputed. This means that every bundle
updated in database with save_attachement will invalidate the cache of
all workers. Rendering a pdf report of any kind using a specific bundle
will invalidate all cache. Starting a debug=assets for the first time
will invalidate all cache, even if the cache is not used in this case.
But for a regenerated bundle we would expect the ormcache to be:
- empty (did not generate the same bundle yet)
- have the same value (concurrent generation of the same bundle)
Having a different value would mean that the bundle was generated with
another version of the sources. In this case it is maybe even better not
to invalidate the cache since it could lead to an invalidation war
between two workers.
The only case where invalidating this cache is useful is when a bundle
changes, Usually if an ir_asset is created, modified, ...
There is still another rare but possible possibility to have a 404 if
the transaction is rollbacked after populating the assets node cache.
In this case, we only need to clear the cache locally in case of
rollback.
Part-of: odoo/odoo#121376
This error was intoduced in #121159
The javascript unique should also be based on templates.
closesodoo/odoo#121842
X-original-commit: 2a7c6640357b4d6fc0c539b5d8d5e6dd80882d5b
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The generation of the assets may make this step fail randomly.
The pregenerate is not enough because this is generating assets for
website2.
This issue will be partially solved in master with this PR [1] by not
generating an asset if a close one is found in the database.
For now just increase the timeout for this test.
[1]: https://github.com/odoo/odoo/pull/121376closesodoo/odoo#121777
X-original-commit: c3209aec9741cda8597525add88b0a932fb7532a
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
The Weakset Used for envs can lead to unpredictible behaviour when
calling flush on the transaction because we iterate on the `envs`.
Since WeakSet.data is a set, the iteration will depends on the python
hash seed.
This commit proposes to replace Transaction.envs.data by an OrderedSet
to solve this issue.
Some test demonstrates that it does not affect the garbage collection
and that the order is now deterministic.
The question to now if we should reverse the envs to find the most
suitable envs remains.
closesodoo/odoo#121604
Signed-off-by: Raphael Collet <rco@odoo.com>
One of the most costly part of a page loading when the ormcache is cold
is computing the assets node, the unique identifier of an attachment to
validate whether the existing attachment is still valid with the current
version of the static files.
This operation needs to glob assets path in the filesystem,
get the modification date, check attachments, ...
Right now this task is not really optimized and can take some time
because of an excessive number of glob on the filesystem, unnecessary
exists to define absolute path, double computation of file list and
modified times when getting js and css bundle separately, ...
A list of modifications mainly discussed in the pr message are made
with this commit to speedup things.
- split css and js unique
- prepare api for an in memory glob
- change api to propagate absolute path and meta information through
`ir.asset._get_paths`-> _get_asset_paths -> `_get_asset_content` ->
`AssetsBundle`
closesodoo/odoo#121159
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When inside an HttpCase, the end of a successful request will
`signal_changes` meaning that the registry_invalidated flag is removed.
A second issue is that this flag is thread local meaning that if a
request set the flag, it won't be visible from the test thread.
For those reasons, this commit ensures the registry sequences are
incremented as in production mode, and adds a check that the sequence
didn't change during the tests, calling `setup_models` the registry
manually if needed.
closesodoo/odoo#121268
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
For a strange reason, some bug where revealed in this tour with this
pull request.
closesodoo/odoo#121103
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When generating the routing map, `Rule.add` will call
`Rule._compile_builder` twice, representing almost one third of the
routing map generation time. `_compile_builder` is actually stored
in `Rule._build` and `Rule._build_unknown` for latter use, in the
`Rule.build` method.
The `Rule.build` method is used to transform a rule, lets say
`/forum/<model("forum.forum"):forum>` in `/forum/basics-of-gardening-2`
Even if a deeper investigation could be interresting, it looks like it
is only used for `is_frontend_multilang` routes and `_enumerate_pages`,
used in the `sitemap` and `search_pages`.
The proposed solution si to make this part lazy, in order to call
_compile_builder on demand, once per rule. This will speedup the initial
routing map generation and postpone the heavy work when we need it,
only for the part we need most of the time.
On a database with all modules installed, generation of the routing map:
Before: ~600 ms
After: ~200 ms
An alternative implementation was also overriding the Rule.build method
instead of having a callable LazyCompiledBuilder, this implementation
was choosen since it is less dependant off the werkzeug implementation.
closesodoo/odoo#120542
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
`re` is using an internal cache for regexes with a default size of 512
This is correct for small apps running on small infrastructure
but as a simple example, the routing map will compile ~1100 regex in a
row. This means that generating two routing map in a row don't benefit
from this cache at all. It looks reasonnable to increase this limit
globally for odoo.
This is visible when generating two routing map in a row.
After this change:
Generating routing map for key None
Routing map web generated in 0.194s
Generating routing map for key 1
Routing map website1 generated in 0.196s
After this change:
Generating routing map for key None
Routing map web generated in 0.200s
Generating routing map for key 1
Routing map website1 generated in 0.062s
Part-of: odoo/odoo#120542
Before this commit an ir_assets generated automaticaly by the web editor
will generate an url ending with ...custom.addon.bundle_name.ext
After this commit the url will start with /_custom/addon.bundle_name/...
This will make it easier to spot at immediately if it is a custom asset
and thus it is useless to apply the glob. Actually, it will fail on the
/_custom when trying to glob, making it faster.
This is mainly useful to clarify and debug but in a case with mainly
customised ir_asset for one bundle, it may have an impact on speed.
An upgrade script was created for this change.
closesodoo/odoo#120699
Related: odoo/upgrade#4636
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Some glob are more complex than they should.
Since `**` matches no/one or many directories, the use of /**/**/ is
the same as /**/.
Since **.js is similar to *.js except if you except to have a js file in
a directory ending with js. Moreover ** will match any directory
recursively before a js file.
closesodoo/odoo#120880
Related: odoo/enterprise#40842
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Add a test_logs_assets_check_time test to make stats on the time needed
to validate all asset bundle (will be added to runbot stats)
Add a test_logs_pregenerate_time, mainly to profile this part during
devlopment. This test is -standard and will only run if requested.
closesodoo/odoo#120767
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This test will help making stats on routing_map generation performances
This will help to mesure the time for the main `None` routing map as
well as for website1, the idea being that it would be possible to
mutualize a part of this computation between routing maps.
closesodoo/odoo#120591
X-original-commit: 6646fa7e2ef64e5f060bf38b0d85bf1188d79e4f
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When instantiating a new registry, this piece of code is called
try:
odoo.modules.load_modules(registry, force_demo, status, update_module)
except Exception:
odoo.modules.reset_modules_state(db_name)
raise
For a new database, load_modules will create the table ir_module_module
in the same transaction as everything else. This means that if any error
occurs, the transaction is rollbacked and the table ir_module may not
exist.
`reset_modules_state` will try to access ir_module_module table leading
to another error and an unecessary and confusing error log.
2023-04-26 12:05:34,823 616958 ERROR ? odoo.sql_db: bad query: UPDATE ir_module_module SET state='installed' WHERE state IN ('to remove', 'to upgrade')
ERROR: relation "ir_module_module" does not exist
LINE 1: UPDATE ir_module_module SET state='installed' WHERE state IN...
^
2023-04-26 12:05:34,823 616958 ERROR ? odoo.modules.registry: Failed to load registry
2023-04-26 12:05:34,824 616958 CRITICAL ? odoo.service.server: Failed to initialize database `test-base`.
Traceback (most recent call last):
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 90, in new
odoo.modules.load_modules(registry, force_demo, status, update_module)
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 386, in load_modules
raise Exception('An error')
Exception: An error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/xdo/osrc/master/odoo/odoo/service/server.py", line 1302, in preload_registries
registry = Registry.new(dbname, update_module=update_module)
File "<decorator-gen-14>", line 2, in new
File "/home/xdo/osrc/master/odoo/odoo/tools/func.py", line 87, in locked
return func(inst, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 92, in new
odoo.modules.reset_modules_state(db_name)
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 622, in reset_modules_state
cr.execute(
File "/home/xdo/osrc/master/odoo/odoo/sql_db.py", line 311, in execute
res = self._obj.execute(query, params)
psycopg2.errors.UndefinedTable: relation "ir_module_module" does not exist
LINE 1: UPDATE ir_module_module SET state='installed' WHERE state IN...
With this commit, we check the ir_module_module table existance avoiding
an exception and revealing the minimal traceback.
2023-04-26 12:11:21,218 617810 INFO ? odoo.modules.loading: skipping reset_modules_state, ir_module_module table does not exists
2023-04-26 12:11:21,218 617810 ERROR ? odoo.modules.registry: Failed to load registry
2023-04-26 12:11:21,218 617810 CRITICAL ? odoo.service.server: Failed to initialize database `test-base`.
Traceback (most recent call last):
File "/home/xdo/osrc/master/odoo/odoo/service/server.py", line 1302, in preload_registries
registry = Registry.new(dbname, update_module=update_module)
File "<decorator-gen-14>", line 2, in new
File "/home/xdo/osrc/master/odoo/odoo/tools/func.py", line 87, in locked
return func(inst, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 90, in new
odoo.modules.load_modules(registry, force_demo, status, update_module)
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 386, in load_modules
raise Exception('An error')
closesodoo/odoo#119820
Exception: An error
Signed-off-by: Raphael Collet <rco@odoo.com>
Making a test post_install using @tagged should always remove the
at_install tag.
The main reason for that is that runbot split config select if an
at_install or post_install tests should be executed is using negation:
`--test-tags -post_install`. The reason for that is that giving a positive tag will
replace the "standard" tag and non standard tag could be executed if
giving `--test-tags at_install` (without negation)
Since runbot tests in parallel builds, one of them using
`--test-tags -post_install` and the other `--test-tags -at_install`,
a test that is both post install and at install wont be executed at all.
Also, a tests with both tags will be executed twice
in a normal flow, usually not intended.
The correct way to make a test post_install is to use
@tagged('post_install', '-at_install')
closesodoo/odoo#118969
X-original-commit: d1db306b212d4abb5b2faab9e56c8e83b85c53b9
Related: odoo/enterprise#39966
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The current profiler will add default value in the user session
using disk space without valid reason.
This commit makes those parameters optional in the session.
closesodoo/odoo#118762
X-original-commit: eb3bf03b119105f56806d7460fccd4c0133deaab
Signed-off-by: Jérémy Kersten <jke@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
is_transpiled is only needed when generating a asset bundle while
JavascriptAsset can be generated to compute the version hash.
is_transpiled needs to read the content to be defined which is quite
slow. Transforming is_transpiled into a lazy property will speedup the
cold loading of generate_assets_node, especially when attachment already
exists.
Locally:
- /web with all modules in debug=assets goes from ~350 to ~150 ms
- generate_assets_nodes part goes from ~230 to ~55 ms
closesodoo/odoo#116123
X-original-commit: 47b44e1ae61583a881bb2ede06cc02f4b9423cbc
Signed-off-by: Simon Genin (ges@odoo) <ges@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The test-file is used by some dev to run all test classes from a file.
But test-file is always post install and doesn't always have the same
behaviour of a normal test execution.
This commits modifies the module test tags behaviour to be able to
give a file.
closesodoo/odoo#113850
X-original-commit: 5aff8cf22ab0cbac7a9a3cebf39860f55617f79e
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Odoo Test environments requires to modify many parts of the unittest
TestCase, Suite and Result.
The main initial reason is to **avoid to postpone result at the end of
the test suite**, because even if it is convenient to have all errors
visible after the tests in some case, odoo logs adds information during
the execution that can be useful to debug when a test fail, to have
context for an error. (see **OdooTestResult**)
We are also fixing the stack trace comming from a unittest and since
there is no proper way to hook inside the TestPartExecutor, a dirty hack
injects anoter result on the outcome to manage the error and complete
the stack trace. This was also a way to avoid to postpone subtest logs
at the end of the test case (see _ErrorCatcher)
`_feedErrorsToResult` was used to test the test suite behavior since
there are many customization and this is quite fragile, especially if
unittest changes behavior in other python version.
**Python 3.11** introduced python/cpython#664448d8 That, in a way, goes
in the same direction of the changed introduced with _ErrorCatcher:
immediately feed errors to resut instead of postponing it. But this also
removes `_feedErrorsToResult` that was used to test this behaviors, as
well as other ones.
Since odoo should remain multi-version, this amount of changes on the
initial behavior become to complicate to keep cross-version and the
(already in our mind for a while) solution to **vendor unittest** will
help to simplify most of our test code base.
This commit modified the vendored unittest files to simplify them as
much as possible to suite our needs.
Since the runner is still the unittest one, we need to inherit from
unittest.Testcase in order to have the right type.
This also means that we still have access to all TestCase methods
without overriding them all. This is convenient for assertion methods as
an example but the initial idea is to vendor our own version of TestCase
to avoid having trouble to adapte our miscommunications to future python
versions. A trade-off must be done to chose what should remain in our
code base. The idea is to keep logic closely linked to our changes in
our code base, mainly around the run method, but also addClassCleanup
wich need to be vendored for python 3.7, but assertions methods are
independent. Any logic can be moved fom unittest to our
vendored version in the future if needed.
X-original-commit: 9a5d1ea54be49e4cc8208c33e76a6bbd2414d5d0
Part-of: odoo/odoo#113850
Vendor some unitest file before modifying them in next commit
Chosen files are suite, case, and result since they are working together
and are the most modified classes in odoo.
mock, signals, runner and utils will still be imported from unittest.
X-original-commit: 742d165b9a1bff23deb736dee6b266c76f3ce727
Part-of: odoo/odoo#113850
A set iteration was creating a random number of queries on some tests.
This was noticed in TestEventPerformance were a random additionnal query
could appear in default_get depending on _get_description and
_get_default_stage_id order.
This commit uses a list to get a deterministic order. It looks like
the set was not useful anyway.
closesodoo/odoo#113563
X-original-commit: 13572df256ccc6351588851d385c381ace155e0d
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Warn when constrains name combine with table name will be more than
63 characters.
This is to avoid case like the one fixed in #103148
Unlike index, since constrains name are defined in the code, we prefer
to avoid automatic truncate and add a warning since devs can chose
an appropriate short-enough name.
The linked fixes will just truncate the name to the max length
to match the name in existing databases. Renaming could be done in other
pull requests with upgrade scripts to avoid constrains re-computation.
Part-of: odoo/odoo#109065
Breaking on 2022->2033 transition
Those tests should be adapted to work without hardcoded dates.
closesodoo/odoo#108884
X-original-commit: 6945966b49a58ce3c0976dce37e324b37af8367a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
With the new year transition, some demo data relative to the current
time are now inconsistent with each others:
- Some payslips starts 2 mount before now.
- the contract starts this year, first January.
This means that
- the contract starts 2023-01-01
- the payslips ends 2022-12-01
Leading to an error
"The following employees have a contract outside of the payslip period"
Note that it looks like there is a also a demo/admin inconsistency
in data but this is out of the scope of this fix.
X-original-commit: ae259f93b6ccaaec17a8d32df6282278e015c395
Part-of: odoo/odoo#108884
JSON content can be rendered nicely by browsers when using the
appropriate mimetype
closesodoo/odoo#107708
X-original-commit: 70153bbe233bb81d51752c5f1ed1766414eb875b
Signed-off-by: Olivier Dony (odo) <odo@odoo.com>
The motivation of this commit is to get a correct pathname on an
ir_logging when a test fail .
The main issue comes from subtest since an exception inside a subtest
will have only a partial traceback, not containing the line triggering
the error in the test method.
This can also affect debugging since a part of the stack is missing.
See pull request for more informations
X-original-commit: 2b6a8bc79529578ecd21bbb0fd4fe452f28d7cd7
Part-of: odoo/odoo#108202
Pregenerate doesn't work for rtl languages. It shouldn't be a problem
but it was discovered that pregenerate run during upgrades leading to
errors
```
Traceback (most recent call last):
File "/home/odoo/src/odoo/16.0/odoo/service/server.py", line 1314, in preload_registries
env['ir.qweb']._pregenerate_assets_bundles()
File "/home/odoo/src/odoo/16.0/addons/website/models/ir_qweb.py", line 180, in _pregenerate_assets_bundles
_, _, _, id_unique, name = bundle_url.split('/')
ValueError: too many values to unpack (expected 5)tore the set of known environments as it was at setUp
```
This is because the rtl is an extra in the url breaking the logic.
Pregenerate is mainly usefull for tests on runbot
(or running tests locally), to speedup httpcases
avoiding generation at each page load.
IntegrityCases are postinstall but don't requires generating assets, or
at least no so often meaning that we can avoid the pregeneration in this
case. It should also avoid losing some time on pregeneration since it
is not useful.
This new logic only pregenerate if we have at least one HTTPCase in the
test suite. This should also speedup local testing when starting
non http post_install tests.
closesodoo/odoo#107107
X-original-commit: 43a5c17088707eec134d54ccf9341e86859f88c7
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Using patcher.start() can easily lead to incorrect cleanup.
-> after a copy paste, patcher is working, but stop is forgotten
-> stop is present, but won't be called if something fails during the
test
This commit add an utility `start(patcher)` to always have the add
cleanup.
Using a standard way to start the patcher with an automated addCleanup
should prevent this kind of mistake. This is why this commit also
replaces all valid patch.start() (followed immediately by a addCleanup)
closesodoo/odoo#102873
X-original-commit: 7d5a193d86316965a0908c65cfacfb607dc3f3ad
Related: odoo/enterprise#32618
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The /web loading was slow during test when some themes are installed.
This is because without any context and request, the assets are
generated for website 1 before this fix.
With this default website:
_get_active_addons_list returns only addons that are not themes
_get_asset_paths will return less assets
When loading /web, no website is found because the domain is not empty.
This means that the web.assets_backend will have less assets when
generated outside of a request than in a request not matching any
website.
There is normaly no assets for the backend in theme BUT a tour is added
in each theme
('/theme_anelusia/static/src/js/tour.js', 'theme_anelusia', 'website.assets_editor')
('/theme_artists/static/src/js/tour.js', 'theme_artists', 'website.assets_editor')
('/theme_avantgarde/static/src/js/tour.js', 'theme_avantgarde', 'website.assets_editor')
('/theme_aviato/static/src/js/tour.js', 'theme_aviato', 'website.assets_editor')
('/theme_beauty/static/src/js/tour.js', 'theme_beauty', 'website.assets_editor')
('/theme_bewise/static/src/js/tour.js', 'theme_bewise', 'website.assets_editor')
('/theme_bistro/static/src/js/tour.js', 'theme_bistro', 'website.assets_editor')
('/theme_bookstore/static/src/js/tour.js', 'theme_bookstore', 'website.assets_editor')
('/theme_buzzy/static/src/js/tour.js', 'theme_buzzy', 'website.assets_editor')
('/theme_clean/static/src/js/tour.js', 'theme_clean', 'website.assets_editor')
('/theme_cobalt/static/src/js/tour.js', 'theme_cobalt', 'website.assets_editor')
('/theme_enark/static/src/js/tour.js', 'theme_enark', 'website.assets_editor')
('/theme_graphene/static/src/js/tour.js', 'theme_graphene', 'website.assets_editor')
('/theme_kea/static/src/js/tour.js', 'theme_kea', 'website.assets_editor')
('/theme_kiddo/static/src/js/tour.js', 'theme_kiddo', 'website.assets_editor')
('/theme_loftspace/static/src/js/tour.js', 'theme_loftspace', 'website.assets_editor')
('/theme_monglia/static/src/js/tour.js', 'theme_monglia', 'website.assets_editor')
('/theme_nano/static/src/js/tour.js', 'theme_nano', 'website.assets_editor')
('/theme_notes/static/src/js/tour.js', 'theme_notes', 'website.assets_editor')
('/theme_odoo_experts/static/src/js/tour.js', 'theme_odoo_experts', 'website.assets_editor')
('/theme_orchid/static/src/js/tour.js', 'theme_orchid', 'website.assets_editor')
('/theme_paptic/static/src/js/tour.js', 'theme_paptic', 'website.assets_editor')
('/theme_real_estate/static/src/js/tour.js', 'theme_real_estate', 'website.assets_editor')
('/theme_treehouse/static/src/js/tour.js', 'theme_treehouse', 'website.assets_editor')
('/theme_vehicle/static/src/js/tour.js', 'theme_vehicle', 'website.assets_editor')
('/theme_yes/static/src/js/tour.js', 'theme_yes', 'website.assets_editor')
('/theme_zap/static/src/js/tour.js', 'theme_zap', 'website.assets_editor')
Making the bundles for /web different with or without theme, and with or
without website id.
The proposed fix wont return a website if not specified in context and
if not during a request and if not forced to fallback.
The side effect is that the frontend assets where generated with a
website id 1 before that and it won't be the case anymore. This may be
a problem that could slow down frontend call on website 1 when design
theme is installed.
closesodoo/odoo#102690
X-original-commit: a5ed957b37b966bc50ed1219b6af65ed818f04a0
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
The bundle name is irrelevant in the bundle content and will prevent
attachment to store the same file if the bundles are exactly the same.
closesodoo/odoo#102502
X-original-commit: b1d57adf6ff358aa79f41817f4f7bfe60f9e5ac0
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
During the tests, many attachment can be created and unlink.
They will be eventually cleaned up if the cron is ran but this is
usually not the case during tests. The disk usage can increase to reach
more than one 1 Go when design theme is installed.
This can lead to unnecessary big database dumps.
This is also a problem on odoosh where the filestore max size is limited
to 1Go.
The gc should be quite fast if nothing was changed since it will just
check the content of an empty directory.
The method is made accessible in the test case in order to be able to gc
on demand. This may be useful in the test_01_crawl_every_themes that
can generate around 600~ Mo of attachment in the loop.
X-original-commit: 187309f5a39f8fef9b07959fd73475fd6732efb3
Part-of: odoo/odoo#102502
This test is failling during nightly mainly when executed arround 23
The most likely reason for that is because of the freezetime rounding
the day and creating an error between 23 and 00.
The freezetime only wraps the create but the compute will be triggered
only after that. Adding all assertions inside the freezetime may be
enough to fix this issue.
closesodoo/odoo#101989
X-original-commit: 98693d5543e3c0b4f8f51a05e026fce8ddb77a51
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The test was running by luck because the event and all talk pages
contains the needed elements. The favorite was checked randomly on
other pages because those steps are way faster than loading the page.
This test will fail in rare case if the page loads faster and the step
checking if the "favorite is on" occurs when the talk page is finaly
loaded.
This commit adds some step to try to ensure the page are loaded before
doing anything else.
Also enable ticks on freezetime so that we have an idea of the steps
durations in logs for easier investigation.
X-original-commit: 8405104b14624f27df4d95e21738b253d048063f
Part-of: odoo/odoo#101050
Longpolling port is replaced with gevent port and is deprecated
This commit avoid saving the value.
Not really usefull but when saved the value was None leadind to an error
when casting to int. The default value should be an int.
closesodoo/odoo#100991
X-original-commit: adac9a9ceaa173c0181a4fc57d0ea83b7a37fc71
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
When installing l10n_mx, a error occurs
Traceback (most recent call last):
File "/data/build/odoo/odoo/service/server.py", line 1289, in preload_registries
registry = Registry.new(dbname, update_module=update_module)
File "<decorator-gen-15>", line 2, in new
File "/data/build/odoo/odoo/tools/func.py", line 87, in locked
return func(inst, *args, **kwargs)
File "/data/build/odoo/odoo/modules/registry.py", line 91, in new
odoo.modules.load_modules(registry, force_demo, status, update_module)
File "/data/build/odoo/odoo/modules/loading.py", line 482, in load_modules
processed_modules += load_marked_modules(cr, graph,
File "/data/build/odoo/odoo/modules/loading.py", line 371, in load_marked_modules
loaded, processed = load_module_graph(
File "/data/build/odoo/odoo/modules/loading.py", line 206, in load_module_graph
registry.setup_models(cr)
File "/data/build/odoo/odoo/modules/registry.py", line 289, in setup_models
model._setup_fields()
File "/data/build/odoo/odoo/models.py", line 3294, in _setup_fields
field.setup(self)
File "/data/build/odoo/odoo/fields.py", line 512, in setup
self.setup_nonrelated(model)
File "/data/build/odoo/odoo/fields.py", line 4524, in setup_nonrelated
m2m = model.pool._m2m
AttributeError: 'Registry' object has no attribute '_m2m'
Since #99438 hr is sintalled automatically
This means that when doing a -i l10n_mx, hr is installed too
hr will conditionnaly call a button_immediate_install in this case.
What's going on after that: It's a mess
button_immediate_install will create another registry
This registry will be present on some models that will ne used later
by the initial registry creating the missing _m2m case
For the record, here are some of the strange and key element during
the install
2022-09-21 16:31:27,691 2579263 INFO test_mx odoo.modules.loading: loading 1 modules...
2022-09-21 16:31:27,691 2579263 INFO test_mx odoo.modules.loading: Loading module base (1/1)
...
2022-09-21 16:31:36,222 2579263 INFO test_mx odoo.modules.loading: Module base loaded in 8.53s, 8902 queries (+8902 other)
2022-09-21 16:31:36,222 2579263 INFO test_mx odoo.modules.loading: 1 modules loaded in 8.53s, 8902 queries (+8902 extra)
2022-09-21 16:31:36,240 2579263 INFO test_mx odoo.modules.loading: updating modules list
2022-09-21 16:31:36,241 2579263 INFO test_mx odoo.addons.base.models.ir_module: ALLOW access to module.update_list on [] to user __system__ #1 via n/a
2022-09-21 16:31:36,943 2579263 INFO test_mx odoo.addons.base.models.ir_module: ALLOW access to module.button_install on ['Mexico - Accounting'] to user __system__ #1 via n/a
...
2022-09-21 16:31:49,775 2579263 INFO test_mx odoo.modules.loading: Loading module base_install_request (26/78)
...
2022-09-21 16:31:50,043 2579263 INFO test_mx odoo.addons.base.models.ir_module: ALLOW access to module.button_install on ['Project', 'Email Marketing', 'Employees', 'Knowledge', 'Sign', 'Planning', 'Appointments', 'Surveys'] to user __system__ #1 via n/a
2022-09-21 16:31:50,167 2579263 INFO test_mx odoo.modules.loading: Module base_install_request loaded in 0.39s, 228 queries (+228 other)
...
2022-09-21 16:32:10,816 2579263 INFO test_mx odoo.modules.loading: Loading module l10n_mx (55/78)
...
2022-09-21 16:32:17,618 2579263 INFO test_mx odoo.modules.loading: Module l10n_mx loaded in 6.80s, 4302 queries (+4332 other)
...
2022-09-21 16:32:50,371 2579263 INFO test_mx odoo.modules.loading: Loading module hr (35/112)
2022-09-21 16:32:57,594 2579263 INFO test_mx odoo.addons.base.models.ir_module: ALLOW access to module.button_immediate_install on ['Employees - Mexico'] to user __system__ #1 via n/a
2022-09-21 16:32:57,594 2579263 INFO test_mx odoo.addons.base.models.ir_module: User #1 triggered module installation
2022-09-21 16:32:57,595 2579263 INFO test_mx odoo.addons.base.models.ir_module: ALLOW access to module.button_install on ['Employees - Mexico'] to user __system__ #1 via n/a
...
2022-09-21 16:32:58,558 2579263 ERROR test_mx odoo.modules.registry: Creating Registry <odoo.modules.registry.Registry object at 0x7f02cc0e8130>
Stack (most recent call last):
File "/home/xdo/osrc/master/odoo/odoo-bin", line 8, in <module>
odoo.cli.main()
File "/home/xdo/osrc/master/odoo/odoo/cli/command.py", line 56, in main
o.run(args)
File "/home/xdo/osrc/master/odoo/odoo/cli/server.py", line 179, in run
main(args)
File "/home/xdo/osrc/master/odoo/odoo/cli/server.py", line 173, in main
rc = odoo.service.server.start(preload=preload, stop=stop)
File "/home/xdo/osrc/master/odoo/odoo/service/server.py", line 1391, in start
rc = server.run(preload, stop)
File "/home/xdo/osrc/master/odoo/odoo/service/server.py", line 570, in run
rc = preload_registries(preload)
File "/home/xdo/osrc/master/odoo/odoo/service/server.py", line 1289, in preload_registries
registry = Registry.new(dbname, update_module=update_module)
File "<decorator-gen-15>", line 2, in new
File "/home/xdo/osrc/master/odoo/odoo/tools/func.py", line 87, in locked
return func(inst, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 91, in new
odoo.modules.load_modules(registry, force_demo, status, update_module)
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 482, in load_modules
processed_modules += load_marked_modules(cr, graph,
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 371, in load_marked_modules
loaded, processed = load_module_graph(
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 248, in load_module_graph
getattr(py_module, post_init)(cr, registry)
File "/home/xdo/osrc/master/odoo/addons/hr/__init__.py", line 19, in _install_hr_localization
l10n_mx.button_immediate_install()
File "<decorator-gen-74>", line 2, in button_immediate_install
File "/home/xdo/osrc/master/odoo/odoo/addons/base/models/ir_module.py", line 75, in check_and_log
return method(self, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/addons/base/models/ir_module.py", line 486, in button_immediate_install
return self._button_immediate_function(type(self).button_install)
File "/home/xdo/osrc/master/odoo/odoo/addons/base/models/ir_module.py", line 607, in _button_immediate_function
registry = modules.registry.Registry.new(self._cr.dbname, update_module=True)
File "<decorator-gen-15>", line 2, in new
File "/home/xdo/osrc/master/odoo/odoo/tools/func.py", line 87, in locked
return func(inst, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 79, in new
registry.init(db_name)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 115, in init
_logger.error(self, stack_info=True)
2022-09-21 16:32:58,580 2579263 INFO test_mx odoo.modules.loading: loading 1 modules...
2022-09-21 16:32:58,581 2579263 INFO test_mx odoo.modules.loading: Loading module base (1/1)
...
2022-09-21 16:33:35,294 2579263 INFO test_mx odoo.modules.loading: Loading module base_install_request (30/85)
...
2022-09-21 16:33:35,745 2579263 INFO test_mx odoo.modules.loading: Module base_install_request loaded in 0.45s, 131 queries (+131 other)
...
2022-09-21 16:34:05,789 2579263 INFO test_mx odoo.modules.loading: Loading module l10n_mx (62/85)
...
2022-09-21 16:34:28,497 2579263 INFO test_mx odoo.modules.loading: Loading module hr (35/113)
...
2022-09-21 16:34:32,067 2579263 INFO test_mx odoo.modules.loading: Module hr loaded in 3.57s, 4092 queries (+4092 other)
2022-09-21 16:34:32,067 2579263 INFO test_mx odoo.modules.loading: Loading module link_tracker (37/113)
...
2022-09-21 16:34:32,690 2579263 INFO test_mx odoo.modules.loading: Module link_tracker loaded in 0.62s, 267 queries (+267 other)
...
2022-09-21 16:35:04,063 2579263 INFO test_mx odoo.modules.loading: Modules loaded.
2022-09-21 16:35:04,068 2579263 INFO test_mx odoo.modules.registry: Registry loaded in 125.514s
2022-09-21 16:35:04,068 2579263 INFO test_mx odoo.addons.base.models.ir_module: getting next ir.actions.todo()
2022-09-21 16:35:04,071 2579263 INFO test_mx odoo.addons.base.models.ir_module: next action is "Open Menu"
2022-09-21 16:35:04,094 2579263 INFO test_mx odoo.modules.loading: Module hr loaded in 133.72s, 4344 queries (+81420 other)
...
2022-09-21 16:35:04,094 2579263 INFO test_mx odoo.modules.loading: Loading module link_tracker (37/112)
2022-09-21 16:35:04,163 2579263 ERROR test_mx odoo.modules.registry:
setuping model: ir.model.fields()
registry on model: <odoo.modules.registry.Registry object at 0x7f02cc0e8130>
registry calling setup_models: <odoo.modules.registry.Registry object at 0x7f02e92399d0>
2022-09-21 16:35:04,164 2579263 WARNING test_mx odoo.modules.loading: Transient module states were reset
2022-09-21 16:35:04,165 2579263 ERROR test_mx odoo.modules.registry: Failed to load registry
2022-09-21 16:35:04,165 2579263 CRITICAL test_mx odoo.service.server: Failed to initialize database `test_mx`.
Traceback (most recent call last):
File "/home/xdo/osrc/master/odoo/odoo/service/server.py", line 1289, in preload_registries
registry = Registry.new(dbname, update_module=update_module)
File "<decorator-gen-15>", line 2, in new
File "/home/xdo/osrc/master/odoo/odoo/tools/func.py", line 87, in locked
return func(inst, *args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 91, in new
odoo.modules.load_modules(registry, force_demo, status, update_module)
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 482, in load_modules
processed_modules += load_marked_modules(cr, graph,
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 371, in load_marked_modules
loaded, processed = load_module_graph(
File "/home/xdo/osrc/master/odoo/odoo/modules/loading.py", line 206, in load_module_graph
registry.setup_models(cr)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 293, in setup_models
model._setup_fields()
File "/home/xdo/osrc/master/odoo/odoo/models.py", line 3294, in _setup_fields
field.setup(self)
File "/home/xdo/osrc/master/odoo/odoo/fields.py", line 512, in setup
self.setup_nonrelated(model)
File "/home/xdo/osrc/master/odoo/odoo/fields.py", line 4524, in setup_nonrelated
m2m = model.pool._m2m
AttributeError: 'Registry' object has no attribute '_m2m'
Naive fix here: use button_install instead of button_immediate_install
(not even sure this is 100% correct)
All calls to button_immediate_install should be fixed maybe to avoid
a registryloadingception
closesodoo/odoo#100909
X-original-commit: 919c1f362b1c26dbd04b0501c4d63e2929fb8df4
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The tour test_01_admin_widget_x2many was failing because of a missing
discussion. This discussion was in fact coming from a patch
during at install tests
This wasn't detected on runbot during merge since at_install and
post_install are executed in different builds. This was detected
during the nightly "all no auto tag" build.
Simply use a proper patch to avoid keeping the default value at the
end of the test.
closesodoo/odoo#100491
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Currently the exception is not logged using the logger meaning that the
only indication of the failure is the "module not loaded" error message.
Catching the exception to log it the proper way will help identifying
the cause of the issue, mainly for uninstall tests.
closesodoo/odoo#100431
X-original-commit: b16f850d9aa438aea91b8cbd7c33692e1e1b2f90
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Since #99912 logging an error message doesn't always end qunit tests.
This was mainly to allow to failfast logging qunit errors earlier
without stopping the tests in order to test all qunit anyway.
The logic was to have an end message that stops the test.
Unfortunately some errors will prevent the qunit suite to start
and the test will wait a 1800 long timer. An example was because of
a Missing dependencies. https://runbot.odoo.com/runbot/build/19306352
This new approach will avoid to stop only if the message looks like a
qunit failure and the final message is not there (to be sure).
closesodoo/odoo#100238
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
A previous version tried to log qunit by module but it is still possible
to have multiple random errors in the same test occuring with different
combination. This will help to avoid duplicating automatically
parsed errors in this case.
Also skips the message if it is emppty/undefined.
Part-of: odoo/odoo#100238
The current asset_bundle version uses the last modified date.
This can be problematic in some case.
Even if it is a long time issue, the problem was rediscovered on runbot
with a commit being in the future. Runbot will export all file and set
the write date of the file to the commit date. The main purpose is to
have deterministic bundle version between different builds. This also
allows to generate assets bundle once at install for all post install
subbuild.
The issue here is that the commit date was greater than now(), meaning
that some tests setting custom css in attachment won't trigger the
regeneration of assets bundle. This wasn't really noticeable before
assets pregeneration.
This is not the first time strange issues occurs because of the
last modified logic.
This commit combined all last_modified to generate the bundle
version.
The current adaptation is quick and dirty and this will be reworked in
another post 16.0 freeze assets refactoring and cleanup.
closesodoo/odoo#100160
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The pregenerate of assets_bundle should be done maximum two times
- at the end of an install/update if the module test_assetsbundle is
installed
- at the beginning of post_install tests
Currently, during the nightly (install and all tests are ran at the
same time), the pregeration of assets is triggered at the end of some
tests
File "/home/xdo/osrc/master/odoo/odoo/tests/common.py", line 319, in _tearDownPreviousClass
super()._tearDownPreviousClass(test, result)
File "/usr/lib/python3.8/unittest/suite.py", line 300, in _tearDownPreviousClass
previousClass.doClassCleanups()
File "/usr/lib/python3.8/unittest/case.py", line 731, in doClassCleanups
function(*args, **kwargs)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 704, in reset_changes
self.setup_models(cr)
File "/home/xdo/osrc/master/odoo/odoo/modules/registry.py", line 306, in setup_models
model._register_hook()
File "/home/xdo/osrc/master/odoo/odoo/addons/test_assetsbundle/models/ir_qweb.py", line 13, in _register_hook
The registry.updated_modules is set to allow post_install tests
to select the right tests to execute. This is a way to define if we
updated some modules loading this registry. This list is unfortunatelly
not emptied before running the test, meaning that if register hook are
called on the same registry again, pregenerate will be executed
a second time.
A solution here is to check if the registry is not ready,
meaning that this is the load_modules call to register_hook
and not the setup_models one
closesodoo/odoo#100079
Related: odoo/enterprise#31271
Signed-off-by: Raphael Collet <rco@odoo.com>
Visible_menu check per_read model per model leading to a lot of queries
We can get all readable fields with one query
Computation time goes from ~400 total to less than ~50 total
The different would be even bigger on a server with a distant database.
(higher ping)
Part-of: odoo/odoo#99176
Since bundle generations should be generated before post install tests,
a loading of a /web shouldn't try to save a new version
This is currently breaking for / because of the website_id added in the
extra part of the attachement. Even if the content is the same.
Part-of: odoo/odoo#99176
During tests on runbot, the main reason tours are slow to start is
because the first loading of "/web" need to generate assets bundles.
Generation can take up to 5 seconds time the number of tour.
Before this, the first requests to /web takes around 7 seconds on
runbot and the next ones less than 1 second.
After this commit, the first request to /web takes around 1.5 seconds
Note that the main difficulty is to choose when to generate the assets.
(With optimisations from following commits) the generation time is
arround 30 seconds (21 css + 12 js) the first time, for all modules.
If the attachements exists the generation time is arround 3 seconds
(2.5 js + 0.5 css) mainly because of globs to find usefull files.
In practice for runbot the ideal would be to generate them at
the end of the install so that it is shared for all post_install builds.
Doint it at the end of an install is not wanted in all cases, saas
pregenerated templates and upgrade may avoid doing that.
Doing it in a special runbot step (subcommand) is not possible for niglty
execution (no split, in one go). This needs to be in the code.
It is also not practical for devs wanting to have the same behaviour on
a local machine.
A solution to make the test conditionnal at install is to base the
condion on an existing test_module. test_assetsbundle is a good
candidate here. On runbot, the at_install step (befor split) will
automatically generate assets with this solution.
Assets are also generated before post_install tests. This should be fast
if they already exists and ensure that assets are up to date after
modifying sources locally or after downloading a database on runbot.
It should be fast enough for small database and may be even faster
in the future for small diffs.
Part-of: odoo/odoo#99176
Right now the qunit will log all results at the end.
This means that the runbot may wait for all qunit before detecting the
failure.
This also mean that all failure are in one ir.logging on runbot, making
the automated parsing difficult if multiple modules fails during the
same build.
We could also log all failure immediately, but grouping them my qunit
module will avoid duplicating logs for linked causes (one failure
leading to a `Expected %s assertions, but %s were run` message)
closesodoo/odoo#99912
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Xavier-Do