Avoid to base64 encode, then decode to process assets and images for a ~25% speed improvement.
Change image processing tool to work on images, rather than base64 encoded strings.
Performance is ~25% faster on assets & images:
/web/assets/...frontend.min.css: 13ms to 7ms, base64 enc/dec: 2 -> 0
/web/image/XML_ID: 10ms to 8ms, base64 enc/dec: 3 -> 0
/web/image/res.users/2/avatar_128: 40ms to 20ms, base64 enc/dec: 6 -> 2
closesodoo/odoo#82851
Related: odoo/enterprise#23537
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
Three supported types:
- btree (default for index=True)
- btree not null (when >90% of the data are null)
- gin trigram search (for char fields)
Review of indexes on all objects.
closesodoo/odoo#83015
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
Before https://github.com/odoo/odoo/commit/254d40ffcc21f56a80a1814da7f6355d44e5670b it was possible that when a user cropped an image
using the crop tool, a copy of the original image was created that
had both original_id and theme_template_id carried over, therefore
find could be multi-record. We strengthen that search by adding
original_id=False, which ensures we always fetch the record of the
original image and not also its copies. In addition, databases that
happen to have this issue of corrupted data will be fixed
during an upgrade with https://github.com/odoo/upgrade/pull/3110.
closesodoo/odoo#82725
X-original-commit: e4bb0cd4466404e2fba62ae2d3bbaebd831fbcb2
Signed-off-by: Christophe Simonis <chs@odoo.com>
website{_*}: website, website_blog, website_event, website_forum,
website_sale, website_slides
Since the generic search bar was introduced in [1] all text fields were
truncated in search results.
This caused problems for long URLs which were truncated as well, and
therefore could become invalid.
After this commit URL fields specify `'truncate': False` in their search
detail metadata, which informs the rendering to skip the text truncation
step for that field.
Also added previously missing controller-level tests of the
autocompletion.
Steps to reproduce:
- start odoo with website_forum and demo data
- go to the Help forum
- search for "configure" in the Help forum
- click on the auto-complete suggestion
- => redirected to a 404 page because the URL was shortened
To test the fix on other models, use a long enough name that causes the
problem. E.g.: "This product has such a long name its URL would have
been truncated without the fix contained in this branch".
Note that the problem did not occur on blogs because the URL does not
contain the name, but the same fix was applied for consistency.
[1]: https://github.com/odoo/odoo/commit/7559626c54e34b41e1549e28276a650accec6986
task-2727788
closesodoo/odoo#82621
X-original-commit: 045f741be35e62f5e3a636490c6c1d475b5d78eb
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
The record is read directly in sudo, the prefetch is in the sudo
environment. When reading the other information there is no longer any
need to make a request.
closesodoo/odoo#82475
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Before this commit, if a theme record (eg `theme.ir.ui.view`) was deleted,
its copy_ids would not be.
While this is perfectly normal and wanted behavior when this is done in a
website context (to not alter other websites), it shouldn't be the case when
performing a theme update through CLI/Migration (or if the user find a way to
update the module through the UI).
Fixes https://github.com/odoo/upgrade/pull/3048
task-2593407
opw-2680866
opw-2685951
opw-2685124
opw-2679040
closesodoo/odoo#81953
X-original-commit: 3146dd72e6cb07d6e78ca763325f13dd54de0086
Related: odoo/design-themes#548
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Some theme are enabling an header template. When doing so, they also disable
the default header template.
But this is not enough, as the user could have changed that template and it is
not the default one anymore.
Then, updating that theme (UX, CLI, Migration) will raise a traceback.
Step to reproduce:
- Create a website and install theme avantgarde
- Enter edit mode and select Magazine header template
At that point, update the theme, either:
- Through the UI, on theme switch screen, click on Update
- Through CLI, just run a `-u theme_avantgarde`
-> A traceback will be raised about an xpath error, as both the magazine header
template and the hamburger header template are active at the same time. Only
one template is supposed to be activated.
The issue also impact migration, as migrated website can't be accessed due to
the xpath error on rendering.
Theme being impacted (at least): Avantgarde, Graphene & Nano
Note that when installing one of those themes for the first time on a website,
the error won't occur as `_reset_default_config()` will be called through
`_theme_remove()`.
Note that this fix will ensure the correct template is set (and all others are
disabled), but the scss variable won't be correctly set (as it would be if that
template change was done through the right panel).
This is not that much of a problem (considering what it solves), any later
change from the user through the right panel will solve that mismatch.
Fixes https://github.com/odoo/upgrade/pull/3048
task-2593407
opw-2680866
opw-2685951
opw-2685124
opw-2679040
X-original-commit: f18cd32a936829d8a059db0d259189503ee5317f
Part-of: odoo/odoo#81953
Those were not accounted for, leading to fstrings passing through
unflagged.
Also update the SQL checker to be stricter but smarter:
The previous version would "fail open", unknown nodes would be allowed
through hence f-strings not being flagged when they started appearing
in arg0 position, should now fail-closed, anything that's not allowed
is forbidden.
This flags a few more cases, all of which seem acceptable upon review.
However the previous version would also only resolve arg0 (in case it
had a `NAME`, to see if that resolved to an acceptable form of
query-building). The new version performs resolution during
`_check_concatenation` and should thus allow e.g. format strings to be
separate variables (though not e.g. module-level constants, yet
anyway).
In resolution, replace the ad-hoc process by astroid's built-in
`lookup` which seems to provide the same information. Slightly more in
fact, as it yields every assignment in case of e.g. conditionals, but
making use of that would require a lot more changes in the checker so
leaving the behaviour as-is for now.
It's important to *not* use `ilookup` here, because ilookup is not
"iterable" but "inferring", and we don't want values, we want
expression ASTs for analysis.
NOTE: previous improvements as well as fixes to existing code were
only implemented in 14.0, hence this being merged in 14.0 not 13.0
despite 13.0 still being supported.
closesodoo/odoo#81721
X-original-commit: 376ccf0944dae1bc53ae9c5385977c4e6b23e083
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Before this commit, the "ripple effect" no longer worked because the
assets were never activated for the following reason:
- To activate the ripple effect assets via the editor options, we
activated a template that no longer exists (with
data-customize-website-views). Instead of activating the assets with the
new system of assets using records.
After this commit, a new "data-customize-website-assets" xml attribute
was created so that the assets can enable/disable in the same way as the
views. The "write" method for ir.asset has also been overridden in
website so that each website has its specific assets (via COW).
task-2686370
closesodoo/odoo#81833
X-original-commit: 9f56357cc1f4a7b8606ef4d5fd431fc396bdf1e8
Related: odoo/design-themes#546
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
The http.addons_manifest is a map {module: manifest_dict} that is
populated upon the first http request. This map is basically a module
manifest cache with an extra `addons_path` key, the path of the module
on the file-system. This cache is eagerly populated upon the first http
request, the map is empty in non-http contextes (e.g. cron) which have
been a source of bugs (e.g. 50c8eb1).
A manifest cache is necessary because reading and parsing python files
from the file-system is not that cheap but there is no reason that cache
is located in `odoo.http`. A thin cache layer now wraps
`load_information_from_description_file()`/`load_manifest()` and is
lazily populated.
The `http.addons_manifest` have been removed. The extra `addons_path`
key is now present in the "normal" manifest. The `read_manifest()` was
hardly used so it has been deprecated. The only way to retrieve a
manifest is now `load_information_from_description_file()` which was
renamed `load_manifest()` (no cache) and `get_manifest()` (cache).
Side note about performances, the cache is necessary. Addons manifest
are read-only and reading + parsing python files from the file system is
not a cheap operation. Running the e-commerce tour
`@website_sale.test_04_admin_website_sale_tour` without cache on
`load_manifest()` requires 68,29 secs to complete on my laptop,
exceeding the default 1-minute time frame allowed in tests. Using a
cache the time is down to 36,53 secs. The performance impact is huge.
Part-of: odoo/odoo#79977
Before this commit when covers were edited from their list view (e.g.
changing their background color from the blog posts list or from the
events list), the resize class was lost.
After this commit the resize class is kept from its previously saved
value if it is not a parameter during edition.
Related to task-2359250 and task-2678100
closesodoo/odoo#81349
X-original-commit: 4e5aa57913c4270d0e9de6e7318eb5d25daaddf4
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Co-authored-by: qsm-odoo <qsm@odoo.com>
Prior to this commit, when using a ir.filter on a dynamic snippet,
the filter would not use the product context defined in itself.
This would cause unexpected behaviour like ignoring
the 'display_default_code: False' in the website_sale ir.filter.
This commit adds context to database record lookups.
task-2654924
X-original-commit: d10d406739239e8d8fc240b2c6b2596b1ee6e610
Part-of: odoo/odoo#80867
Generic search bar was introduced with #65871.
But the trigram search would fail if it encounters a (SQL) NULL text
from the database. This is because when word_similarity() receives NULL
as a parameter it returns NULL, and NULL not being a number cannot be
compared with the similarity threshold.
After this commit the trigram search will exclude matches returning a
NULL similarity.
Step to reproduce:
- Enable trigram in the DB, run `CREATE EXTENSION pg_trgm;` in psql
- Create a product, name it `myproduct`
- Insert NULL values inside the translation table (not sure how the
user did it):
`update ir_translation set value = NULL where src = 'myproduct';`
- Search 'mypro' in the eshop search bar
-> Traceback
opw-2680659
closesodoo/odoo#80426
X-original-commit: ef17edc32d4646c93e31e730249be83469951241
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Before this commit when a website page was deleted, its related website
menu was deleted inside the database only through the ondelete="cascade"
of the Many2one relationship. This however did not perform the menu
cleanup on the ORM side.
This caused problems when the related menu id was kept in cache by
website's _get_menu_ids(), because the record could not be fetched
anymore. E.g. deleting a website-specific version of the "Contact Us"
page caused the problem, making the menu bar non renderable on all
pages.
After this commit the caches are cleared when a page is unlinked to
avoid this problem.
task-2686221
closesodoo/odoo#80348
X-original-commit: dd9997753d442f8923b31c3ceaa0cd29b09a06b2
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Before this commit the default template for mega menus was the "Multi
Menus" template.
After this commit the default template for mega menus is the "Odoo Menu"
template.
task-2675252
closesodoo/odoo#80040
X-original-commit: a4d686140133ecf59a3d09c8f930a70b91064599
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Before this commit, the behavior disabling the unused snippet assets was put
directly in the autovacuum CRON instead of its own CRON.
That behavior is leading to some perf-issue running the autovacuum cron since
it would take too much time.
Indeed, the code will loop inside as many snippet as it exists, and every time
then perform an SQL Query to get the HTML fields using the snippet.
An improvement will come later to not loop on every snippet anymore.
task-2694120
closesodoo/odoo#79981
X-original-commit: 96ab95d37c9b442a4ccb737fd867ca91ec1c11b6
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Before Odoo saas-14.4, one should call `_get_http_domain()` on website to get
its domain. Indeed, that method was in charge of cleaning that domain, as it
was done with commit [1].
Since Odoo saas-14.4, that cleaning is automatically performed on domain before
saving it into database, thanks to commit [2].
Thus, we can now remove the `_get_http_domain()` and use directly the domain as
it is considered clean.
Note that migrated databases coming from version older than Odoo saas-14.4
could still have an incorrect domain (trailing slash, no scheme..).
This will be handled during migration with [3].
[1]: https://github.com/odoo/odoo/commit/3ad775aab717b395a5d11527aeb3596af66afa99
[2]: https://github.com/odoo/odoo/commit/042c95b0219bb0aa13e73385e092fa76ff1a1b0a
[3]: https://github.com/odoo/upgrade/pull/2951closesodoo/odoo#78766
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
When using the configurator the user can upload its logo to
generate a color palette. In this case we select the base-1
palette as default palette instead of the theme's default palette.
Doing so avoids the potential customizations the theme palette
may introduces.
task-2665885
closesodoo/odoo#78251
X-original-commit: 50019c8f68b89968615f87c365095c94caeef46c
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
In some case, we know that page will contains dynamic content and that cache
should be ignored.
E.g. as soon as a s_snippet_form will be present in page, we know that we need
the real csrf, and so we cannot use the cached response from another user.
Now, we disable the cache (cache_time=0)
closesodoo/odoo#77935
X-original-commit: 262499a91cc9f2365cf2ff131796d9b61b438f9b
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Some themes replace the website.template_header_default template in the
website layout. This poses a problem when changing to another theme
afterwards because of the interaction between the website templates and
the _reset_default_config method:
1) Since version 15.0 the nav tag inside the header templates was
abstracted into a separate template (website.navbar) and added to each
header layout through t-call.
2) The _reset_default_config deactivates or reactivates views that were
enabled or disabled by the currently active theme. However, when doing
so, the default header is activated before the custom header is
deactivated.
As a consequence, after toggling the default header, there are
temporarily two header extensions in the website.layout template. They
both replace the //header//nav element. Because of the abstraction of
the nav element into a separate template by the header extensions, only
the first replacement will work.
task-2662497
closesodoo/odoo#77917
X-original-commit: b83a2110fd3035cc8466c4fa74f6530be5d00597
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
On theme switch default images of the previous theme are replaced
by the one of the new theme. For website where default images
come from the configurator we don't want them to be replaced by
the theme's images.
task-2614840
closesodoo/odoo#77648
X-original-commit: c25f80c70930f65a928b44d4fe91775c37bc4268
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Initialize the savepoint semi-lazily (on-demand) as cycling a savepoint
generates 4~5 queries (depending whether the savepoint is explicitly
released on COMMIT or not):
SAVEPOINT
-- < do stuff>
-- commit
RELEASE SAVEPOINT -- or not
SAVEPOINT
-- close
ROLLBACK TO SAVEPOINT
RELEASE SAVEPOINT
With a lazy savepoint, this is just 0-1 queries (`SAVEPOINT` at the
first explicit query only, creating a cursor and then closing it
immediately is a no-op).
For reliability use a semi-lazy savepoint: always immediately emit a
`SAVEPOINT` on cursor creation, but don't automatically create one
after each `commit`. That limits the issues of overlapping (but
non-nested) savepoints.
Also fix the `generate` API to not use the request: when `website` was
converted to the new API, `cr`, `uid`, and `context` were dropped as
if it were a model... but it's not. So in order to recover an
execution environment, that was looked up on the session.
That, then, turns out to be an issue when `generate` is triggered from
an RPC call: the RPC layer creates its own cursor and environment
separate from the request's which may not have one at
all. Problematically during testing we're in `mono_db` mode, so the
request's cursor/env can be accessed and will be lazily initialized.
This then causes an issue with the `TestCursor`'s savepoints: rather
than be nested, the lifetimes of the request's and RPC's savepoints
only overlap[0]:
|-- rpc --|
|-- request --|
As a result, when the RPC's cursor is committed and released it
automatically released the request's, and the request's explicit
release then fails. This would break `/website:WithContext.test_search`.
By fixing the API of `generate`, it stops triggering the creation of a
request cursor, and therefore the overlap and resulting error.
[0] the laziness or eagerness of the savepointing in the test cursor
has no impact on this issue, as multiple requests have already been
issued on the RPC's test cursor before the request's is even
created
Part-of: odoo/odoo#76243
Previously the configurator retrieved the snippet lists
needed to build the website pages from IAP. These lists
have been put in the theme manifests and can be accessed
from it.
odoo/design-themes#505odoo/iap-apps#411closesodoo/odoo#77550
X-original-commit: 01c97c697aa0c5cae79952f1d3ac8de69bf2ad1a
Related: odoo/design-themes#510
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Now, iap only return images urls needed to process the svg.
We don't need to 'download' svg preprocess, we use the svg locally and make
the substitution ourself.
It simplify the maintenance:
Preview of theme are in theme repository
It make it blazing fast:
Response contains a list of string, and not more a big string, the svg.
closesodoo/odoo#77199
X-original-commit: 103ec4debb539b04b1783be20f1ccf3e18ab377e
Related: odoo/design-themes#503
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
t-field of a Text field, will use nl2br function that replace \n to <br>.
So it is html that we need to parse.
Add one return line in your sale description, and the rendering will fail
with an error opening and ending missmatch for br...
closesodoo/odoo#77108
X-original-commit: 3b3f9b24362e57b4c1538d8377ef4de5f3d65058
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
`website` added its own sql-compatible version of
`get_unaccent_wrapper` in order to use `psycopg2.sql` for safety.
That is very sad.
Add a condition in the "standard" unaccent wrapper which checks
whether the input type is a `Composable`, and in that case return an
`SQL`. This increases the cost of the wrapper a tad but probably not
to a really sensible amount.
Task 2634851
Part-of: odoo/odoo#76436
A while back, the behavior of this setting was changed from a
related field to a computed as a fix to various issues with: cb9f48d
The setting's value synchronization with what is displayed on the
setting's page is broken, since its behavior changed with commit:
2ccc735 . It resets to its default value for the default website when
editing a secondary one and is not only confusing but breaks the
setting in some scenarios.
When changing the auth_signup_uninvited setting, the changes
are correctly reflected and not reset to default upon changing
the website we are currently editing.
task-2612686
closesodoo/odoo#76538
X-original-commit: 6e7a0664850ccbc30a7a8a3ced22f05347bc6f9e
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
The configurator's palette selection screen didn't support
SVG logo. On SVG upload primary and secondary colors couldn't
be extracted and no palette was recommended. It comes from the
fact that Python PIL library does not support SVG. We therefore
had to convert SVG to PNG before trying to extract primary and
secondary colors. A file format checking has been added. If the
uplodad file is not an image, a notification is displayed to
the user.
task-2602521
closesodoo/odoo#76400
X-original-commit: 33f43618f727bdc1239c52c66b73f0f436035774
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Refactor the Environments object into a Transaction object, which is
bound to one cursor, and is no longer shared among several cursors.
The following methods/properties have been changed:
- Environment.envs no longer works (because of the design change);
- Environment.manage() is deprecated (no longer useful);
- Environment.reset() is now an instance method;
- env.clear_upon_failure() is deprecated in favor of cr.savepoint().
closesodoo/odoo#75598
Related: odoo/enterprise#20451
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
Before this commit the search on pages was only done in the main
language.
After this commit the search on pages is based on the translated "arch"
field.
task-2379555
https://github.com/odoo/odoo/pull/65871
Part-of: odoo/odoo#65871
Before this commit the `_search_get_detail` result contained callback
functions to handle special behavior during fetching and rendering.
After this commit a `website.searchable.mixin` is introduced that must
be inherited by models that participate in website-based searches.
Custom behavior previously achieved with callbacks is now achieved by
overloading methods of this mixin.
task-2379555
https://github.com/odoo/odoo/pull/65871
Part-of: odoo/odoo#65871
Before this commit searched terms were the ones entered by the end-user.
After this commit if the searched term is not found then the search is
performed on a resembling term.
The similarity is obtained from a Levenshtein distance combined with
ratios of common and different letters.
By default the dictionary against which the search term is matched is
based on the content of the search fields containing a word that starts
with the first letter of the search term.
If the `pg_trgm` Postgresql extension is installed the dictionary is
built from matches using the `<%%` operator (similar to the
`word_similarity` function).
Several approaches were benchmarked during development, those results
are available through the task record.
task-2379555
https://github.com/odoo/odoo/pull/65871
Part-of: odoo/odoo#65871
(*: website_blog, website_event, website_forum, website_sale,
website_slides)
Before this commit the search bar was specific to products.
After this commit a generic search bar is available as a general feature
of website which can be configured to inspect specific models.
The snippet is used to replace the old search bar in blog, courses,
event, forum, page and shop.
The search results of these pages and the autocomplete of the search bar
run through the same search mechanism.
A new hybrid results page has also been created as a target of a search
on "Everything".
In each involved module, `website._search_get_details()` is implemented
to return search metadata for every model related to the `search_type`
parameter.
Search metadata for a single model is returned by `_search_get_detail()`
on that specific model.
The autocomplete runs through the additional
`website._search_render_results()` pre-rendering step that prepares the
data to fit in the autocomplete template.
task-2379555
https://github.com/odoo/odoo/pull/65871
Part-of: odoo/odoo#65871
Based on their path, we retrieve snippets assets, assuming that it will
be in the form : /snippets/{snippet_id}/{version}.{scss|js}
Then we check the template definition uses that version, or if any views
or html fields contains a snippet with such version exist.
If not, the asset is deactivated.
For that, we are looking for the data-snippet attribute in snippets
already dropped. For the template definition, we look at the first class
definition if a data version attribute is also defined.
task-2212216
closesodoo/odoo#73005
Signed-off-by: Romain Derie <rdeodoo@users.noreply.github.com>
* = crm_livechat, hr, hr_holidays, im_livechat, mail_bot, purchase, sms,
snailmail, survey, test_discuss_full, test_mail, web_editor, website,
website_livechat
- Create new model `mail.guest` for guests.
- Rewrite some RPCs to target routes rather than model methods so that
guests are able to use them.
- Patch JS and python models to support guests.
- Create a stand-alone page and boot the channel in it.
task-2494829
closesodoo/odoo#75496
Related: odoo/enterprise#20417
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
The saas version of the client is added in each call to
website.api.odoo.com. This allows the server to return
different results based on the client's saas version
task-2602521
closesodoo/odoo#75617
X-original-commit: 6e1ae856584048912cb5eaf53c0aa027b2342b5c
Signed-off-by: Sébastien Mottet <smottet@users.noreply.github.com>
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
*: web
After this commit pre-configured gradients can be used as snippet
backgrounds, snippet filters or as text effects / highlight effects.
The background colors (and now gradients) are now possible to add
*alongside* a color combination class (editing one does not remove the
other). Background colors and gradients are mutually exclusive.
Part of https://github.com/odoo/odoo/pull/73611
task-2599770
closesodoo/odoo#73611
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: Benoit Socias <bso@odoo.com>
Previous implementation always updated *current website*, which is not
always the same as the value from `res.config.settings`.
---
opw-2623007
X-original-commit: d66d44db031be76403ce7c01616e334ef196fd03
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes
closesodoo/odoo#68299
Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
Fabien Pinckaers