There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.
The report rendering and call `ir.qweb` instead of `ir.ui.view`.
Part-of: odoo/odoo#85110
Issue
-----
When website is installed, the rendering of template uses a side effect
of the ORM cache (cache shared between sudoed env vs non-sudoed env) and
the fields prefetching feature to work correctly.
The `self.visibility` in (`_handle_visibility`, website/ir_ui_view.py)
is done in sudo mode, then it will fetch all prefetchable fields and put
them in the cache (that will be read in non-sudo mode in the render of
the template). Another example of issue related to this:
https://github.com/odoo/odoo/pull/83341.
Because of this, the fields of mixin `website.seo.metadata` were forced
to be prefetchable (the default for translate is to be not prefetchable
since https://github.com/odoo/odoo/pull/82896), which causes a useless
LEFT JOIN on "ir_translation" in most of business flow.
Fix
---
Remove the `prefetch=True` on mixin fields, and add a extra read to fill
the cache in case of website rendering. It also allows to read these
fields at the same time.
Part-of: odoo/odoo#85220
When serving a page, fetch the requested page and the menu's pages in
one go.
This will only impact pages that do not have a menu entry, which is the
most common form of pages.
So, instead of:
- read page
- read page's view
and later:
- read menu pages
- read menu pages' views
Fetch everything together:
- read served page + menu pages
- read served page's menu + menu pages' views
task-2774979
closesodoo/odoo#85456
X-original-commit: 16bf7cca8e8d452ce9131d9d86ff5b23789da4e1
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Getting the `res.company` record by browsing the `company_id` cached
value from `website` instead of reading it directly on the website
record will avoid a read on the website table.
While this might seems unusual and not elegant, since this is a very
low level method used to render every view, it seems fine.
task-2774979
X-original-commit: bc643a573a7339c82661cbcff2cc59bdd0eec3e3
Part-of: odoo/odoo#85456
We can easily avoid a read on the website table when serving the
homepage by simply caching the website.homepage_id value and then
browsing the record (no query needed) instead of reading it on the
website record.
While this code is not really elegant, it seems like a good tradeoff to
gain a query on this particular homepage serve controller, which is the
main entry point of a website and generally the page the more often
served.
task-2774979
X-original-commit: 69b443b85656c895c66a1f83b235f5d1e53727d4
Part-of: odoo/odoo#85456
* website, website_blog (perf)
If the source values is only composed of empty elements (list of `None`
values), don't try to translate it, as there is nothing to translate
anyway.
While the translation method being called (xml_translate,
html_translate) won't be costly as they will return instantly if the
provided value is falsy, there is still an SQL Query made to get the
callable translation method, which we won't actually use.
This commit avoids that query if we already know we won't do anything
with the translation method.
task-2774979
X-original-commit: e1eb5a6e1610a9550a59ba229a83b9e70e10aa88
Part-of: odoo/odoo#85456
* test_website
The `nocache` URL param was introduced with [1] which allows to bypass
the cache when serving a page.
Bypassing that cache, despite not being the most common flow, was done
in the perf tests since that commit. Ultimately, it was not testing
real use cases anymore (despite still being useful as it would still
prevent perf killer feature to be merged).
This commit ensures the perf tests are also tested with cache.
[1]: https://github.com/odoo/odoo/commit/cbf8d3b3047c94a2ff7fb197bbae92a402e0b58b
task-2774979
closesodoo/odoo#85261
X-original-commit: 5d44d890021cfc7767daa1f366e636288e84f1f6
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
=> `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
=> `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.
Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.
Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.
PR: odoo#78857
Task: 2571224
Issue
-----
When website is installed, the rendering of template uses a side effect
of the ORM cache (cache shared between sudoed env vs non-sudoed env) and
the fields prefetching feature to work correctly.
The `self.visibility` in (`_handle_visibility`, website/ir_ui_view.py)
is done in sudo mode, then it will fetch all prefetchable fields and put
them in the cache (that will be read in non-sudo mode in the render of
the template). Another example of issue related to this:
https://github.com/odoo/odoo/pull/83341.
Because of this, the fields of mixin `website.seo.metadata` were forced
to be prefetchable (the default for translate is to be not prefetchable
since https://github.com/odoo/odoo/pull/82896), which causes a useless
LEFT JOIN on "ir_translation" in most of business flow.
Fix
---
Remove the `prefetch=True` on mixin fields, and add a extra read to fill
the cache in case of website rendering. It also allows to read these
fields at the same time.
Part-of: odoo/odoo#83818
The record is read directly in sudo, the prefetch is in the sudo
environment. When reading the other information there is no longer any
need to make a request.
closesodoo/odoo#82475
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Initialize the savepoint semi-lazily (on-demand) as cycling a savepoint
generates 4~5 queries (depending whether the savepoint is explicitly
released on COMMIT or not):
SAVEPOINT
-- < do stuff>
-- commit
RELEASE SAVEPOINT -- or not
SAVEPOINT
-- close
ROLLBACK TO SAVEPOINT
RELEASE SAVEPOINT
With a lazy savepoint, this is just 0-1 queries (`SAVEPOINT` at the
first explicit query only, creating a cursor and then closing it
immediately is a no-op).
For reliability use a semi-lazy savepoint: always immediately emit a
`SAVEPOINT` on cursor creation, but don't automatically create one
after each `commit`. That limits the issues of overlapping (but
non-nested) savepoints.
Also fix the `generate` API to not use the request: when `website` was
converted to the new API, `cr`, `uid`, and `context` were dropped as
if it were a model... but it's not. So in order to recover an
execution environment, that was looked up on the session.
That, then, turns out to be an issue when `generate` is triggered from
an RPC call: the RPC layer creates its own cursor and environment
separate from the request's which may not have one at
all. Problematically during testing we're in `mono_db` mode, so the
request's cursor/env can be accessed and will be lazily initialized.
This then causes an issue with the `TestCursor`'s savepoints: rather
than be nested, the lifetimes of the request's and RPC's savepoints
only overlap[0]:
|-- rpc --|
|-- request --|
As a result, when the RPC's cursor is committed and released it
automatically released the request's, and the request's explicit
release then fails. This would break `/website:WithContext.test_search`.
By fixing the API of `generate`, it stops triggering the creation of a
request cursor, and therefore the overlap and resulting error.
[0] the laziness or eagerness of the savepointing in the test cursor
has no impact on this issue, as multiple requests have already been
issued on the RPC's test cursor before the request's is even
created
Part-of: odoo/odoo#76243
The attributes field.depends and field.depends_context are problematic
for sharing fields across registries, because they depend on the model's
registry class, which may vary from one registry to another. In order
to make computed fields shareable, we have to move those values away
from fields.
For the same reason, field.recursive should not be inferred, because its
value may depend on the registry, although it is generally not the case.
Moreover, the flag recursive=True is set on a field when field triggers
are determined (on the registry). A compute method may be called before
the flag is set (if no update has been done yet), and that can lead to
incorrect computations.
This happened in test TestUsers2.test_reified_groups in module 'base'.
The user groups view was apparently determined without the flag being
set, and the view depends on the recursive field 'trans_implied_ids',
which was not correctly computed.
We thus force developers to be explicit about recursive computed fields.
The code now logs a warning when the flag is not set up properly.
This prevents some code (in controllers) to retrieve an attachment for a
field that is not stored, as the code only relies on `field.attachment`.
This also makes the field definition more consistent.
closesodoo/odoo#57980
X-original-commit: 7325c3f8c538a8b8c0435963cdda3ced6fe4b324
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
1) Avoid the storage of all country flags as ir_attachment (230+
ir_attachment in a new db) to reduce the filestore of databases.
The country flags are nearly static and not expected to be modified on
Odoo instances.
This change is based on the new "image_url" widget logic (see previous commits).
2) Extend the flag coverage for countries
Add the missing country flags & specify a mapping to provide flags
for overseas administrated countries/territories.
This commit introduce multiple improvements regarding social image:
- (perf) Don't read ir.attachment through `social_default_image` when it is
not needed. Use a stored boolean to know if the field should be accessed.
This will remove one SQL query in attachment for public user.
- Show website logo, not the company logo since we now have a different logo
for website.
- Don't show images lower than 200 width or 200 height px. Logo will be
shown regardless of his size.
- Don't show website logo if there is a website social_default_image.
Indeed, the spec was to prevent showing logo and social_default_image if
they are the same image. Technically, this is hard to identify as they
could be the same image uploaded with different resolutions (media dialog),
especially if one of those was uploaded through the backend and one from
the frontend.
It is most likely we will never correctly identify duplicate as they won't
be exactly the same.
For this reason, it makes more sense to hide the website logo if the
social_default_image is set. It avoids every issues while it makes sense
since you won't want to use the logo over the social_default_image. If you
really want to, you could reupload it through the SEO media dialog.
closesodoo/odoo#47848
Related: odoo/upgrade#1012
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Co-authored-by: Romain Derie <rde@odoo.com>
Co-authored-by: Jeremy Kersten <jke@odoo.com>
In case of short controller that don't use qweb template, we don't need
anyhting else that this url code that don't change frequently.
It make only sense for model like lang, website, ... that will not change
frequently. And are called on each call by the dispatcher.
In case of a website page, we will btw browse lang later, but in case of
small controller like /favicon.ico, or page without qweb, ... we can just
use the same from last query.
If model is already an ir.attachment we don't need to do a new search_read.
We can use it record directly.
After this commit, we don't do extra request if we already have the info,
else we don't change the behavior.
task-2211013
The dispatching layer of Odoo (common to any call) can avoid a query on website
if those 3 values are cached.
Note that for complex business flows, there won't be any gain since website
infos will have to be fetched at some point.
About user_id:
In most cases, for a website page you will need to browse the website btw so
we don't remove this request previously. But in case of a simple image, the
controller /web/content need the public user just to set the request.uid in
auth_public but no other info from website.
With this commit, we don't do the 'select * from website where id in()'
request on each controller declared in auth='public' but use the user_id
from the cache. (Invalidated by the write on website_id)
task-2211013
Co-authored-by: Jérémy Kersten <jke@odoo.com>
Co-authored-by: Romain Derie <rde@odoo.com>
Most of the time, the record exists, but we still ensure it does, every single
time, making an SQL query.
Doing a try catch will result in the same behavior, but won't make that query
most of the time.
task-2211013
If we are rendering a page, the menus will most likely be rendered as well.
Note that even in a 403 case when the page is found but is not visible, the
menus will also be shown on the 403 page.
Fetching the menus before accessing the requested page will prefetch that page
as well in one go if that page is in the menus, without any costs for the pages
not in the menus.
There is a tradeoff for 'non-layout' pages, which only occurs in advanced
technical cases:
1. Create a page with specific extension as name suffix (page.css) in which
case the page will be bootstraped accordingly, without call to layout.
2. Remove the call to layout in HTML editor or backend
3. Remove the call to submenus template in HTML editor or backend
For those cases, the menus will be prefetched for no reason.
Note that homepage '/' is rendered through a controller, same logic is applied
there.
task-2211013
Render page flow:
- Search website.page in _serve_page for requested URL
- Check if the page is visible with `is_visible`
- `is_visible` will get the `ir.ui.view` of the page
Note that this is not 'cachable' as being visible depends of now() for the
publishing date
- Render layout which is getting the website menus
Note that this can't be cached either as a menu visibility might also
depends of it's website.page visibility which depends of now()
- Checking menus `is_visible` fetch the menus pages and those pages views.
Before this commit, every level of menu hierarchy would add 2 queries.
This can be avoided by prefetching everything in once during the dispatch if we
know we will render the website menu.
In such a case, read `is_visible` on every website menus will fetch every
pages, views and menus in the lowest SQL queries possible, eg one query for
every table.
For a 3 level menus, requests goes from 20 to 12 to load a untracked page.
task-2211013
- Refactor for easier readability
- Add a test to ensure menu hierarchy scale correctly.
- Add tests for image controllers
- Add test for assets controllers
task-2211013
This will ensure the performances stay correct for a blank page (no layout),
a standard page and a homepage (which has its own controller).
Related to #47257
task-2211013
X-original-commit: 08e01b5fc4a71b8f7639a953137fbb546ddf3b3e
Gorash