Romain Derie
2391d0994a
== Issue ==
A business code error was detected by the internal team on our
production. The cache and assets where invalidated !WAY! too often for
the past months.
It was hard to figure but finally the error was tracked down to be
located in the assets retrieval stack of our code when a database is
accessed through multiple different domains.
In our production use case, whenever one was accessing `odoo.com/web`
after someone accessed `accounts.odoo.com/web`, the assets would be
invalidated and recomputed, again and again, whenever someone accessed
the backend on a domain after someone else did with another domain.
Obviously, on our production, this could be occuring multiple time per
minute.
Technically, this is because the "assets retrieval stack" had a mismatch
in multiple endpoint when trying to find if a current website was
involved (serving for the frontend).
Some business method were using `env.context.get('website_id')` while
others were using `env['website'].get_current_website(fallback=False)`.
From there, when the code was called without a `website_id` in the
context, `get_current_website()` would still return a `website_id` when
called from `http://odoo.com` as there is a website having its domain
set to it. `get_current_website()` is then finding it and returning it.
But it would not when the user is on `http://accounts.odoo.com`.
Since we have a custom scss override (done through our website builder,
basically an ir.asset linked to a "url type" attachment:
`/website/static/src/scss/options/colors/user_color_palette.scss`) for
our website to define the website colors which is shadowing the scss
file from disk.
So, depending of the host/domain, either the real file disk for this URL
or the ir.asset linked to our website for this URL would be fetched to
generate the bundle hash (which is basically the last modification date
of the files/attachments).
Obviously, the file on disk and the ir.assets have a different last
modification date.
The system would then consider the assets as outdated and would
regenerate it.
You can see it in the logs where the attachment id of the assets URL
would get higher and higher everytime you access the DB through another
domain.
Using `get_current_website(fallback=False)`:
- `_get_related_assets()` https://github.com/odoo/odoo/blame/30d3b97b5ece379d9ddcbceda9d12c03dc7f4a48/addons/website/models/ir_asset.py#L14
- `filter_duplicate()` https://github.com/odoo/odoo/blame/30d3b97b5ece379d9ddcbceda9d12c03dc7f4a48/addons/website/models/ir_asset.py#L41
- ..
Using `get_current_website()`:
- `_get_custom_attachment()` https://github.com/odoo/odoo/blame/30d3b97b5ece379d9ddcbceda9d12c03dc7f4a48/addons/website/models/assets.py#L162
- ..
Using `context.get('website_id')`:
- `_get_asset_url_values()` https://github.com/odoo/odoo/blame/30d3b97b5ece379d9ddcbceda9d12c03dc7f4a48/addons/website/models/ir_qweb.py#L23
- ..
== Fix ==
A fix could have been to aligned those to use the same way of retrieving
the website but it would be too fragile (definitely some other places
where the same bug is involved but not yet found).
What is done in this commit is something we wanted to do for a long time
(see [1]) but was based purely on guess and feeling rather than concrete
bug / use case, but now that we found a real use case, we will do it:
- It doesn't seems to make sense to consider the request host/domain
when we are in the backend
- Same for the forced session, those should only impact the frontend
calls.
But this seems to have too much impact in stable to be changed, as it
would require to check every caller to also check for the session if
it makes sense. This will be done in master as not really needed to
prevent the critical bug fixed here.
- When something wants to alter the backend with a website, it should
explicitely be passed in the context, which is still considered
regardless if it's a backend/frontend call.
- If something needs to consider the forced website in session in the
backend, it should explicitely check it, not relying on
`get_current_website()`.
== Step to reproduce ==
- Start a db with website installed
- Enter the website builder in edit mode and change the "Theme Colors"'s
first "Color Presets"'s background color (it is white by default).
- Set the website domain to `http://127.0.0.1:8069/`
- Go to `http://127.0.0.1:8069/web` and login
- Go to `http://127.0.0.2:8069/web` and login
- Now start refreshing those 2 pages one after each other.
Everytime you will refresh the page, it will take a very long time
(~5-10 seconds) before loading the page, and monitoring the logs will
show something about invalidating the cache and huge query count.
== Benchmark ==
For the explained "multiple domain access" case, the backend /web will
now be loaded in less than 10ms and with ~10 SQL Queries when website is
installed, while it was taking ~4 seconds and ~200 Sql Queries before
the fix.
Before the fix:
```
odoo.modules.registry: At least one model cache has been invalidated, signaling through the database
GET host1.com/web HTTP/1.1 200 - 222 0.135 3.840 <-- 222 Queries, ~4s
odoo.modules.registry: At least one model cache has been invalidated, signaling through the database
GET host2.com/web HTTP/1.1 200 - 181 0.101 3.692 <-- 181 Queries, ~4s
odoo.modules.registry: At least one model cache has been invalidated, signaling through the database
GET host1.com/web HTTP/1.1 200 - 215 0.121 3.704 <-- 215 Queries, ~4s
odoo.modules.registry: At least one model cache has been invalidated, signaling through the database
GET host2.com/web HTTP/1.1 200 - 181 0.100 3.616 <-- 181 Queries, ~4s
```
After the fix:
```
odoo.modules.registry: At least one model cache has been invalidated, signaling through the database
GET host1.com/web HTTP/1.1 200 - 101 0.043 0.353 <-- 101 Queries, ~0.3s
GET host2.com/web HTTP/1.1 200 - 11 0.004 0.007 <-- 11 Queries, ~10ms
GET host1.com/web HTTP/1.1 200 - 11 0.003 0.005 <-- 11 Queries, ~10ms
GET host2.com/web HTTP/1.1 200 - 11 0.003 0.008 <-- 11 Queries, ~10ms
```
[1]: https://github.com/odoo/odoo/pull/94161#discussion_r904780031 (Also other PR/task but couldn't find those.)
closes odoo/odoo#120364
X-original-commit: 28dd35eb3c681b630f0b3c109a7d8209f9fa42d8
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Odoo Website Builder
Get an awesome and free website, easily customizable with the Odoo website builder.
Create enterprise grade website with our super easy builder. Use finely designed building blocks and edit everything inline.
Benefit from out-of-the-box business features; e-Commerce, events, blogs, jobs announces, customer references, call-to-actions, etc.
Edit Anything Inline
Create beautiful websites with no technical knowledge. Odoo's unique 'edit inline' approach makes website creation surprisingly easy. No more complex backend; just click anywhere to change any content.
"Want to change the price of a product? or put it in bold? Want to change a blog title?" Just click and change. What you see is what you get. Really.
Awesome. Astonishingly Beautiful.
Odoo's building blocks allow to design modern websites that are not possible with traditional WYSIWYG page editors.
Whether it's for products descriptions, blogs or static pages, you don't need to be a professional designer to create clean contents. Just drag and drop and customize predefined building blocks.
Enterprise-Ready, out-of-the-box
Activate ready-to-use enterprise features in just a click; e-commerce, call-to-actions, jobs announces, events, customer references, blogs, etc.
Traditional eCommerce and CMS have poorly designed backends as it's not their core focus. With the Odoo integration, you benefit from the best management software to follow-up on your orders, your jobs applicants, your leads, etc.
A Great Mobile Experience
Get a mobile friendly website thanks to our responsive design based on bootstrap. All your pages adapt automatically to the screen size. (mobile phones, tablets, desktop) You don't have to worry about mobile contents, it works by default.
SEO tools at your finger tips
The Promote tool suggests keywords according to Google most searched terms. Search Engine Optimization tools are ready to use, with no configuration required.
Google Analytics tracks your shopping cart events by default. Sitemap and structured content are created automatically for Google indexation.
Multi-Languages Made Easy
Get your website translated in multiple languages with no effort. Odoo proposes and propagates translations automatically across pages, following what you edit on the master page.
Designer-Friendly Templates
Templates are awesome and easy to design. You don't need to develop to create new pages, themes or building blocks. We use a clean HTML structure, a bootstrap CSS.
Customize every page on the fly with the integrated template editor. Distribute your work easily as an Odoo module.
Fluid Grid Layouting
Design perfect pages by drag and dropping building blocks. Move and scale them to fit the layout you are looking for.
Building blocks are based on a responsive, mobile friendly fluid grid system that appropriately scales up to 12 columns as the device or viewport size increases.
Professional Themes
Design a custom theme or reuse pre-defined themes to customize the look and feel of your website.
Test new color scheme easily; you can change your theme at any time in just a click.
Integrated With Odoo Apps
e-Commerce
Promote products, sell online, optimize visitors' shopping experience.
Blogs
Write news, attract new visitors, build customer loyalty.
Online Events
Schedule, organize, promote or sell events online; conferences, trainings, webinars, etc.