Commit Graph
466 Commits
Author SHA1 Message Date
Romain Derie 5456dc499a [IMP] website, *: rename group publisher to restricted_editor
* portal, web_unsplash, website_*

This commit renames the `website.group_website_publisher` into
`website.group_website_restricted_editor`.

While the change in itself might look unuseful, it will help the dev and
tech community figuring which group is related to which feature.

Even internally when we discuss specs, we always have to remind which
group is the restricted editor: the publisher one or the designer one?

While it is probably, after all those years, now anchored in some dev
mind, there is no easy way to directly figure which of those 2 groups is
the restricted editor one.
Note that I myself always got confused about it.

Now, the "Restricted Editor" right will be reflected in its technical
name `group_website_restricted_editor`.
Same as for the "Editor & Designer" which technical name is
`group_website_designer`.

As we would like to have a fully working and ready system in v17 for the
community to be able to build themes easily, removing that dubious part
is a nice to have.

Part-of: odoo/odoo#98200
2022-08-31 23:50:06 +02:00
Romain Derie 5ad404fbb5 [REM] website, *: remove the now useless backend_dashboard tour
* website_sale

That tour keeps failing. A fix attempt was made with [1] but wasn't
enough.
The tour was actually useless as since [2] google analytics dashboard
was not part of the website dashboard anymore (due to google not
allowing GA4 dashboard to be embed).
The tour, which was initially made to ensure that google analytics was
correctly shown even if not yet connected to was then not useful
anymore.

The opportunity is also taken to rename the misleading ID of related
records still mentioning Google.

[1]: https://github.com/odoo/odoo/commit/1a191357e3ac0c4542b2fc7c3a2aee18dc54699d
[2]: https://github.com/odoo/odoo/commit/985e49bdb5e22fa197ba267aa41d7a8ba7e50550

runbot-4498
runbot-4499
runbot-4466

closes odoo/odoo#98090

Related: odoo/upgrade#3774
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-08-17 19:48:28 +02:00
Guillaume (gdi) caf1ca67e9 [FIX] website: permit to open the image wall modal correctly
Since the bootstrap 5 merge at [1], when you click on an image in the
Image Wall block, the modal no longer opens. This commit restores that
and adds a test to make sure this problem does not happen again.

Steps to reproduce:
- Drop the "Image wall" block in a page
- Save
- Click on an image
-> The modal does not open

[1]: https://github.com/odoo/odoo/commit/971e5a91aab96d36129a823e03f1f9f1b1293968

task-2937538

closes odoo/odoo#97053

Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-08-17 19:48:21 +02:00
Arthur Detroux (ard) ddf2e74b4c [FIX] website: fix gray color picker
Commit [1] changed the way styles are handled. Now that the edition of
website is done in an iframe, the styles are computed inside the iframe
instead of the top window / global document element.

But for computing the gray colors, a base style defined
by `web_editor/color_palette.scss` is needed. This style is not part of
the frontend assets, so using the iframe to fetch it creates bogus gray,
which results in the feature not working.

Steps to reproduce:
- Go on the website editor
- Select the THEME tab
- At the bottom expand the gray color picker
- Use the settings
- The preview is not updated while sliding.
- The colors are not updated in the page.

This commit fixes the bug by fetching the base style from the top
window (where backend assets are present).

[1]: https://github.com/odoo/odoo/commit/212a8bfdd21269b18054200b9e2585e1c95540d6

task-2687506

Part-of: odoo/odoo#94564
2022-08-10 20:28:48 +02:00
Arthur Detroux (ard) 24ca4ae3fb [FIX] website: (re)start widgets on a cloned snippet
Commit [1] moved the website builder in the backend and in doing so
transferred event handling to the wysiwyg_adapter from the edit button
widget. Unfortunately, during that process, handling of event for
cloned snippets was forgotten.

This commit restores that and adds a test.

[1]: https://github.com/odoo/odoo/commit/31cc10b91dc7762e23b4bde9b945be0c4ce3fe3b

task-2687506

Part-of: odoo/odoo#94564
2022-08-10 20:28:48 +02:00
Younn Olivier 1a191357e3 [FIX] website: fix backend_dashboard tour
The tour .test_03_backend_dashboard was failing inconsistently with a
"failed to fetch" error (which happens when fetching the frontend
assets, from the iframe).

The test is adapted to start directly in the backend, which seems to
prevent that error from happening.
It is also adapted to [1] which was merged during the investigation of
that error.

[1]: https://github.com/odoo/odoo/commit/985e49bdb5e22fa197ba267aa41d7a8ba7e50550

runbot-4418
runbot-4003

Part-of: odoo/odoo#95890
2022-08-09 11:31:59 +02:00
Younn Olivier e8112e2865 [FIX] website: keep focus on text when using snippet options
Before [1] was merged, for this flow:
- Drop a snippet in the page
- Click on some paragraph
- Click on a snippet option's widget
=> The text selection was lost on Chrome, but kept on Firefox.

With [1] instantiating the SnippetsMenu on a different document than the
snippets one, it was possible to keep the text selection (the snippets
document selection) when clicking on the snippets options (on the global
document).
Some code was added to preserve the Chrome behaviour and lose the text
selection. This code is reverted to keep the Firefox behaviour.

[1]: https://github.com/odoo/odoo/commit/31cc10b91dc7762e23b4bde9b945be0c4ce3fe3b

task-2687506

Part-of: odoo/odoo#96401
2022-08-08 12:32:50 +02:00
Romain Derie 7f1653a1de [IMP] website: remove the iframefallback in test mode
This commit gets rid of the iframefallback in test mode. It is not
helpful there and slows down the tests as it involves some HTTP
requests.
Its purpose is only for the end users to have a smoother UX during page
transition. It is irrelevant in tests. Note that it means the tests
wouldn't break if that iframe fallback was to be breaking something in
real use cases.

task-2687506

closes odoo/odoo#96229

Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-08-08 10:50:56 +02:00
Romain Derie 1ac43fab05 [IMP] website: fix python linter errors
Just a nice to have. It will prevent those errors to be replicated when
copy pasted and will help reading the files in the IDE.

closes odoo/odoo#97282

Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-08-04 20:11:00 +02:00
Romain Derie 701a775ec4 [FIX] website: add longer timeout to long website form test
This test is quite long as it is testing all the website form behaviors
in edit mode.
It is sometime failing due to the tour taking longer than 60 seconds.

runbot-4257

closes odoo/odoo#97336

Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-08-02 18:44:02 +02:00
Younn Olivier cd2a649b8b [FIX] web, mass_mailing, website: remove frontend debug icon
After [1] moved the website edition in the backend, we feel that the
code required to make the additional debug icon in the footer of the
frontend still work is not worth it.

Before this commit, on a website page, it was not working: clicking on
it from the WebsitePreview client action would only remove the debug
mode of the frontend, and the backend would keep it.
The user can now leave the debug mode from the debug menu systray item
and we consider that leaving debug mode as a visitor (from the
frontend) is not a feature worth keeping.

[1]: https://github.com/odoo/odoo/commit/31cc10b91dc7762e23b4bde9b945be0c4ce3fe3b

task-2687506

closes odoo/odoo#95499

Related: odoo/upgrade#3721
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-07-29 15:53:12 +02:00
Nasreddin Boulif (bon) 1ebba3ea27 [FIX] website: fix test_01_get_current_website_id
Cause:

  The test fails because there is some demo data that added additional
  website(s) and when trying to retrieve the current website without
  domain, it might retrieve the wrong one.

Solution:

    Unlink unused website(s).

opw-2899680

closes odoo/odoo#96810

X-original-commit: 45dfbae7301effe0d96163d67218a7854c3bde6f
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-07-28 03:50:53 +02:00
Benjamin Vray 9fe6d82deb [FIX] website: fix "over the content" option not saved
Since this commit [1], when you select the Header position option and
set it to "Over The Content", the change is not saved if you did not
edit something else in the page too.

Indeed, now that the "save" works correctly (and that it no longer saves
the page every time). It does not detect the change made by this option
which is to add a class on the '#wrapwrap' element. (The editor looks
for changes inside the wrapwrap but not on it).

This commit checks if any page option is dirty on save on top of
checking if the content of the page has been modified.

Additionally, this commit fixes a bug that would occur on Firefox based
browsers where the hidden input used to display the status of these
options would be auto-completed by the browser in some conditions.
(https://bugzilla.mozilla.org/show_bug.cgi?id=520561).
This could result in an incorrect state being displayed.
To reproduce:
  - Change the option, click on cancel, start the editor
  - the incorrect state is displayed

A test is also added by this commit to verify that all the page options
works correctly.

[1]: https://github.com/odoo/odoo/commit/bab673488e185ddd7792aedecc3870663290fed3

task-2871426

X-original-commit: ea7a69b344d970cc2b31555f4cd8fd7c5e2fc6d2
Part-of: odoo/odoo#94949
2022-07-27 02:18:05 +02:00
Benoit Socias b91fdd65f1 [FIX] website: write website-specific views before they get a new id
When both a specific inheriting view and a non-specific base view are
written simultaneously, the specific inheriting base view must be
updated even though its id will change during the COW of the base view.

This commit sorts the list of written views in order to first handle the
website-specific ones then the base ones which might change some of the
ids of the already updated view.

Steps to reproduce in 14.0+ (no scenario found in 13.0):
- Create a new website.
- Configure the language selector layout to "Inline".
- Configure the language selector layout to "None".
=> Error notification was displayed and change was not applied.

task-2885882

closes odoo/odoo#96248

X-original-commit: 317eea48186c948009399daedf70dd407f6a9ca8
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-07-18 21:03:12 +02:00
Younn Olivier 22d25c94ab [FIX] website: use registerEditionTour for edition tours
Before this commit, the tours default_shape_gets_palette_colors and
edit_link_popover were not using the tour utils function
registerEditionTour, that starts a tour on the iframe, in edit mode,
with a timeout on the first step.

Because this timeout was not there, loading the iframe and the snippets
menu could take longer than the default timeout and the test would fail
inconsistently.

task-2687506

closes odoo/odoo#96164

Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-07-18 19:55:00 +02:00
Romain Derie 5505ef355c [FIX] website: make enable_editor=1 links work from backend
Since frontend to backend commit [1], links having `enable_editor=1` to
enable edit mode were not working anymore in certain cases.

Case 1 [OK]*:
From frontend, click on `enable_editor=1` link without `/@/` in it.

Case 2 [OK]:
From frontend, click on `enable_editor=1` link with `/@/` in it.

Case 3 [KO]*:
From backend, click on `enable_editor=1` link without `/@/` in it.

Case 4 [KO]:
From backend, click on `enable_editor=1` link with `/@/` in it.

This commit fixes case 4 and adds a test for all cases, while leaving
case 3 commented as it will be fixed later with a larger fix related to
URL change listener.

* Note that we will probably change the behavior for case 1 and 3: if
  the version with /@/ + enable_editor=1 enters edit mode properly in
  both frontend and backend contexts (case 2 and 4), it's probably not
  worth having code to support case 1 and 3.

The issue for case 4 was that it was trying to load the whole Odoo app
inside of website preview iframe instead of making the parent window
load that URL.

[1]: https://github.com/odoo/odoo/commit/31cc10b91dc7762e23b4bde9b945be0c4ce3fe3b

closes odoo/odoo#96054

Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-07-15 14:22:11 +02:00
Julien Castiaux dce5dade01 [FIX] website: missing alternate URLs for pages
Install multiple langages, each time translating the website. In a
private browsing session access /contactus, show the page source, the
multiple alternate URL (`<link rel="alternative">` in the `<head>`) are
all pointing the canonical URL instead of the alternative.

closes odoo/odoo#95683

X-original-commit: 3d4b4d3dcff864613a9e7038137e21674425ed08
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
2022-07-08 19:53:07 +02:00
Gorash b3a3969b2e [FIX] website, *: href form link alternate depends on the current url
*: test_website

Issue: The url is cached and no longer depends on the url. This part
should not be cached.

closes odoo/odoo#95222

X-original-commit: 4b255ed129b0e458d3cf7f432d076913b14d5450
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-07-06 17:10:17 +02:00
Gorash ae3354b306 [FIX] web, *: debug icon consistency according to the mode and the url
*: website

X-original-commit: 0cd55d90121cd5bcebdf4c7b0212910ef44a9d85
Part-of: odoo/odoo#95222
2022-07-06 17:10:17 +02:00
Raphael ColletandVincent Schippefilt eb67feb590 [FIX] *: cache consistency
In module mail, invalidating 'message_ids' on a mail thread also
invalidates its inverse field 'res_id' on messages.  If you haven't
flushed it before, your cache will be inconsistent, as shown by the test
/mail:TestMailgateway.test_message_process_bounce_records_channel.

In module purchase_stock, add depends on report.stock.quantity.  This
ensures that when the model is queried after changes in other models,
the data on which the SQL view depends is flushed to the database before
querying that model's table.

closes odoo/odoo#66938

Related: odoo/enterprise#16722
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Vincent Schippefilt <vsc@odoo.com>
2022-07-05 11:35:01 +02:00
Romain Derie 13efb309b4 [FIX] website: correct the website editor tour
Introduced with [1] the goal of this tour was to check some lazy loading
stuff regarding the edit mode.
It was then "wrongly" adapted with [2] where the tour actually still
started in the frontend and then clicked on a the new UI button (top
left floating icon of the frontend > backend task) to indirectly enter
edit mode.
While it was working, that's not what the test was aiming to test. The
test should remain as it was, testing the edit mode. While the edit btn
is now in the backend and not the frontend, the tour should just start
there.

Note that this commit is required as the behavior the tour was now
relying on is removed by the previous commit: the Edit button is not
entering edit mode anymore.
Instead of adding a new step to click on the Edit btn, let's just start
the tour as before on the step where you can click on Edit.
It will make the test less dependent of other behavior. This is
especially true since that frontend "Edit" button will most likely
change again or be removed (and introduced back?). We don't want the
tour to rely on this.
If those new UI button need to be tested, it should be in another test.

[1]: https://github.com/odoo/odoo/commit/6f4c60fe1bb6e460ec6da00c59a9825271893729
[2]: https://github.com/odoo/odoo/commit/99b50d18e220aedf14de806f4bf1b2d35c32de35

closes odoo/odoo#95002

Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-07-04 14:07:41 +02:00
Romain Derie 030d3cb10e [IMP] website: use custom URL for website in backend
The website architecture has been refactored for the internal users and
admins with [1].
Now, they can access the website in the backend inside the website app.
Long story short, the website will be displayed in an iframe in a client
action. The URL of the browser will be tweaked to reflect the iframe one
instead of the real one (which is something like /web#action=..).

It had a few drawbacks:
- On page refresh (F5 or browser button), the user would land on the
  frontend version of the website instead of remaining in the backend.
- When the user edited the URL (Like removing `/shop` and typing `/jobs`
  instead, he would land on the frontend version too.
- Impossible to directly go to the backend version of the website.

Those are improved with this commit by using a slightly different URL
when we are in the backend. A `/@/` will prefix the iframe URL.

If logged in, the user will land on the backend. If not, it will simply
redirect to the frontend version of the website.

[1]: https://github.com/odoo/odoo/commit/31cc10b91dc7762e23b4bde9b945be0c4ce3fe3b

task-2687506

closes odoo/odoo#94580

Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-06-30 18:52:01 +02:00
Younn Olivier 95c8f1a6a8 [FIX] website: fix restricted_editor test
See merge commit for more information.

task-2687506
2022-06-24 10:28:07 +02:00
99b50d18e2 [REF] test_website, *: adapt tours to new client action
*: website, website_blog, website_crm, website_event, website_forum,
   website_hr_recruitment, website_mass_mailing, website_sale,
   website_sale_loyalty, website_slides

This commit adds the current website displayed view xmlid outside of the
iframe, on its container, so that the tours registered with
registerThemeHomepageTour can continue to prepend their triggers with
the view xml id.

Introduce registerEditionTour: this function will allow tour maker to
more easily register a tour that needs to start in the iframe's backend
and go in edit mode.

See merge commit for more information.

task-2687506

Co-authored-by: Benjamin Vray <bvr@odoo.com>
Co-authored-by: Younn Olivier <yol@odoo.com>
2022-06-24 10:28:07 +02:00
Jeremy Kersten a6a83b6d62 [FIX] website: fix / optimize canonical url
Homepage was never considered as canonical due to the trailing /
domain.com/fr/ != domain.com/fr

Now _get_canonical_url_localized for homepage is fixed.

Remove all computation related to canonical that is usless for connected
user. It is mainly used for SEO tools/bots/crawler.

Initially planed for 13.0 with [1] but reverted with [2].

[1]: https://github.com/odoo/odoo/commit/33167b3928c767a08fdc7eedc3e6204aac9cac08
[2]: https://github.com/odoo/odoo/commit/deb23450f18196c7f8d9e819db603fccd407b82a

closes odoo/odoo#94289

X-original-commit: 375abe5e369b59a57b352d68d4835f2ef275b560
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Jérémy Kersten <jke@odoo.com>
2022-06-23 10:14:58 +02:00
Denis Ledoux a177db910d [IMP] website: unit test to cover invalid IP V6 URL
This is a unit test to cover the fix in revision
bfdd54e815

closes odoo/odoo#93170

Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2022-06-09 12:18:51 +02:00
Romain Derie 5acddcd304 [IMP] website, test_website: new standalone tags to ease runbot configs
As we have more and more standalone tests related to the website app, it
has been decided with the runbot team to introduce a unique tag to
easily identify those tests.

The goal is to make the runbot build config easier to manage, without
the need to add the new tag name in the command.
Instead, this build will be fired by finding and testing all the
`website_standalone` tests.
The other tags are kept to easily identify what are the test related to.

Command without this (which need to be edited for each new tag):
```
--standalone cow_views,cow_views_inherit,theme_views,theme_upgrade
```
Now:
```
--standalone website_standalone
```

X-original-commit: a28c181b7f934bed06adccd2678d6c6db47422fe
Part-of: odoo/odoo#93109
2022-06-08 16:50:56 +02:00
Romain Derie d348bed1ad [IMP] website, *: use upsert to improve visitor perf
* im_livechat, test_event_full, website_blog, website_crm,
  website_event, website_event_track, website_event_track_quiz,
  webite_livechat, website_sale

There is 6 main changes in this commit:

1. Using raw SQL Upsert instead of the ORM methods. While raw SQL should
generally be avoided, it makes sense for such a low level behavior which
is impacting every flows.
Indeed, tracking visitors is a generic behavior done on all pages and
controllers. It is important to optimize it to reduce processing time
and SQL Queries.
Benchmark of that change alone:
> Rendering a tracked page improves from ~19.5ms to ~17ms (using `ab`
  with 1000 loop) and the requests involved in the tracking process are
  reduced from 8 SQL Queries to 3:
  - 1 request to upsert the visitor
  - 1 request to fetch the visitor data
  - 1 request to add the tracking record

2. Adding in that upsert query the `visitor.track` insert, creating both
records in one go, bringing the query count from 3 to 2.

3. Refactoring of the `parent_id` behavior that was introduced in stable
with [1]. The purpose was to keep track of multiple visitor linked to a
same user to merge the tracking together. Especially useful for tracking
a same visitor on different devices (when logged in).
Only one visitor was kept as active, others would be archived and their
tracks would be set/moved to the main partner.
Removing those duplicate visitor was not possible because those archived
duplicated visitor were holding the devices notification push token.
Since [2], those token were moved to their own table, all related to the
main visitor.
We can then now safely remove those duplicate visitors after merging
their track to the main visitor. Thus, the `parent_id` field is no more
useful. Removing it removes a layer of complexity.
Note that thanks to this part, the `active` field can also be removed.

4. Deeper functionnal change, inspired from Plausible: The access_token
is no more stored in a cookie but is the result of a hashing method
based on <IP Adress, User Agent>.
The reason behind that change is that, in an upcoming refactoring,
sessions won't be stored anymore unless absolutely needed (login, add to
cart..). It will also ship a no cookies policy, trying to get rid of all
cookies.
This change is bringing some functional changes:
- Since the IP is included in the hash to generate the token, it means
  that:
  A. If an anonymous user switch IP (eg from 4G to wifi), it is
     considered as a new visitor.
  B. If 2 anonymous users with the exact same user agent (same browser,
     same browser version, same exact os or phone) are on the same IP,
     those will be considered as the same visitor.
- Since the request host is not included in the hash, it means that
  visiting a DB from 2 differents URLs (domain and/or ip) on the same
  device and same browser will result in a shared visitor.
  It shouldn't imply any issue as this is A. not wrong and B. mostly
  used for tests.
As all this is only related to non logged in user, it shouldn't be a
real issue as anonymous visitors are not supposed to be meant to be
business critical, even if we use them for "a bit more" than simple
analytics data.

5. The access_token is now replaced by the partner_id once the user logs
in, so:
- We don't need to either search on the partner_id field or the
access_token field (depending if the user is logged in or not), we can
only use the access_token row/field to do both.
- On logout, everything works out of the box as the access_token will be
regenerated since there is no partner_id anymore.
- On login, if an access_token matches the user's partner_id, that
visitor is returned.
If there is no such token, a new visitor is created for that partner_id.
In both 2 cases, tracks are moved to that visitor and the anonymous
visitor is removed.
- We can remove the code that was in charge of checking if the
access_token / visitor cookie was wrong (coming from another user eg,
different user login on same device). Indeed, such collision is not
possible anymore as the access_token automatically match the logged in
user.
- We can remove the code that was in charge of checking if the
access_token / visitor cookie was wrong (coming from a logged in user
while the current visitor is not loggedin). Such collision is not
possible anymore as the access_token is (re)generated as an anonymous
token (hash) when not logged in.

6. There is no more check to prevent a track to be created if there was
already a track for that URL in the last 30 minutes.
While this can easily be re-introduced (one CTE on the upsert), it was
adding ~100ms (from ~20 to ~110ms) to the request on a big database as
Odoo where there is ~100 millions tracks and ~100 millions visitors.
It has been validated that it was not a real issue as it is not
fundamentally wrong. If a visitor visited 20 times a product or a
specific page in that short amount of time, you might want to know that
because the user is most likely interested by it.

Changes (1+2), 3, (4+5) and 6 are all independant from each other and
could have existed on their own.

[1]: https://github.com/odoo/odoo/commit/c6b8a44b970a46dcd87a4e2cb1ad52fa340b209f
[2]: https://github.com/odoo/enterprise/pull/16781/commits/f75090fe8b42484e89e933976e8441d2f5eb9415

task-2867045

closes odoo/odoo#87857

Related: odoo/enterprise#28004
Related: odoo/upgrade#3566
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-06-07 16:31:20 +02:00
Gorash 0452d0701f [IMP] website: remove cache from website.page
The cache placed on the pages is no longer useful thanks to the use of
the new directive t-cache.

closes odoo/odoo#88276

Related: odoo/enterprise#27582
Related: odoo/documentation#2056
Related: odoo/upgrade#3451
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
2022-06-03 16:40:40 +02:00
Gorash b0a2a41d78 [IMP] all/website: using lazy values and t-cache in templates
Using lazy values allows you to not do query when the content is not
displayed or if it is already cached. Adding `t-cache` only reduces
render time, but combining it with lazy values saves browses, computes
and query.
Thus for the `/shop` page the page is displayed in 60ms (before: 250ms).

Part-of: odoo/odoo#88276
2022-06-03 16:40:40 +02:00
Julien Castiaux da8def8e41 [IMP] core, web: Delegate delivery of static files
Rationnals
----------

Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.

Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.

X-Sendfile
----------

In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.

Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.

Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:

    location /web/filestore {  # custom path, hardcoded within Odoo
        # Prevent access from the outside world, i.e. makes this
        # route only accessible via X-Accel. MANDATORY!!!
        internal;

        # Give access to the filestore using this server's
        # permissions. Odoo is in charge of verifying the access
        # rights.
        alias /path/to/odoo/data-dir/filestore;
    }

The Odoo [deployment documentation] has been updated accordingly.

[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments

Changes to the API
------------------

To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.

Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.

I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.

A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.

Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:

- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`

The new `ir.binary` abstract model exposes the following utilities:

**`_find_record`**

Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.

**`_get_stream_from`**

Create a Stream from an attachment or a record with a binary-field.

**`_get_image_stream_from`**

Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.

**`_placeholder`**

Get the image placeholder blob.

Testing
-------

It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.

    odoo-bin -i test_http --stop-after-init
    WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init

closes odoo/odoo#88134

Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
2022-06-01 02:53:59 +02:00
Raphael Collet 6cf8db906f [REF] *: adapt code to new flush API
closes odoo/odoo#87527

Related: odoo/upgrade#3497
Related: odoo/enterprise#26939
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-05-25 18:00:47 +02:00
Nicolas Bayet 36e80e02a8 [FIX] web_editor: fetch undraw images in website and note
task-2794153

closes odoo/odoo#91970

X-original-commit: 536b2ff35f28836ef14cbb353b4749adee21aaad
Signed-off-by: David Monjoie (dmo) <dmo@odoo.com>
2022-05-20 20:14:23 +02:00
Benoit Socias 254421b5d8 [FIX] website: increase stability of ace test tour again
This is an attempt at fixing a runbot "race" condition.
A first attempt was done in [1] but the selector introduced to make
sure the page was reloaded after the assets were changed was wrong.

This commit fixes that selector.

During the investigation of the problem, it was also discovered that the
SCSS change could take quite a long time on a busy server becasue it
triggers the SCSS compilation each time.

To reproduce that problem locally, run the `stress` command while
executing the `test_html_editor_scss` test.
E.g.:
```sh
$ stress --cpu 32 --timeout 120
```

The error that shows up in that case is misleading because it makes it
look like the confirmation popup was not clicked on.
What is actually happening is the following:
- the Reset button is clicked on
=> the confirmation popup is shown
- the confirmation is clicked
=> the popup is closed and the RPC call is made
- the RPC does not respond within the tour step's timeout
=> the last step is not shown as succesful and the screenshot shows the
non-reset state since the server response did not arrive yet

To be safe on busy servers this commit also increases the timeouts
related to steps that follow updates of SCSS files so that they can be
recompiled on time.

[1]: https://github.com/odoo/odoo/commit/b35f127c86c1d271a76dae7186a96190e8558c73

runbot-3744

closes odoo/odoo#91725

X-original-commit: e04c4f06086ff4287ef4acfc65d5b3d957396ae7
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Benoit Socias (bso) <bso@odoo.com>
2022-05-19 11:13:55 +02:00
qsm-odooandIvan Yelizariev b246def6d5 [FIX] website: consider configured email_to on the contact us page
Configuring the email_to field on the contact us page was not working
anymore. The email was always sent to the company email whatever the
user chooses in edit mode. This was due to [1] which gave the priority
to the data-for and prefill system over default values. While this makes
sense in general (e.g. if the URL on the contact us page contains
?name=John, then the form is prefill with "John" whatever the value that
is set as default), it does not make sense for the email_to field which
is always prefilled with the company email.

The data-for/prefill system priority probably has to be reviewed in the
future. Meanwhile, this commit will make the email_to field as an
exception for this system and only use the data-for value if nothing
has been configured by the user.

This commit also handles another case. Indeed "if nothing has been
configured by the users" is currently kinda broken as just clicking on
the form and saving will force the value "info@yourcompany.example.com"
as the email_to value... thus making the form use it instead of the
company email. We will make that as an exception of the exception: use
the data-for value (company email) if what was configured by the user is
the dummy default value "info@yourcompany.example.com".

A test for each of those two exceptions has been added (only the first
test breaks before this commit as [1] made the second test pass by
chance).

[1]: https://github.com/odoo/odoo/commit/7b23d3aacd22f87cb0c22ecd0478eba4a17b4bf3

opw-2842211

Original idea for the first exception:

closes odoo/odoo#91407

X-original-commit: a08574a04ff2ce8ea2d6fd2b79f56cc57bba953b
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: Ivan Yelizariev <iel@odoo.com>
2022-05-16 12:47:56 +02:00
Paul Morelle ece83df99a [FIX] website,http: make EndPoint object reusable
It may happen than the routing map is cleared while rendering a qweb
view. For example, if an asset bundle is regenerated, the previous one
is unlinked, which causes a cache clearing.

Previously-generated EndPoint objects aren't found any more in the new
routing map, so `request.endpoint` cannot be used any more after a cache
clearing.

This commit adds hash and comparison magic methods on http.EndPoint so
that EndPoint objects created by a previous routing map generation can
still be used after a cache clearing.

Two tests were added: one to test the comparison and hash methods, and
the other to test them in a real-case rendering.

Commit 80a04f7ebed fixed this bug too, but introduced another issue
which caused many OPW, so it was quickly reverted by deb23450f18 along
with its performance improvement 33167b3928c. This commit replaces
80a04f7ebed with another way to fix the issue.
A third test has been added in order to avoid reintroducing this other
issue.

By the way, this commit also removes the EndPoint.arguments attribute,
as commit 17f1992698 removed all usages of this attribute in Odoo 9.0,
but left this initialization here.

OPW-2834546
OPW-2834549
OPW-2834625

X-original-commit: bdc45422f5c715782b147eecda2403386ba7eb4a
Part-of: odoo/odoo#91034
2022-05-12 13:42:57 +02:00
Romain Derieandroen-odoo b8bd897fe4 [FIX] website: allow user to access HTML/CSS editor
Current behavior:
When an admin user modified a file with the HTML/CSS/JS editor in a
website, users that had "editor and designer" right couldn't access
the HTML/CSS/JS editor anymore.

Steps to reproduce:
- Have user 1 (U1) admin
- Have user 2 (U2) with website access set to "Editor and Designer"
- U1 modify the css user_custom_rules.scss with the HTML/CSS Editor
- U2 can't open the editor of that website because he is not in the
setting groups or the one that created it

opw-2733109

X-original-commit: d20468c3463927a54c34688769a22b3949c5c465
Part-of: odoo/odoo#89211
Co-authored-by: roen-odoo <roen@odoo.com>
2022-04-20 22:56:11 +02:00
Benoit Socias e9ea0692bb [FIX] website: not split fuzzy matching candidates on hyphen
Since [1] when the fuzzy search was introduced, the candidate words used
for the fuzzy matching were split on non-'\w' (all non alphanumeric
characters + underscore).

This commit adds '-' (hyphen) to the list of characters that should be
considered part of the same word.

Steps to reproduce:
- Specify a product with name 'micro-vis'
- search for 'micro-vis' or 'microvis'
=> Was not returning any match
=> Now returns fuzzy match even for 'micro-vs'

opw-2801704

[1]: https://github.com/odoo/odoo/commit/c6ba756e4b4704089d02434871788a82d90cb195

closes odoo/odoo#88591

X-original-commit: 10d54d3c6ba46afa86abac5275a02c4b5ec7d5a1
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-04-13 11:55:37 +02:00
Benjamin Vray 53717c27d7 [FIX] web_editor, website, mass_mailing: fix snippets menu tabs
Before this commit, this bug was present in edit mode:

- Go into edit mode on an empty page.
- Click on the THEME tab on the right.
- Click on the empty "DRAG BUILDING BLOCKS HERE" area.
- Click on the THEME tab again.
- The content of THEME is empty.

It was because since this commit [1], clicking on an 'oeStructure'
activate the Blocks tab but without updating the options in the tabs.

Another similar bug was present when clicking the style tab and then
clicking on the theme tab.

About the first bug, actually a call of '_updateRightPanelContent' is
already done in '_activateSnippet()' (and the Blocks tab is activate by
default) so we only have to use '_activateSnippet(false)' to activate
the Blocks tab. This commit also remove '_updateRightPanelContent' from
the click event of the Blocks tab where it was not necessary.

For the second bug, we can't do the same fix, because it's not possible
to activate the options tab with only '_activateSnippet(false)' so we
had to combine the two functions.

[1]: https://github.com/odoo/odoo/commit/fbe048c202ff7878984adf26bef0651a45851f0a

task-2794266

closes odoo/odoo#88537

X-original-commit: 52d32e26c34bef10823582ef5c1f840492b73d7a
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2022-04-12 15:49:06 +02:00
Romain Derie 5a8d6c351b [IMP] website, test_website: avoid a query on res_company_user_rel
Manual forward port, commit was lost from 15.2 to master. Either it
seemed not needed after the httpocalypse refactoring, or was simply
lost during the rebase.

-----

We can remove an useless SQL Query for public user on every single page
serve, if we directly set the website's company_id in the available
company_ids of the public user.

Indeed, the public user from a website always have the same company_id
as the website.

We can then do that directly and bypass the next line which will
always be true in the case of the public user, making a read for
nothing.

task-2774979

closes #85419

X-original-commit: c95ab6f

[1]: https://github.com/odoo/odoo/commit/728ade674cb12c64718efda56b0d5d542e319cba

closes odoo/odoo#87856

Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-04-06 19:32:44 +02:00
Romain Derie aa1c1b0bcb [FIX] website, *: correct wrong query count after httpocalypse
*: test_website, website_blog (test perfs)

An error regarding tests was not catch during httpocalypse [1] review.
The test query counts were lowered in website by 2 everywhere, but it
shouldn't have as there were no SQL perf improvement from this
refactoring regarding website.
The query count actually decreased because the test-only SQL Savepoints
got divided by 2, probably thanks to a low level test improvement in
the PR.
But outside of tests, there were no SQL Query gained.
This should have been reflected in the `EXTRA_REQUEST` variable
instead.

Before:
Savepoint > Rollback > Savepoint > [Actual SQL Queries] > Rollback
After:
Savepoint > [Actual SQL Queries] > Rollback

It is important to fix it because:
1. (Minor) There is a de-sync between the query count in the test and
   the actual query count (in the logs outside of tests), making it
   hard to figure.
2. (Major) Some controller got then wrongly lowered to 0 query counts
   instead of 2. An incoming PR in master would then make those tests
   expects negative query counts, which doesn't make any sense.

[1]: https://github.com/odoo/odoo/pull/78857
Note that the exact commit can't be found by bisect as those don't work
on their own, DB can't be started due to circular import.

Part-of: odoo/odoo#87856
2022-04-06 19:32:44 +02:00
Julien Castiaux 04e972660b [IMP] core: don't save visitor default session
Every request comes with a session, a dictionary that is persisted on
the filesystem and that saves various information such as the user
cart on the ecommerce.

When a user simply visits the website, a default session is created and
saved on disk, this bloats the filestore with many sessions. Creating
the session on-the-fly is cheaper than loading it from the filesystem.
With this work the default session is not saved on disk anymore unless
explicitly asked via `session.touch()`.

An exception to the statement "creating the session on-the-fly is
cheaper" is geoip, the ip geolocalization is not cheap. In this work,
geoip have been moved from http_routing/request.session.geoip to a
lazy property core/request.geoip. When requested the info is persisted
on the session. Like other keys from the default session, geoip will not
be persisted unless there is non-default stuff in the session.

Because the CSRF-TOKEN is based on the session-id, it is important the
session-id stays the same across multiples requests even when the
session is not persisted on disk. Even when a session is not persisted
on disk, the session-id cookie is still set so that the next session
created on-the-fly uses the same session-id.

Technical note regarding the session, it has been decided to drop the
session-snapshot protocol and to reintroduce a "modified" flag. It has
been decided not to use werkzeug's session (which natively comes with a
"modified" flag) and to keep our own session object. We decided to
extend MutableMapping instead of dict; using MutableMapping we only
have to override __setitem__ and __detitem__; using dict we would had to
override update()/pop()/... too.

Task: 2789035
Part-of: odoo/odoo#86015
2022-04-05 14:13:54 +02:00
Martin Trigaux f41ec39abe [IMP] base: get_view_id becomes private
Add new helper get_view
Remove dangerous documented comment

Part-of: odoo/odoo#85110
2022-03-29 10:56:16 +02:00
Gorash 880954ebfc [IMP] *: remove _render from ir.ui.view and simplify report
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.

The report rendering and call `ir.qweb` instead of `ir.ui.view`.

Part-of: odoo/odoo#85110
2022-03-29 10:56:15 +02:00
Fabio Barbero 47041f2d45 [IMP] base: allow user to activate multiple languages at once
Purpose
=======
Add "Activate" button when selecting multiple languages, select multiple
languages when clicking on "Add languages" in settings.

Specifications
=============
`lang` variable in `base.language.install` changed from Selection to
Many2many to allow multiple languages being activated at once.
Hide globe icon for language fields from view mode (only visible when
editing). Remove state in base.language.install since it's no longer
need to keep track of the installation step.

Task-2662548

closes odoo/odoo#78287

Related: odoo/upgrade#2921
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2022-03-14 08:42:58 +00:00
Rémy Voet (ryv) d2b68d186e [FIX] website: remove force prefetch for translate fields
Issue
-----
When website is installed, the rendering of template uses a side effect
of the ORM cache (cache shared between sudoed env vs non-sudoed env) and
the fields prefetching feature to work correctly.

The `self.visibility` in (`_handle_visibility`, website/ir_ui_view.py)
is done in sudo mode, then it will fetch all prefetchable fields and put
them in the cache (that will be read in non-sudo mode in the render of
the template).  Another example of issue related to this:
https://github.com/odoo/odoo/pull/83341.

Because of this, the fields of mixin `website.seo.metadata` were forced
to be prefetchable (the default for translate is to be not prefetchable
since https://github.com/odoo/odoo/pull/82896), which causes a useless
LEFT JOIN on "ir_translation" in most of business flow.

Fix
---
Remove the `prefetch=True` on mixin fields, and add a extra read to fill
the cache in case of website rendering.  It also allows to read these
fields at the same time.

Part-of: odoo/odoo#85220
2022-03-03 11:03:55 +00:00
Romain Derie 826c0a9119 [IMP] website: fetch all pages and views at once
When serving a page, fetch the requested page and the menu's pages in
one go.

This will only impact pages that do not have a menu entry, which is the
most common form of pages.

So, instead of:
- read page
- read page's view
and later:
- read menu pages
- read menu pages' views

Fetch everything together:
- read served page + menu pages
- read served page's menu + menu pages' views

task-2774979

closes odoo/odoo#85456

X-original-commit: 16bf7cca8e8d452ce9131d9d86ff5b23789da4e1
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-02-28 13:53:20 +00:00
Romain Derie a61ba018a7 [IMP] website: avoid query on website table when serving empty page
Getting the `res.company` record by browsing the `company_id` cached
value from `website` instead of reading it directly on the website
record will avoid a read on the website table.

While this might seems unusual and not elegant, since this is a very
low level method used to render every view, it seems fine.

task-2774979

X-original-commit: bc643a573a7339c82661cbcff2cc59bdd0eec3e3
Part-of: odoo/odoo#85456
2022-02-28 13:53:19 +00:00
Romain Derie 28b9a6490c [IMP] website: don't query website table when serving homepage
We can easily avoid a read on the website table when serving the
homepage by simply caching the website.homepage_id value and then
browsing the record (no query needed) instead of reading it on the
website record.

While this code is not really elegant, it seems like a good tradeoff to
gain a query on this particular homepage serve controller, which is the
main entry point of a website and generally the page the more often
served.

task-2774979

X-original-commit: 69b443b85656c895c66a1f83b235f5d1e53727d4
Part-of: odoo/odoo#85456
2022-02-28 13:53:19 +00:00
Romain Derie 3e2d30fdbd [IMP] models.py, *: don't read ir_translation table if empty source
* website, website_blog (perf)

If the source values is only composed of empty elements (list of `None`
values), don't try to translate it, as there is nothing to translate
anyway.

While the translation method being called (xml_translate,
html_translate) won't be costly as they will return instantly if the
provided value is falsy, there is still an SQL Query made to get the
callable translation method, which we won't actually use.

This commit avoids that query if we already know we won't do anything
with the translation method.

task-2774979

X-original-commit: e1eb5a6e1610a9550a59ba229a83b9e70e10aa88
Part-of: odoo/odoo#85456
2022-02-28 13:53:18 +00:00