This module allows to add secret key to add the turnstile captcha on
each snippet website_form.
Cloudflare Turnstile
--------------------
A friendly, free CAPTCHA replacement
Turnstile delivers frustration-free, CAPTCHA-free web experiences to
website visitors.
Turnstile stops abuse and confirms visitors are real without the data
privacy concerns or awful UX that CAPTCHAs thrust on users.
closesodoo/odoo#119246
X-original-commit: 4aca39a533e9d41f5f452f36a1ffc001f586b4f4
Signed-off-by: Jérémy Kersten <jke@odoo.com>
Before this commit, if a link to a page was not correct because of a
case mismatch, it would simply land on a 404 page.
While it's correct, as URL are case sensitive, it leads to a few bad UX
flow at the admin/editor level:
- Create a link in your page (on a text or a button eg), type an URL
which does not exists (to create it after) like /Page
- Click on the link/button you just made, you are redirected to /Page
which display a 404 with the "Create page" option (correct)
- When you click on that button, it will actually create a page with
/page URL, leading to a mismatch between the URL you created and the
page URL.
Your link/button will still lead to a 404 URL as it points to /Page.
Since it's just a fallback when an exact URL match is not found, it
should not break anything and should not have bad impact at any level
(seo/speed etc).
Indeed:
- It's done through a 302 redirect
- `_serve_page()` is already a fallback case, so it will only make
the `website.redirect` and 404 cases a bit slower due to the extra
search query.
The only possible scenario seems to be if the user (mind the uppercase):
- Created a /Page page
- Created a redirect from /page to /another-page
In this case, /page won't land on /another-page but on /Page.
This flow seems unlikely and is not actually wrong either way.
At least, it certainly is less important than ensuring a case
insensitive fallback.
Finally, note that another solution would have been to either:
- Force page URL to lower case.
-> This is not stable friendly, people might be relying on this to
create pages with different casing:
`/Batman-VII-The-Dark-Knight-Whatevers`, while not recommended,
doesn't sounds idiot.
On top of not being stable friendly, we probably want to keep
offering this possibility
- Redirect all URLs to lowercase endpoints.
-> This is obviously not stable and not Odoo's jobs. It should be
something decided by the sysadmin and done at nginx (etc) level.
task-3110294
opw-3104030
closesodoo/odoo#111736
X-original-commit: f05491105f93939490cbeb078cb7653c38685644
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
request.geoip is no more a dictionnary cached in the session. It is now
a full blown object with lazy and smart geolocalisation capabilities.
Among other things, the previous dictionnary API is now deprecated. The
changes are:
* `request.geoip['country_name']` -> `request.geoip.country_name`
* `request.geoip['country_code']` -> `request.geoip.country_code`
* `request.geoip['city']` -> `request.geoip.city.name`
* `request.geoip['latitude']` -> `request.geoip.location.latitude`
* `request.geoip['longitude']` -> `request.geoip.location.longitude`
* `request.geoip['region']` -> `(request.geoip.subdivisions[0].iso_code if request.geoip.subdivisions else None)`
* `request.geoip['time_zone']` -> `request.geoip.location.time_zone`
It is safe to access all the attributes. Doing `request.geoip.city.name`
when the geolocalization failed (missing db, invalid address, ...)
evaluates to None. It does not raise an AttributeError.
Task: 2848206
Part-of: odoo/odoo#91337
One can very well select a "auth=user route" as homepage for his
website, like /my.
There would then be an issue with such an URL being set as homepage:
- As the user landed in the homepage controller which is auth=public,
the system will add the public user as env user (see
`_auth_method_public()`).
```
@http.route('/', type='http', auth="public", website=True, sitemap=True)
def index(self, **kw):
```
- Then, that controller will reroute to the homepage url (/my). The
request.httprequest.path will now be /my
- Then, that controller will recall the dispatcher stack:
`request._serve_ir_http()`
- From there, this call won't fail as it should because the user is
considered as logged in as it went already through the
`_auth_method_public()`, adding the public user as env.user.
`_auth_method_user()` won't raise its error.
- The /my page will be rendered despite not being logged in.
Once the user land on that page, the system will actually detect him as
logged out on a page supposed to be accessed when logged in.
It will then:
1. Show a toaster to inform the user
2. Auto reload the page
As the current URL is still `/` (due to the reroute and not a redirect),
the same flow will happen again, and again, looping forever.
--------
Note that accessing /my directly won't be an issue, as it won't go
through the homepage controller (which is auth=public), so there won't
be a user_id set on the env (the public user), meaning that the dispatch
layer will reject the access and raise an access error (through
`_auth_method_user()`.
Also note that the same behavior will occur when going through the first
menu fallback mechanism:
- if the user didn't setup any homepage_url
- and deleted his / website.page
- and his first website menu is /my
In that case, it will go through the first menu redirect fallback, which
will be working fine (as it's a redirect and not a reroute).
With both those 2 flows (going through redirect), the user will
correctly land on the login page (which will redirect to /my once logged
in).
------
Finally, the other solution would be to prevent such a configuration
(/my as homepage_url) but it was not easily doable (if doable at all),
see https://github.com/odoo/odoo/pull/99100#discussion_r963055251
------
A test is also added and over the more complexe case (which should cover
everything):
- With /my as homepage_url
- With the / website.page deleted
- With /my as first menu URL
-> Accessing / as public user should:
1. Reroute to /my (because of the homepage_url set to it) which
should fail now thanks to this commit
2. Since the reroute / re-serve failed, it should reach the
"first menu fallback" mechanism, which is also /my
3. That fallback should be a redirect, not a reroute, so the user
should actually land on the login page
4. Once logged in on that page, it should properly redirect to /my
opw-3077339
X-original-commit: 4f5899f769266edc27a0b2d812df092b95f43c71
Part-of: odoo/odoo#107759
With commit [1] we refactored the cookies bar to let the user decide if
he wants to accept the cookies (and/or only part of it).
Before that commit, the `website_cookies_bar` cookie could only hold
`true` as value, which now is holding an object like
`{"required": true, "optional": false}`.
This creates an issue if a user is coming from a previous version with
`true` as cookie value because since the refactoring it will crash both
in JS and PY because `in` instruction with a boolean value will fail in
both languages.
The decision taken here is simply to remove the cookie if we face such a
case so the user can decide again what he wants (since there is more
choices now).
It also means that we won't be holding an outdated value in the cookie
any longer.
[1]: https://github.com/odoo/odoo/commit/2cbda6c98ee947cea1d06c09880eee8c758304a8
opw-3074303
closesodoo/odoo#106621
X-original-commit: 22392755ed4c504e6ba2a5a9c9eb6643f2c95994
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
This fix allows generating the sitemap (via `website.search_pages`) via
RPC. The changes are necessary as since #99667 the `request` object is
no more available to RPC-executed functions.
See also: the `test_search` test case of `website.tests.test_page`.
closesodoo/odoo#104567
X-original-commit: 8f4e214417cfb1aed7e595b8d6ab71f33a347b29
Signed-off-by: Julien Castiaux <juc@odoo.com>
*: im_livechat, survey, utm, website_crm_iap_reveal, website_forum,
website_livechat, website_sale, website_sale_comparison
Before this commit all cookies were considered essential.
This commit makes some of them optional. It also makes it possible for
the website visitor to only accept the essential cookies.
task-2800976
X-original-commit: 9a8a9463289a7446e9be0ef62ff895feb37a4de4
Part-of: odoo/odoo#101845
Co-authored-by: Benoit Socias <bso@odoo.com>
XML files are now declared in python module manifests. During the qweb
't-call-asset' directive, assetbundle will fetch the declared xml files,
apply the inheritance (t-inherit) and create a javascript service (for
eg: 'web.assets_backend.bundle.xml') which is added at the end of the
*.js mimifier file.
When the debug mode is activated, comments are added in the template
indicating which file the template comes from as well as the
inheritances applied to it.
****
JavaScript:
assets.js (module @web/core/assets) takes care of loading libraries,
javascripts and styles.
`loadJS(url)` (loads the javascript and returns a resolved promise when
the templates are also loaded via the '*.bundle.xml' service)
`loadCSS(url)` (loads the style a resolved promise when the file is
loaded)
`loadXML(xml, app=assets.defaultApp)` (load template into
application/owl, used by the `*.bundle.xml` services)
`getBundle(bundleName)` (get the bundle descriptor)
`loadBundle(desc)` (load the files and bundle from a descriptor)
templates (XML element content all owl templates)
A new `ready(serviceName)` method on boot.js lets you know when a
service is loaded are the require.
The xmlDependencies attribute no longer exists.
Python:
The xmls taken into account by assetbundle.py, applying `t-inherit`
inheritances and adding an `name_of_the_bundle.bundle.xml` service in
the generated JavaScript file.
****
Every manifest changes is into the next commit, except 'web_tour' in
this current commit as example.
Part-of: odoo/odoo#95500
* website_event, website_slides
Note that `restricted_editor` is the new technical name used instead of
`publisher`
Before this commit, the `is_restricted_editor()` method was actually
checking for `designer` rights and not `restricted editor` ones.
Indeed, a restricted editor (previously publisher) never had the write
right on ir.ui.view.
Despite the method name being wrong, it was still correctly used where
we actually checked for designer rights and not publisher/restricted
editor ones.
Also, the method was checking for write access on ir.ui.view instead of
checking directly if the user had the related group.
While it wasn't really wrong, it was also including people who would
have the write right on ir.ui.view but were not part of the designer
group.
This doesn't really makes sens, as we don't want someone which received
that write right on ir.ui.view from another module (not related to the
website) to be considered as a website admin -> we don't want to show
them website admin UI and stuff like that.
The method could have simply be renamed to `is_designer()` and could
have just checked for `has_group(designer)`, but having such an helper
seems overkill, we would end up with as many utils as there is groups.
At the end, this PR:
- removes the confusing util method
- replace it by has_group checks
Part-of: odoo/odoo#98200
* portal, web_unsplash, website_*
This commit renames the `website.group_website_publisher` into
`website.group_website_restricted_editor`.
While the change in itself might look unuseful, it will help the dev and
tech community figuring which group is related to which feature.
Even internally when we discuss specs, we always have to remind which
group is the restricted editor: the publisher one or the designer one?
While it is probably, after all those years, now anchored in some dev
mind, there is no easy way to directly figure which of those 2 groups is
the restricted editor one.
Note that I myself always got confused about it.
Now, the "Restricted Editor" right will be reflected in its technical
name `group_website_restricted_editor`.
Same as for the "Editor & Designer" which technical name is
`group_website_designer`.
As we would like to have a fully working and ready system in v17 for the
community to be able to build themes easily, removing that dubious part
is a nice to have.
Part-of: odoo/odoo#98200
Just a nice to have. It will prevent those errors to be replicated when
copy pasted and will help reading the files in the IDE.
closesodoo/odoo#97282
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
* im_livechat, test_event_full, website_blog, website_crm,
website_event, website_event_track, website_event_track_quiz,
webite_livechat, website_sale
There is 6 main changes in this commit:
1. Using raw SQL Upsert instead of the ORM methods. While raw SQL should
generally be avoided, it makes sense for such a low level behavior which
is impacting every flows.
Indeed, tracking visitors is a generic behavior done on all pages and
controllers. It is important to optimize it to reduce processing time
and SQL Queries.
Benchmark of that change alone:
> Rendering a tracked page improves from ~19.5ms to ~17ms (using `ab`
with 1000 loop) and the requests involved in the tracking process are
reduced from 8 SQL Queries to 3:
- 1 request to upsert the visitor
- 1 request to fetch the visitor data
- 1 request to add the tracking record
2. Adding in that upsert query the `visitor.track` insert, creating both
records in one go, bringing the query count from 3 to 2.
3. Refactoring of the `parent_id` behavior that was introduced in stable
with [1]. The purpose was to keep track of multiple visitor linked to a
same user to merge the tracking together. Especially useful for tracking
a same visitor on different devices (when logged in).
Only one visitor was kept as active, others would be archived and their
tracks would be set/moved to the main partner.
Removing those duplicate visitor was not possible because those archived
duplicated visitor were holding the devices notification push token.
Since [2], those token were moved to their own table, all related to the
main visitor.
We can then now safely remove those duplicate visitors after merging
their track to the main visitor. Thus, the `parent_id` field is no more
useful. Removing it removes a layer of complexity.
Note that thanks to this part, the `active` field can also be removed.
4. Deeper functionnal change, inspired from Plausible: The access_token
is no more stored in a cookie but is the result of a hashing method
based on <IP Adress, User Agent>.
The reason behind that change is that, in an upcoming refactoring,
sessions won't be stored anymore unless absolutely needed (login, add to
cart..). It will also ship a no cookies policy, trying to get rid of all
cookies.
This change is bringing some functional changes:
- Since the IP is included in the hash to generate the token, it means
that:
A. If an anonymous user switch IP (eg from 4G to wifi), it is
considered as a new visitor.
B. If 2 anonymous users with the exact same user agent (same browser,
same browser version, same exact os or phone) are on the same IP,
those will be considered as the same visitor.
- Since the request host is not included in the hash, it means that
visiting a DB from 2 differents URLs (domain and/or ip) on the same
device and same browser will result in a shared visitor.
It shouldn't imply any issue as this is A. not wrong and B. mostly
used for tests.
As all this is only related to non logged in user, it shouldn't be a
real issue as anonymous visitors are not supposed to be meant to be
business critical, even if we use them for "a bit more" than simple
analytics data.
5. The access_token is now replaced by the partner_id once the user logs
in, so:
- We don't need to either search on the partner_id field or the
access_token field (depending if the user is logged in or not), we can
only use the access_token row/field to do both.
- On logout, everything works out of the box as the access_token will be
regenerated since there is no partner_id anymore.
- On login, if an access_token matches the user's partner_id, that
visitor is returned.
If there is no such token, a new visitor is created for that partner_id.
In both 2 cases, tracks are moved to that visitor and the anonymous
visitor is removed.
- We can remove the code that was in charge of checking if the
access_token / visitor cookie was wrong (coming from another user eg,
different user login on same device). Indeed, such collision is not
possible anymore as the access_token automatically match the logged in
user.
- We can remove the code that was in charge of checking if the
access_token / visitor cookie was wrong (coming from a logged in user
while the current visitor is not loggedin). Such collision is not
possible anymore as the access_token is (re)generated as an anonymous
token (hash) when not logged in.
6. There is no more check to prevent a track to be created if there was
already a track for that URL in the last 30 minutes.
While this can easily be re-introduced (one CTE on the upsert), it was
adding ~100ms (from ~20 to ~110ms) to the request on a big database as
Odoo where there is ~100 millions tracks and ~100 millions visitors.
It has been validated that it was not a real issue as it is not
fundamentally wrong. If a visitor visited 20 times a product or a
specific page in that short amount of time, you might want to know that
because the user is most likely interested by it.
Changes (1+2), 3, (4+5) and 6 are all independant from each other and
could have existed on their own.
[1]: https://github.com/odoo/odoo/commit/c6b8a44b970a46dcd87a4e2cb1ad52fa340b209f
[2]: https://github.com/odoo/enterprise/pull/16781/commits/f75090fe8b42484e89e933976e8441d2f5eb9415
task-2867045
closesodoo/odoo#87857
Related: odoo/enterprise#28004
Related: odoo/upgrade#3566
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Before this commit, every visit on the website would create a visitor
and tracks, including visits from bots and crawlers (e.g. google bot).
Now, we prevent that as it doesn't bring any value to track those bots.
Note that it means there won't be any track created if you are using
tools like `ab` for perf investigation or `curl` commands.
task-2867045
Part-of: odoo/odoo#87857
The `t-cache` directive allows you to keep the rendered result
of a template part. The supplied key must be a tuple. This tuple
can contain recordset in this case the zone will be invalidated
each time the write_date of these records changes.
The `t-nocache` directive makes it possible to force rendering
of a part even if it is in a `t-cache`. The values available in
the `t-nocache` are the one provided when calling the template
(and therefore ignores any t-set that could have been done).
Part-of: odoo/odoo#88276
Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closesodoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
In this commit we add a request header X-Disable-Tracking that enables the
client to ask the server not to track a request.
This is useful for example when the client prefetches pages to be able to work
offline. In that case, the prefetched pages must not be considered as viewed
by the user. In such scenario and without such header, it is not possible to
get an accurate page tracking.
Task-2476513
Part-of: odoo/odoo#86031
Manual forward port, commit was lost from 15.2 to master. Either it
seemed not needed after the httpocalypse refactoring, or was simply
lost during the rebase.
-----
We can remove an useless SQL Query for public user on every single page
serve, if we directly set the website's company_id in the available
company_ids of the public user.
Indeed, the public user from a website always have the same company_id
as the website.
We can then do that directly and bypass the next line which will
always be true in the case of the public user, making a read for
nothing.
task-2774979
closes#85419
X-original-commit: c95ab6f
[1]: https://github.com/odoo/odoo/commit/728ade674cb12c64718efda56b0d5d542e319cbaclosesodoo/odoo#87856
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Commit "[IMP] core: don't save visitor default session" moved the geoip
from the session to the request with a deprecation warning. This commit
adapts the remaining modules to use `request.geoip` instead of
`request.session.geoip`.
Geoip is always set on the request but it can be an empty dictionnary in
case the geolocalization failed.
closesodoo/odoo#86015
Task: 2789035
Related: odoo/enterprise#25192
Signed-off-by: Julien Castiaux <juc@odoo.com>
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.
The report rendering and call `ir.qweb` instead of `ir.ui.view`.
Part-of: odoo/odoo#85110
When serving a page, fetch the requested page and the menu's pages in
one go.
This will only impact pages that do not have a menu entry, which is the
most common form of pages.
So, instead of:
- read page
- read page's view
and later:
- read menu pages
- read menu pages' views
Fetch everything together:
- read served page + menu pages
- read served page's menu + menu pages' views
task-2774979
closesodoo/odoo#85456
X-original-commit: 16bf7cca8e8d452ce9131d9d86ff5b23789da4e1
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Install website, create a custom web page, we'll call it page_1. Ensure
you are not in debug mode (go to /web/health?debug=0 to disable it).
Open the web page enabling the debug-mode /page_1?debug=1, the page
opens but the debug mode is disabled.
Because web pages are served using another routing mechanism than
controlers we have to ensure we load the debug query-string in those
mechanisms too.
closesodoo/odoo#85340
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
This commit is the 13th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
Here be dragons.
First and foremost, `http_routing` is a technical module that aim to
provide the minimum viable compatibility code between portal and
website. Its primary job is to take care of the lang inserted in the
path of URLs, e.g. `en` in `/en/my_blog`. Both when routing a request
with a lang in the URL and when rendering templates with multilang
support.
Next to `http_routing` is website, the module used by customers to
create pretty web page accessible online. Website uses a different
routing logic than the backend, webpages are **not** registered in the
routing map of werkzeug but instead delivered by website dedicated
code. It means that **all** request targeting a website page thrown at
the werkzeug router **fail** with a HTTP 404 error. The reality is that
website abuses the fallback mechanism (`_serve_fallback`) to deliver
its pages.
Using `_serve_fallback` as the standard way to deliver pages is broken
by design. It is the least crappy way to deliver content as long as
website page don't have a dedicated path prefix. Using a path prefix
(e.g. `p` in `/p/fr/mon_blog`) it would have been possible to route the
request to a dedicated endpoint using the same router as the backend and
with no change to the HTTP dispatching code. Sadly, the business does
not want such prefix so we have to stick with a broken design.
It is broken because prior to serving a page, website needs to setup A
LOT of stuff on the system. It needs to ensure a proper user is set on
the environment, it needs to setup the GeoIP database, it also needs
to determine the lang the user requested the page and it also needs to
save multiple attributes on the request objet itself (`is_frontend`,
`is_frontend_multilang`, `routing_iteration`, `website`, `lang`,
`rerouting` and `website_routing`). Since `base/ir.http@_match()` will
fail, everything must be set either prior to calling this method or in
`website/ir.http@_serve_fallback`.
---
The original implementation was overriding the `_dispatch` method which
was responsible to call the four `_match()`, `_authenticate()`,
`_postprocess_args` and finally `WebRequest._dispatch()`. The override
was very special, here is an attempt to explain it:
1) try to match an endpoint using the backend router, 404-errors are
ignored.
2) include the geoip stuff.
3) authenticate using the `auth` @route argument if an endpoint matched
in (1), otherwise authenticate with the public user.
4) if not endpoint matched or if a frontend endpoint matched in (1):
a) call `_add_dispatch_parameters` which sets many arguments on the
`request` object, including the lang found in the request cookies
b) try to extract a lang from the URL: abort with a redirection when
the lang is missing or wrong, remove the lang from the request
path when it is set (updating both `request.lang` and the cookie).
5) return the result of `super()._dispatch()`
Note that `_serve_fallback()` is called during `super()._dispatch()`
when the path still does not route to an endpoint. Thanks to the
`_dispatch` overrides in http_routing and website, it is garanteed that
the system is setup prior to calling `_serve_fallback`.
---
Because it is now `http.py@Request._serve_ir_http()` that is responsible
of calling the four`_match()`, `_authenticate()`, `_pre_dispatch` and
finally `_(http|json)_dispatch()` it is no more possible to override it
to take over the dispatching to perform the http_routing/website magic.
The prior implementation can not work with the new design thus is has
been refactored too.
To render a website page, there must be a user configured on the
environment (not None) and the various special attributes must be set on
the request object. The special `lang` attribute is popped from the
request path but backend endpoints must be delivered in priority.
Using the new design, it has been decided to override the `_match`
method to implement the lang-in-path logic, to override both
`_pre_dispatch` and `_serve_fallback` to call `_add_dispatch_parameters`
which have been renamed `_frontend_pre_dispatch` and to also grant the
public user in the `_serve_fallback` override.
The `_handle_error` override in website has similar needs, the function
is called upon error (4xx/5xx) in order to render a pretty
website-looking page. Because such error can occurs as early as in
`_match()` (page not found and no fallback), when nothing has been setup
yet, website is yet again responsible for setuping everything: request,
orm, frontend.
Many other small improvement are not described here. Hopefully the added
comments in the source code are enought.
PR: odoo#78857
Task: 2571224
Before this commit the phone fields inside forms did start in a blank
state.
After this commit the empty phone fields are pre-filled with "+" and the
country phone code.
The country is determined based on the IP address with the GeoIP
database. The phone code is obtained from `res.country` where it is
kept in cache.
task-2463563
closesodoo/odoo#78068
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Co-authored-by: Benoit Socias <bso@odoo.com>
The record is read directly in sudo, the prefetch is in the sudo
environment. When reading the other information there is no longer any
need to make a request.
closesodoo/odoo#82475
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Initialize the savepoint semi-lazily (on-demand) as cycling a savepoint
generates 4~5 queries (depending whether the savepoint is explicitly
released on COMMIT or not):
SAVEPOINT
-- < do stuff>
-- commit
RELEASE SAVEPOINT -- or not
SAVEPOINT
-- close
ROLLBACK TO SAVEPOINT
RELEASE SAVEPOINT
With a lazy savepoint, this is just 0-1 queries (`SAVEPOINT` at the
first explicit query only, creating a cursor and then closing it
immediately is a no-op).
For reliability use a semi-lazy savepoint: always immediately emit a
`SAVEPOINT` on cursor creation, but don't automatically create one
after each `commit`. That limits the issues of overlapping (but
non-nested) savepoints.
Also fix the `generate` API to not use the request: when `website` was
converted to the new API, `cr`, `uid`, and `context` were dropped as
if it were a model... but it's not. So in order to recover an
execution environment, that was looked up on the session.
That, then, turns out to be an issue when `generate` is triggered from
an RPC call: the RPC layer creates its own cursor and environment
separate from the request's which may not have one at
all. Problematically during testing we're in `mono_db` mode, so the
request's cursor/env can be accessed and will be lazily initialized.
This then causes an issue with the `TestCursor`'s savepoints: rather
than be nested, the lifetimes of the request's and RPC's savepoints
only overlap[0]:
|-- rpc --|
|-- request --|
As a result, when the RPC's cursor is committed and released it
automatically released the request's, and the request's explicit
release then fails. This would break `/website:WithContext.test_search`.
By fixing the API of `generate`, it stops triggering the creation of a
request cursor, and therefore the overlap and resulting error.
[0] the laziness or eagerness of the savepointing in the test cursor
has no impact on this issue, as multiple requests have already been
issued on the RPC's test cursor before the request's is even
created
Part-of: odoo/odoo#76243
* Remove AST in favor of pure Pyhon. This should make it easier for
developers to understand and create new directives because they do not
need to know AST.
* Remove `t-call-options` as it has been merged into `t-options` for more
consistency. Support for t-call-options is retained.
* Use generators for lists. This increases performances as the rendering
can be sent directly without having to wait for the creation of the
entire list.
* Optimize expressions runtime computation by pre-computing the static
parts.
Example:
'<' + 'div' + '>' + '<' + dynamic_value + '>'
Now compiles as:
'<div><' + dynamic_value + '>'
*: web_editor
Provide the user with an option to hide or display a section tag
depending on the visitor's location (if geoip enabled), language,
utm_medium, utm_source and utm_campaign.
Users can use a M2M widget to select different records. These will be
used to create a CSS selector on save that will hide the section tag
depending on the various options they selected.
This CSS selectors will be applied, as well as various data attributes
when the page loads to hide the targeted tag. All of these operations
are handled client side.
The geoip country had to be added to the session for the feature to work
with countries.
Part of https://github.com/odoo/odoo/pull/67140
task-2381049
closesodoo/odoo#67140
Related: odoo/enterprise#19907
Related: odoo/upgrade#2658
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
Co-authored-by: qsm-odoo <qsm@odoo.com>
This branch adds request.redirect on all requests.
In case of a front end request, we do an url_for to the location.
We removed redirect_with_hash that was only for retro compatibility
local_redirect has been renamed to redirect_query, and param keep_hash has been
removed and moved.
Default code for redirect is 303 now instead of 302.
Now redirect and redirect_query make local redirect by default, you need to
pass local=False to make external redirect.
All werkeug.utils.redirect has been replaced by request.redirect.
Http.redirect now use an http.Response type, and it become easy to add an
override like 'set_cookies' e.g.
Dispatch of a website.page return an http.response too, so we first need to
check if it is a cached version before to check if it is an Odoo Response.
Migrate your code:
http.redirect -> request.redirect(location, code, local)
http.local_redirect -> request.redirect_query(location, query, code, local)
http.redirect_with_hash -> request.redirect
Courtesy of odony for help and review ;)
closesodoo/odoo#72599
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Before to search if a redirect exists, we try to match a page via serve_page.
But this one redirect without the trailing /. So if none page is found, we
look for redirection, but we need to search with and without this trialing /.
opw-2527063
closesodoo/odoo#72847
X-original-commit: df6f92f1c290276fed8d0643315cc0973f3729e5
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
The method read_combined() has been deprecated in favor of explicit
calls to read() and get_combined_arch().
There are places where we call _get_combined_arch() instead, in order to
avoid parsing the XML that has just been serialized. This saves useless
serialization-deserialization.
Task: 2541577
Meta task: 2463632
Co-authored-by: Fabien Pinckaers <fp@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
This commit fixes multiple bugs when serving a `website.page` which has a
trailing slash:
1. We use a 301 instead of a 302 as it is done for every other dispatching
flows (mostly through the redirect in `_postprocess_args()`)
2. The language is no more lost during the dispatching, before this commit
`/fr_BE/contactus/` would redirect to `/contactus`
3. The URL params are no more lost during the dispatching, before this commit
`/contactus/?a=b` would redirect to `/contactus`
opw-2505818
opw-2513575
Community: https://github.com/odoo/odoo/pull/71065
Enterprise: https://github.com/odoo/enterprise/pull/18615
Before this commit, the routing map generated and used would be the one from
the website the request is performed, instead of the one from the `fw` website
ID which will be the one we redirect the user to.
This issue was introduced with the routing map by website, be8fc2296b and is
restricted to a single case: a publisher using the website switcher, and it
won't happen on next page naviguation/refresh as the `fw` website id will be
the same as the current website's ID. Thus there won't be any routing map
mismatch.
Step to reproduce:
- Create a page on website 2, set it as homepage
- Naviguate to website 1 on '/' url
- Naviguate to website 2 on '/' url
This will raise a werkzeug error about `EndPoint not iterable`.
----- Technical analysis ------
This is the current flow:
1. `_dispatch()` is setting `website_routing` to `get_current_website()` -> 2
2. `_dispatch()` is calling `_match()`
3. `_match()` is calling `routing_map()` with key = `website_routing`, which
was set to 2 in step 1.
4. `routing_map()` is calling `_generate_routing_rules()` which generate the
rules based on `website_routing`, which was set to 2 in step 1.
5. `_dispatch()` authenticate the user by calling `_authenticate()`
6. `_dispatch()` is calling `_add_dispatch_parameter()`, where URL param `fw`
is forced in session, so `get_current_website()` now return the correct
`website_id` -> 1
The issue: in order to handle the `fw` URL parameter (step 6.), we need to
check the rights to ensure we can allow the website switch.
To check rights, user need to be authenticated (step 5.), which is done after
generating the routing map (2. & 3. & 4.).
The routing map is generated based on the current website (step 1.)
Step 6 depends of steps 5 which depends of steps 2/3/4 which depend of step 1,
but step 1 should depend of step 6, which is an impossible cycle.
closesodoo/odoo#70397
X-original-commit: 4eb26497a774e934d7e1a6be9f505400fd9e2cdb
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Signed-off-by: Romain Derie <rdeodoo@users.noreply.github.com>
Before this commit, a 308 on a route with a modelconverter for a model
that have a seo_name field will crash with an exception:
Cannot iterate on RequestUID
Another simplest solution was to use a with_user(SUPERUSER_ID) but in this case
it bypass the security set and display the name of the record even if not yet
published.
How to reproduce:
Create a 308 from /shop/<product> to /mag/<product>
Unpublish product 10
Try to access /shop/product-10
You have an unmanaged '500 internal error"
because slug_matching -> build -> to_url -> slug with a record with Requestuid
as env._uid.
X-original-commit: 4ac2cab96655a3c5e673b0a04be5599dba507850
Before this commit, routes were prevented to be accessed if the route model was
not accessible from the current website, eg you couldn't access blog1 which is
set to website2 from website 1.
That would raise a 404 even from admin/editor.
This commit introduce that behavior at a lowel level in a generic way, instead
of having to write it on every route.
Note that routes without a model converter won't benefit from this.
Closes#63499closesodoo/odoo#64313
Related: odoo/enterprise#15688
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
This commit adapts the business code in which
class/module/function/method redefinition took place so that it no
longer happens and the pylint test passes.
Since 3d65424708d, if the homepage does not exists (no page with '/' as URL,
no page set as homepage), it would loop between / controller and _serve_page.
task-2250381
closesodoo/odoo#59548
X-original-commit: 07b701c581bd960bed42fcf97fae0dfed0332656
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Signed-off-by: Romain Derie <rdeodoo@users.noreply.github.com>
After this commit, if an url with trailing slash is not found we fallback
to the url without the slash with a redirect to respect seo.
With this change we have the same behavior that controllers.
We continue to support trailing slash if page explicitely created with /
via backend e.g. and for retrocompatibility.
task-2346185
closesodoo/odoo#58603
X-original-commit: 3d65424708d1623654756d58d78cec5f8d71276f
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
To avoid to manually invalid the lru cache and probably miss some place
(assets, website menu, qweb view, website, company, ...) we use the classic
cache.
Courtesy of @rco-odoo for the help and review
closesodoo/odoo#56744
X-original-commit: 98e12901c97a67dc87f21647473754cd6f384d4a
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Before this commit we cehck first if user was publisher before to check if the
exception was a 404 NotFound.
What's means that we make this check (1 query sql) most of the time for no
reason, and worst case, when the exception crashes the cursor, this check will
reraise a new exception because isPublisher check is done with a closed cursor.
closesodoo/odoo#55789
X-original-commit: 8afa25a9ca3b165033ab664ce3185c453fb2dd85
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
render, render_template, load, activity_schedule_with_view,
get_website_pages should all be private:
It should not be possible to render an aribtrary template only with
its name or id
Still need to render some qweb views from js so the method
render_template is kept public.
This explains why the website editor still need read access on
ir.ui.view as we want to allow any snippet to be rendered.
TL;DR: remember `osv` and `except_orm` ? You can forget about them.
* Deprecated `except_orm` dropped.
* `UserError` elevated as super type of all user-related
errors.
* Unused `DeferredException` dropped.
* Unused `QWebException` dropped (real one is in `qweb.py`).
* `MailDeliveryException` made a python exception.
* `name` legacy exception attribute made an alias of the python standard
`args[0]` attribute and deprecated.
* `value` legacy exception attribute dropped.
* `exception_type` RPC error response key dropped.
* Deprecated `osv` module dropped.
* `--osv-memory-age-limit` cli option made an alias of
`--transient-age-limit` and deprecated.
The `odoo.exceptions.Warning` have long been a deprecated alias to
`UserError`. It is going to be removed in a future version but first we
explicitly deprecate it with a warning.
The `odoo.exceptions.DeferredException` was a very old internal
exception, it has been removed without deprecation notice as it is never
raised.
The `odoo.exceptions.except_orm` has been a deprecated exception type
with deprecation warning for 5 years, it has been removed in favor of
UserError which becomes the super class of all user-related errors.
The `odoo.base.models.ir_mail_server.MailDeliveryException` was
inheriting `except_orm`. As it is not related to a user error but is
more of a problem an admin much take care of, the exception has been
made a Python error.
The `exception_type` JSON key in RPC error responses was holding an
hardcoded value derived from the exception type. Its usage has been
dropped in favor of the `name` JSON key that holds the precise exception
name. Again as it was hardly used in the source code (beside the crash
manager) it has been dropped without deprecation warning.
Since we are here trying to clean odoo custom exceptions, we are also
deprecating the `name` exception attribute in favor of the more standard
`args[0]` attribute.
The `name` (along with `value`) were two attributes used to raise
`except_orm` exceptions before the introduction of `UserError`,
`AccessError` and related exceptions. The `name` attribute, at the time,
was holding the exception type/title. Nowadays it contains the error
message. The `value` attribute, at the time, was holding the error
message. Nowadays it is no more used.
The `osv` module contains very old deprecated aliases. There is no
simple way to log a deprecation warning for osv, osv_memory and
osv_abstract but as they have not been in use for ages, they have been
removed too. To be consistent, the `--osv-memory-age-limit` cli option
has been made a deprecated alias to the `--transient-age-limit`.
closesodoo/odoo#45723
Task: 2187728
Related: odoo/enterprise#9162
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The dispatching layer of Odoo (common to any call) can avoid a query on website
if those 3 values are cached.
Note that for complex business flows, there won't be any gain since website
infos will have to be fetched at some point.
About user_id:
In most cases, for a website page you will need to browse the website btw so
we don't remove this request previously. But in case of a simple image, the
controller /web/content need the public user just to set the request.uid in
auth_public but no other info from website.
With this commit, we don't do the 'select * from website where id in()'
request on each controller declared in auth='public' but use the user_id
from the cache. (Invalidated by the write on website_id)
task-2211013
Co-authored-by: Jérémy Kersten <jke@odoo.com>
Co-authored-by: Romain Derie <rde@odoo.com>
If we are rendering a page, the menus will most likely be rendered as well.
Note that even in a 403 case when the page is found but is not visible, the
menus will also be shown on the 403 page.
Fetching the menus before accessing the requested page will prefetch that page
as well in one go if that page is in the menus, without any costs for the pages
not in the menus.
There is a tradeoff for 'non-layout' pages, which only occurs in advanced
technical cases:
1. Create a page with specific extension as name suffix (page.css) in which
case the page will be bootstraped accordingly, without call to layout.
2. Remove the call to layout in HTML editor or backend
3. Remove the call to submenus template in HTML editor or backend
For those cases, the menus will be prefetched for no reason.
Note that homepage '/' is rendered through a controller, same logic is applied
there.
task-2211013
Jeremy Kersten