pengyuthon/odoo_source
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[REF] core: HTTPocalypse (13) http_routing/website

Browse Source 
This commit is the 13th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.

Here be dragons.

First and foremost, `http_routing` is a technical module that aim to
provide the minimum viable compatibility code between portal and
website. Its primary job is to take care of the lang inserted in the
path of URLs, e.g. `en` in `/en/my_blog`. Both when routing a request
with a lang in the URL and when rendering templates with multilang
support.

Next to `http_routing` is website, the module used by customers to
create pretty web page accessible online. Website uses a different
routing logic than the backend, webpages are **not** registered in the
routing map of werkzeug but instead delivered by website dedicated
code. It means that **all** request targeting a website page thrown at
the werkzeug router **fail** with a HTTP 404 error. The reality is that
website abuses the fallback mechanism (`_serve_fallback`) to deliver
its pages.

Using `_serve_fallback` as the standard way to deliver pages is broken
by design. It is the least crappy way to deliver content as long as
website page don't have a dedicated path prefix. Using a path prefix
(e.g. `p` in `/p/fr/mon_blog`) it would have been possible to route the
request to a dedicated endpoint using the same router as the backend and
with no change to the HTTP dispatching code. Sadly, the business does
not want such prefix so we have to stick with a broken design.

It is broken because prior to serving a page, website needs to setup A
LOT of stuff on the system. It needs to ensure a proper user is set on
the environment, it needs to setup the GeoIP database, it also needs
to determine the lang the user requested the page and it also needs to
save multiple attributes on the request objet itself (`is_frontend`,
`is_frontend_multilang`, `routing_iteration`, `website`, `lang`,
`rerouting` and `website_routing`). Since `base/ir.http@_match()` will
fail, everything must be set either prior to calling this method or in
`website/ir.http@_serve_fallback`.

---

The original implementation was overriding the `_dispatch` method which
was responsible to call the four `_match()`, `_authenticate()`,
`_postprocess_args` and finally `WebRequest._dispatch()`. The override
was very special, here is an attempt to explain it:

1) try to match an endpoint using the backend router, 404-errors are
   ignored.
2) include the geoip stuff.
3) authenticate using the `auth` @route argument if an endpoint matched
   in (1), otherwise authenticate with the public user.
4) if not endpoint matched or if a frontend endpoint matched in (1):
  a) call `_add_dispatch_parameters` which sets many arguments on the
     `request` object, including the lang found in the request cookies
  b) try to extract a lang from the URL: abort with a redirection when
     the lang is missing or wrong, remove the lang from the request
     path when it is set (updating both `request.lang` and the cookie).
5) return the result of `super()._dispatch()`

Note that `_serve_fallback()` is called during `super()._dispatch()`
when the path still does not route to an endpoint. Thanks to the
`_dispatch` overrides in http_routing and website, it is garanteed that
the system is setup prior to calling `_serve_fallback`.

---

Because it is now `http.py@Request._serve_ir_http()` that is responsible
of calling the four`_match()`, `_authenticate()`, `_pre_dispatch` and
finally `_(http|json)_dispatch()` it is no more possible to override it
to take over the dispatching to perform the http_routing/website magic.
The prior implementation can not work with the new design thus is has
been refactored too.

To render a website page, there must be a user configured on the
environment (not None) and the various special attributes must be set on
the request object. The special `lang` attribute is popped from the
request path but backend endpoints must be delivered in priority.

Using the new design, it has been decided to override the `_match`
method to implement the lang-in-path logic, to override both
`_pre_dispatch` and `_serve_fallback` to call `_add_dispatch_parameters`
which have been renamed `_frontend_pre_dispatch` and to also grant the
public user in the `_serve_fallback` override.

The `_handle_error` override in website has similar needs, the function
is called upon error (4xx/5xx) in order to render a pretty
website-looking page. Because such error can occurs as early as in
`_match()` (page not found and no fallback), when nothing has been setup
yet, website is yet again responsible for setuping everything: request,
orm, frontend.

Many other small improvement are not described here. Hopefully the added
comments in the source code are enought.

PR: odoo#78857
Task: 2571224
This commit is contained in:
Julien Castiaux
2022-02-24 13:30:50 +00:00
parent f04b90b6e8
commit 06cc322e7e
6 changed files with 425 additions and 415 deletions
Download Patch File Download Diff File Expand all files Collapse all files
addons/http_routing/models/ir_http.py
+239 -238
View File
@@ -1,13 +1,16 @@
# -*- coding: utf-8 -*-
import contextlib
import logging
import os
import re
import traceback
import threading
import unicodedata
import werkzeug.exceptions
import werkzeug.routing
import werkzeug.urls
from werkzeug.exceptions import HTTPException, NotFound
# optional python-slugify import (https://github.com/un33k/python-slugify)
try:
@@ -16,7 +19,7 @@ except ImportError:
slugify_lib = None
import odoo
from odoo import api, models, registry, exceptions, tools, http
from odoo import api, models, exceptions, tools, http
from odoo.addons.base.models import ir_http
from odoo.addons.base.models.ir_http import RequestUID
from odoo.addons.base.models.ir_qweb import QWebException
@@ -254,6 +257,7 @@ class ModelConverter(ir_http.ModelConverter):
_uid = RequestUID(value=value, match=matching, converter=self)
record_id = int(matching.group(2))
env = api.Environment(request.cr, _uid, request.context)
if record_id < 0:
# limited support for negative IDs due to our slug pattern, assume abs() if not found
if not env[self.model].browse(record_id).exists():
@@ -289,7 +293,7 @@ class IrHttp(models.AbstractModel):
IrHttpModel = request.env['ir.http'].sudo()
modules = IrHttpModel.get_translation_frontend_modules()
user_context = request.session.get_context() if request.session.uid else {}
user_context = request.session.context if request.session.uid else {}
lang = user_context.get('lang')
translation_hash = request.env['ir.translation'].get_web_translations_hash(modules, lang)
@@ -337,15 +341,184 @@ class IrHttp(models.AbstractModel):
:param lang_code: the lang `code` (en_US)
"""
if not lang_code:
return False
short_match = False
return None
lang_codes = cls._get_frontend_langs()
if lang_code in lang_codes:
return lang_code
short = lang_code.partition('_')[0]
for code in cls._get_frontend_langs():
if code == lang_code:
return code
if not short_match and code.startswith(short):
short_match = code
return short_match
return next((code for code in lang_codes if code.startswith(short)), None)
@classmethod
def _match(cls, path):
"""
Grant multilang support to URL matching by using http 3xx
redirections and URL rewrite. This method also grants various
attributes such as ``lang`` and ``is_frontend`` on the current
``request`` object.
1/ Use the URL as-is when it matches a non-multilang compatible
endpoint.
2/ Use the URL as-is when the lang is not present in the URL and
that the default lang has been requested.
3/ Use the URL as-is saving the requested lang when the user is
a bot and that the lang is missing from the URL.
4/ Redirect the browser when the lang is missing from the URL
but another lang than the default one has been requested. The
requested lang is injected before the original path.
5) Use the url as-is when the lang is missing from the URL, that
another lang than the default one has been requested but that
it is forbidden to redirect (e.g. POST)
6/ Redirect the browser when the lang is present in the URL but
it is the default lang. The lang is removed from the original
URL.
7/ Redirect the browser when the lang present in the URL is an
alias of the preferred lang url code (e.g. fr_FR -> fr)
8/ Redirect the browser when the requested page is the homepage
but that there is a trailing slash.
9/ Rewrite the URL when the lang is present in the URL, that it
matches and that this lang is not the default one. The URL is
rewritten to remove the lang.
Note: The "requested lang" is (in order) either (1) the lang in
the URL or (2) the lang in the ``frontend_lang`` request
cookie or (3) the lang in the context or (4) the default
lang of the website.
"""
# The URL has been rewritten already
if hasattr(request, 'is_frontend'):
return super()._match(path)
# See /1, match a non website endpoint
with contextlib.suppress(NotFound):
rule, args = super()._match(path)
routing = rule.endpoint.routing
request.is_frontend = routing.get('website', False)
request.is_frontend_multilang = request.is_frontend and routing.get('multilang', routing['type'] == 'http')
if not request.is_frontend:
return rule, args
_, url_lang_str, *rest = path.split('/', 2)
path_no_lang = '/' + (rest[0] if rest else '')
allow_redirect = request.httprequest.method != 'POST'
# There is no user on the environment yet but the following code
# requires one to set the lang on the request. Temporary grant
# the public user. Don't try it at home!
real_env = request.env
try:
request.registry['ir.http']._auth_method_public() # it calls update_env
nearest_url_lang = cls.get_nearest_lang(request.env['res.lang']._lang_get_code(url_lang_str))
cookie_lang = cls.get_nearest_lang(request.httprequest.cookies.get('frontend_lang'))
context_lang = cls.get_nearest_lang(real_env.context.get('lang'))
default_lang = cls._get_default_lang()
request.lang = request.env['res.lang']._lang_get(
nearest_url_lang or cookie_lang or context_lang or default_lang._get_cached('code')
)
request_url_code = request.lang._get_cached('url_code')
finally:
request.env = real_env
if not nearest_url_lang:
url_lang_str = None
# See /2, no lang in url and default website
if not url_lang_str and request.lang == default_lang:
_logger.debug("%r (lang: %r) no lang in url and default website, continue", path, request_url_code)
# See /3, missing lang in url but user-agent is a bot
elif not url_lang_str and request.env['ir.http'].is_a_bot():
_logger.debug("%r (lang: %r) missing lang in url but user-agent is a bot, continue", path, request_url_code)
request.lang = default_lang
# See /4, no lang in url and should not redirect (e.g. POST), continue
elif not url_lang_str and not allow_redirect:
_logger.debug("%r (lang: %r) no lang in url and should not redirect (e.g. POST), continue", path, request_url_code)
# See /5, missing lang in url, /home -> /fr/home
elif not url_lang_str:
_logger.debug("%r (lang: %r) missing lang in url, redirect", path, request_url_code)
redirect = request.redirect_query(f'/{request_url_code}{path}', request.httprequest.args)
redirect.set_cookie('frontend_lang', request.lang._get_cached('code'))
werkzeug.exceptions.abort(redirect)
# See /6, default lang in url, /en/home -> /home
elif url_lang_str == default_lang.url_code and allow_redirect:
_logger.debug("%r (lang: %r) default lang in url, redirect", path, request_url_code)
redirect = request.redirect_query(path_no_lang, request.httprequest.args)
redirect.set_cookie('frontend_lang', default_lang._get_cached('code'))
werkzeug.exceptions.abort(redirect)
# See /7, lang alias in url, /fr_FR/home -> /fr/home
elif url_lang_str != request_url_code and allow_redirect:
_logger.debug("%r (lang: %r) lang alias in url, redirect", path, request_url_code)
redirect = request.redirect_query(f'/{request_url_code}{path_no_lang}', request.httprequest.args, code=301)
redirect.set_cookie('frontend_lang', request.lang._get_cached('code'))
werkzeug.exceptions.abort(redirect)
# See /8, homepage with trailing slash. /fr_BE/ -> /fr_BE
elif path == f'/{url_lang_str}/' and allow_redirect:
_logger.debug("%r (lang: %r) homepage with trailing slash, redirect", path, request_url_code)
redirect = request.redirect_query(path[:-1], request.httprequest.args, code=301)
redirect.set_cookie('frontend_lang', default_lang._get_cached('code'))
werkzeug.exceptions.abort(redirect)
# See /9, valid lang in url
elif url_lang_str == request_url_code:
# Rewrite the URL to remove the lang
_logger.debug("%r (lang: %r) valid lang in url, rewrite url and continue", path, request_url_code)
cls.reroute(path_no_lang)
path = path_no_lang
else:
_logger.warning("%r (lang: %r) couldn't not correctly route this frontend request, url used as-is.", path, request_url_code)
# Re-match using rewritten route and really raise for 404 errors
try:
rule, args = super()._match(path)
routing = rule.endpoint.routing
request.is_frontend = routing.get('website', False)
request.is_frontend_multilang = request.is_frontend and routing.get('multilang', routing['type'] == 'http')
return rule, args
except NotFound:
# Use website to render a nice 404 Not Found html page
request.is_frontend = True
request.is_frontend_multilang = True
raise
@classmethod
def reroute(cls, path, query_string=None):
"""
Rewrite the current request URL using the new path and query
string. This act as a light redirection, it does not return a
3xx responses to the browser but still change the current URL.
"""
if query_string is None:
query_string = request.httprequest.environ['QUERY_STRING']
# Change the WSGI environment
environ = request.httprequest.environ.copy()
environ['PATH_INFO'] = path
environ['QUERY_STRING'] = query_string
environ['RAW_URI'] = f'{path}?{query_string}'
# REQUEST_URI left as-is so it still contains the original URI
# Create and expose a new request from the modified WSGI env
httprequest = werkzeug.wrappers.Request(environ)
httprequest.parameter_storage_class = (
werkzeug.datastructures.ImmutableOrderedMultiDict)
threading.current_thread().url = httprequest.url
request.httprequest = httprequest
@classmethod
def _geoip_setup_resolver(cls):
@@ -366,191 +539,50 @@ class IrHttp(models.AbstractModel):
record = odoo._geoip_resolver.resolve(request.httprequest.remote_addr) or {}
request.session['geoip'] = record
@classmethod
def _add_dispatch_parameters(cls, func):
Lang = request.env['res.lang']
# only called for is_frontend request
if request.routing_iteration == 1:
context = dict(request.context)
path = request.httprequest.path.split('/')
is_a_bot = request.env['ir.http'].is_a_bot()
lang_codes = [code for code, *_ in Lang.get_available()]
nearest_lang = not func and cls.get_nearest_lang(Lang._lang_get_code(path[1]))
cook_lang = request.httprequest.cookies.get('frontend_lang')
cook_lang = cook_lang in lang_codes and cook_lang
if nearest_lang:
lang = Lang._lang_get(nearest_lang)
else:
nearest_ctx_lg = not is_a_bot and cls.get_nearest_lang(request.env.context.get('lang'))
nearest_ctx_lg = nearest_ctx_lg in lang_codes and nearest_ctx_lg
preferred_lang = Lang._lang_get(cook_lang or nearest_ctx_lg)
lang = preferred_lang or cls._get_default_lang()
request.lang = lang
context['lang'] = lang._get_cached('code')
# bind modified context
request.context = context
@classmethod
def _dispatch(cls):
""" Before executing the endpoint method, add website params on request, such as
- current website (record)
- multilang support (set on cookies)
- geoip dict data are added in the session
Then follow the parent dispatching.
Reminder : Do not use `request.env` before authentication phase, otherwise the env
set on request will be created with uid=None (and it is a lazy property)
"""
request.routing_iteration = getattr(request, 'routing_iteration', 0) + 1
func = None
routing_error = None
# handle // in url
if request.httprequest.method == 'GET' and '//' in request.httprequest.path:
new_url = request.httprequest.path.replace('//', '/') + '?' + request.httprequest.query_string.decode('utf-8')
return request.redirect(new_url, code=301)
# locate the controller method
try:
rule, arguments = cls._match(request.httprequest.path)
func = rule.endpoint
request.is_frontend = func.routing.get('website', False)
except werkzeug.exceptions.NotFound as e:
# either we have a language prefixed route, either a real 404
# in all cases, website processes them exept if second element is static
# Checking static will avoid to generate an expensive 404 web page since
# most of the time the browser is loading and inexisting assets or image. A standard 404 is enough.
# Earlier check would be difficult since we don't want to break data modules
path_components = request.httprequest.path.split('/')
request.is_frontend = len(path_components) < 3 or path_components[2] != 'static' or '.' not in path_components[-1]
routing_error = e
request.is_frontend_multilang = not func or (func and request.is_frontend and func.routing.get('multilang', func.routing['type'] == 'http'))
# check authentication level
try:
if func:
cls._authenticate(func)
elif request.uid is None and request.is_frontend:
cls._auth_method_public()
except Exception as e:
return cls._handle_exception(e)
def _pre_dispatch(cls, rule, args):
super()._pre_dispatch(rule, args)
cls._geoip_setup_resolver()
cls._geoip_resolve()
# For website routes (only), add website params on `request`
if request.is_frontend:
cls._add_dispatch_parameters(func)
cls._frontend_pre_dispatch()
path = request.httprequest.path.split('/')
default_lg_id = cls._get_default_lang()
if request.routing_iteration == 1:
is_a_bot = request.env['ir.http'].is_a_bot()
nearest_lang = not func and cls.get_nearest_lang(request.env['res.lang']._lang_get_code(path[1]))
url_lg = nearest_lang and path[1]
# update the context of "<model(...):...>" args
for key, val in list(args.items()):
if isinstance(val, models.BaseModel):
args[key] = val.with_context(request.context)
# The default lang should never be in the URL, and a wrong lang
# should never be in the URL.
wrong_url_lg = url_lg and (url_lg != request.lang.url_code or url_lg == default_lg_id.url_code)
# The lang is missing from the URL if multi lang is enabled for
# the route and the current lang is not the default lang.
# POST requests are excluded from this condition.
missing_url_lg = not url_lg and request.is_frontend_multilang and request.lang != default_lg_id and request.httprequest.method != 'POST'
# Bots should never be redirected when the lang is missing
# because it is the only way for them to index the default lang.
if wrong_url_lg or (missing_url_lg and not is_a_bot):
if url_lg:
path.pop(1)
if request.lang != default_lg_id:
path.insert(1, request.lang.url_code)
path = '/'.join(path) or '/'
routing_error = None
redirect = request.redirect(path + '?' + request.httprequest.query_string.decode('utf-8'))
redirect.set_cookie('frontend_lang', request.lang.code)
return redirect
elif url_lg:
request.uid = None
if request.httprequest.path == '/%s/' % url_lg:
# special case for homepage controller, mimick `_postprocess_args()` redirect
path = request.httprequest.path[:-1]
if request.httprequest.query_string:
path += '?' + request.httprequest.query_string.decode('utf-8')
return request.redirect(path, code=301)
path.pop(1)
routing_error = None
return cls.reroute('/'.join(path) or '/')
elif missing_url_lg and is_a_bot:
# Ensure that if the URL without lang is not redirected, the
# current lang is indeed the default lang, because it is the
# lang that bots should index in that case.
request.lang = default_lg_id
request.context = dict(request.context, lang=default_lg_id.code)
if request.lang == default_lg_id:
context = dict(request.context)
context['edit_translations'] = False
request.context = context
if routing_error:
return cls._handle_exception(routing_error)
# removed cache for auth public
result = super(IrHttp, cls)._dispatch()
cook_lang = request.httprequest.cookies.get('frontend_lang')
if request.is_frontend and cook_lang != request.lang.code and hasattr(result, 'set_cookie'):
result.set_cookie('frontend_lang', request.lang.code)
return result
if request.is_frontend_multilang:
# A product with id 1 and named 'egg' is accessible via a
# frontend multilang enpoint 'foo' at the URL '/foo/1'.
# The preferred URL to access the product (and to generate
# URLs pointing it) should instead be the sluggified URL
# '/foo/egg-1'. This code is responsible of redirecting the
# browser from '/foo/1' to '/foo/egg-1', or '/fr/foo/1' to
# '/fr/foo/oeuf-1'. While it is nice (for humans) to have a
# pretty URL, the real reason of this redirection is SEO.
if request.httprequest.method in ('GET', 'HEAD'):
try:
_, path = rule.build(args)
except odoo.exceptions.MissingError:
raise werkzeug.exceptions.NotFound()
assert path is not None
generated_path = werkzeug.urls.url_unquote_plus(path)
current_path = werkzeug.urls.url_unquote_plus(request.httprequest.path)
if generated_path != current_path:
if request.lang != cls._get_default_lang():
path = f'/{request.lang.url_code}{path}'
redirect = request.redirect_query(path, request.httprequest.args, code=301)
werkzeug.exceptions.abort(redirect)
@classmethod
def _redirect(cls, location, code=303):
if request and request.db and getattr(request, 'is_frontend', False):
location = url_for(location)
return super()._redirect(location, code)
@classmethod
def reroute(cls, path):
if not hasattr(request, 'rerouting'):
request.rerouting = [request.httprequest.path]
if path in request.rerouting:
raise Exception("Rerouting loop is forbidden")
request.rerouting.append(path)
if len(request.rerouting) > cls.rerouting_limit:
raise Exception("Rerouting limit exceeded")
request.httprequest.environ['PATH_INFO'] = path
# void werkzeug cached_property. TODO: find a proper way to do this
for key in ('path', 'full_path', 'url', 'base_url'):
request.httprequest.__dict__.pop(key, None)
return cls._dispatch()
@classmethod
def _postprocess_args(cls, arguments, rule):
super(IrHttp, cls)._postprocess_args(arguments, rule)
try:
_, path = rule.build(arguments)
assert path is not None
except odoo.exceptions.MissingError:
return cls._handle_exception(werkzeug.exceptions.NotFound())
except Exception as e:
return cls._handle_exception(e)
if getattr(request, 'is_frontend_multilang', False) and request.httprequest.method in ('GET', 'HEAD'):
generated_path = werkzeug.urls.url_unquote_plus(path)
current_path = werkzeug.urls.url_unquote_plus(request.httprequest.path)
if generated_path != current_path:
if request.lang != cls._get_default_lang():
path = '/' + request.lang.url_code + path
if request.httprequest.query_string:
path += '?' + request.httprequest.query_string.decode('utf-8')
return request.redirect(path, code=301)
def _frontend_pre_dispatch(cls):
request.update_context(lang=request.lang._get_cached('code'))
if request.httprequest.cookies.get('frontend_lang') != request.lang._get_cached('code'):
request.future_response.set_cookie('frontend_lang', request.lang._get_cached('code'))
@classmethod
def _get_exception_code_values(cls, exception):
@@ -560,7 +592,9 @@ class IrHttp(models.AbstractModel):
exception=exception,
traceback=traceback.format_exc(),
)
if isinstance(exception, exceptions.UserError):
if isinstance(exception, exceptions.AccessDenied):
code = 403
elif isinstance(exception, exceptions.UserError):
values['error_message'] = exception.args[0]
code = 400
if isinstance(exception, exceptions.AccessError):
@@ -592,61 +626,29 @@ class IrHttp(models.AbstractModel):
return code, env['ir.ui.view']._render_template('http_routing.%s' % code, values)
@classmethod
def _handle_exception(cls, exception):
is_frontend_request = bool(getattr(request, 'is_frontend', False))
if not is_frontend_request:
# Don't touch non frontend requests exception handling
return super(IrHttp, cls)._handle_exception(exception)
try:
response = super(IrHttp, cls)._handle_exception(exception)
def _handle_error(cls, exception):
response = super()._handle_error(exception)
if isinstance(response, Exception):
exception = response
else:
# if parent excplicitely returns a plain response, then we don't touch it
return response
except Exception as e:
if 'werkzeug' in config['dev_mode']:
raise e
exception = e
is_frontend_request = bool(getattr(request, 'is_frontend', False))
if not is_frontend_request or not isinstance(response, HTTPException):
# neither handle backend requests nor plain responses
return response
# minimal setup to serve frontend pages
if not request.uid:
cls._auth_method_public()
request.registry['ir.http']._frontend_pre_dispatch()
request.params = request.get_http_params()
code, values = cls._get_exception_code_values(exception)
if code is None:
# Hand-crafted HTTPException likely coming from abort(),
# usually for a redirect response -> return it directly
return exception
if not request.uid:
cls._auth_method_public()
# We rollback the current transaction before initializing a new
# cursor to avoid potential deadlocks.
# If the current (failed) transaction was holding a lock, the new
# cursor might have to wait for this lock to be released further
# down the line. However, this will only happen after the
# request is done (and in fact it won't happen). As a result, the
# current thread/worker is frozen until its timeout is reached.
# So rolling back the transaction will release any potential lock
# and, since we are in a case where an exception was raised, the
# transaction shouldn't be committed in the first place.
request.env.cr.rollback()
with registry(request.env.cr.dbname).cursor() as cr:
env = api.Environment(cr, request.uid, request.env.context)
if code == 500:
_logger.error("500 Internal Server Error:\n\n%s", values['traceback'])
values = cls._get_values_500_error(env, values, exception)
elif code == 403:
_logger.warning("403 Forbidden:\n\n%s", values['traceback'])
elif code == 400:
_logger.warning("400 Bad Request:\n\n%s", values['traceback'])
try:
code, html = cls._get_error_html(env, code, values)
except Exception:
code, html = 418, env['ir.ui.view']._render_template('http_routing.http_error', values)
request.cr.rollback()
if code == 500:
values = cls._get_values_500_error(request.env, values, exception)
try:
code, html = cls._get_error_html(request.env, code, values)
except Exception:
code, html = 418, request.env['ir.ui.view']._render_template('http_routing.http_error', values)
return werkzeug.wrappers.Response(html, status=code, content_type='text/html;charset=utf-8')
@@ -661,8 +663,7 @@ class IrHttp(models.AbstractModel):
except werkzeug.exceptions.MethodNotAllowed:
endpoint = router.match(path, method='GET', query_args=query_args)
except werkzeug.routing.RequestRedirect as e:
# get path from http://{path}?{current query string}
new_url = e.new_url.split('?')[0][7:]
new_url = e.new_url.split('?')[0][7:] # remove scheme
_, endpoint = self.url_rewrite(new_url, query_args)
endpoint = endpoint and [endpoint]
except werkzeug.exceptions.NotFound:
addons/portal/models/ir_http.py
+6 -2
View File
@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
from odoo import models
@@ -15,6 +14,11 @@ class IrHttp(models.AbstractModel):
@classmethod
def _get_frontend_langs(cls):
if request and request.is_frontend:
# _get_frontend_langs() is used by @http_routing:IrHttp._match
# where is_frontend is not yet set and when no backend endpoint
# matched. We have to assume we are going to match a frontend
# route, hence the default True. Elsewhere, request.is_frontend
# is set.
if request and getattr(request, 'is_frontend', True):
return [lang[0] for lang in filter(lambda l: l[3], request.env['res.lang'].get_available())]
return super()._get_frontend_langs()
addons/website/models/ir_http.py
+96 -108
View File
@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
import contextlib
import logging
from lxml import etree
import os
@@ -15,7 +15,7 @@ from functools import partial
import odoo
from odoo import api, models
from odoo import registry, SUPERUSER_ID
from odoo import SUPERUSER_ID
from odoo.exceptions import AccessError, MissingError
from odoo.http import request
from odoo.tools.safe_eval import safe_eval
@@ -69,8 +69,8 @@ class Http(models.AbstractModel):
@classmethod
def clear_caches(cls):
super(Http, cls)._clear_routing_map()
return super(Http, cls).clear_caches()
super()._clear_routing_map()
return super().clear_caches()
@classmethod
def _slug_matching(cls, adapter, endpoint, **kw):
@@ -80,11 +80,6 @@ class Http(models.AbstractModel):
qs = request.httprequest.query_string.decode('utf-8')
return adapter.build(endpoint, kw) + (qs and '?%s' % qs or '')
@classmethod
def _match(cls, path_info, key=None):
key = key or (request and request.website_routing)
return super(Http, cls)._match(path_info, key=key)
@classmethod
def _generate_routing_rules(cls, modules, converters):
website_id = request.website_routing
@@ -94,25 +89,24 @@ class Http(models.AbstractModel):
rewrites = dict([(x.url_from, x) for x in request.env['website.rewrite'].sudo().search(domain)])
cls._rewrite_len[website_id] = len(rewrites)
for url, endpoint, routing in super(Http, cls)._generate_routing_rules(modules, converters):
routing = dict(routing)
for url, endpoint in super()._generate_routing_rules(modules, converters):
if url in rewrites:
rewrite = rewrites[url]
url_to = rewrite.url_to
if rewrite.redirect_type == '308':
logger.debug('Add rule %s for %s' % (url_to, website_id))
yield url_to, endpoint, routing # yield new url
yield url_to, endpoint # yield new url
if url != url_to:
logger.debug('Redirect from %s to %s for website %s' % (url, url_to, website_id))
_slug_matching = partial(cls._slug_matching, endpoint=endpoint)
routing['redirect_to'] = _slug_matching
yield url, endpoint, routing # yield original redirected to new url
endpoint.routing['redirect_to'] = _slug_matching
yield url, endpoint # yield original redirected to new url
elif rewrite.redirect_type == '404':
logger.debug('Return 404 for %s for website %s' % (url, website_id))
continue
else:
yield url, endpoint, routing
yield url, endpoint
@classmethod
def _get_converters(cls):
@@ -120,7 +114,7 @@ class Http(models.AbstractModel):
match Rule. This override adds the website ones.
"""
return dict(
super(Http, cls)._get_converters(),
super()._get_converters(),
model=ModelConverter,
)
@@ -128,16 +122,14 @@ class Http(models.AbstractModel):
def _auth_method_public(cls):
""" If no user logged, set the public user of current website, or default
public user as request uid.
After this method `request.env` can be called, since the `request.uid` is
set. The `env` lazy property of `request` will be correct.
"""
if not request.session.uid:
env = api.Environment(request.cr, SUPERUSER_ID, request.context)
website = env['website'].get_current_website()
request.uid = website and website._get_cached('user_id')
website = request.env(user=SUPERUSER_ID)['website'].get_current_website() # sudo
if website:
request.update_env(user=website._get_cached('user_id'))
if not request.uid:
super(Http, cls)._auth_method_public()
super()._auth_method_public()
@classmethod
def _register_website_track(cls, response):
@@ -159,104 +151,95 @@ class Http(models.AbstractModel):
return False
@classmethod
def _postprocess_args(cls, arguments, rule):
processing = super()._postprocess_args(arguments, rule)
if processing:
return processing
def _match(cls, path):
if not hasattr(request, 'website_routing'):
website = request.env['website'].get_current_website()
request.website_routing = website.id
return super()._match(path)
@classmethod
def _pre_dispatch(cls, rule, arguments):
super()._pre_dispatch(rule, arguments)
for record in arguments.values():
if isinstance(record, models.BaseModel) and hasattr(record, 'can_access_from_current_website'):
try:
if not record.can_access_from_current_website():
return request.env['ir.http']._handle_exception(werkzeug.exceptions.NotFound())
raise werkzeug.exceptions.NotFound()
except AccessError:
# record.website_id might not be readable as unpublished `event.event` due to ir.rule,
# return 403 instead of using `sudo()` for perfs as this is low level
return request.env['ir.http']._handle_exception(werkzeug.exceptions.Forbidden())
# record.website_id might not be readable as
# unpublished `event.event` due to ir.rule, return
# 403 instead of using `sudo()` for perfs as this is
# low level.
raise werkzeug.exceptions.Forbidden()
@classmethod
def _dispatch(cls):
"""
In case of rerouting for translate (e.g. when visiting odoo.com/fr_BE/),
_dispatch calls reroute() that returns _dispatch with altered request properties.
The second _dispatch will continue until end of process. When second _dispatch is finished, the first _dispatch
call receive the new altered request and continue.
At the end, 2 calls of _dispatch (and this override) are made with exact same request properties, instead of one.
As the response has not been sent back to the client, the visitor cookie does not exist yet when second _dispatch call
is treated in _handle_webpage_dispatch, leading to create 2 visitors with exact same properties.
To avoid this, we check if, !!! before calling super !!!, we are in a rerouting request. If not, it means that we are
handling the original request, in which we should create the visitor. We ignore every other rerouting requests.
"""
is_rerouting = hasattr(request, 'routing_iteration')
def _get_web_editor_context(cls):
ctx = super()._get_web_editor_context()
if request.is_frontend_multilang and request.lang == cls._get_default_lang():
ctx['edit_translations'] = False
return ctx
if request.session.db:
reg = registry(request.session.db)
with reg.cursor() as cr:
env = api.Environment(cr, SUPERUSER_ID, {})
request.website_routing = env['website'].get_current_website().id
@classmethod
def _frontend_pre_dispatch(cls):
super()._frontend_pre_dispatch()
response = super(Http, cls)._dispatch()
if not request.context.get('tz'):
with contextlib.suppress(pytz.UnknownTimeZoneError):
tz = request.session.get('geoip', {}).get('time_zone', '')
request.update_context(tz=pytz.timezone(tz).zone)
if not is_rerouting:
cls._register_website_track(response)
website = request.env['website'].get_current_website()
user = request.env.user
# This is mainly to avoid access errors in website controllers
# where there is no context (eg: /shop), and it's not going to
# propagate to the global context of the tab. If the company of
# the website is not in the allowed companies of the user, set
# the main company of the user.
company_id = website._get_cached('company_id')
request.update_context(
allowed_company_ids=(
[company_id] if company_id in user.company_ids.ids else user.company_id.ids
),
website_id=website.id,
**cls._get_web_editor_context(),
)
request.website = website.with_context(request.context)
@classmethod
def _dispatch(cls, endpoint):
response = super()._dispatch(endpoint)
cls._register_website_track(response)
return response
@classmethod
def _add_dispatch_parameters(cls, func):
# DEPRECATED for /website/force/<website_id> - remove me in master~saas-14.4
# Force website with query string paramater, typically set from website selector in frontend navbar and inside tests
force_website_id = request.httprequest.args.get('fw')
if (force_website_id and request.session.get('force_website_id') != force_website_id
and request.env.user.has_group('website.group_multi_website')
and request.env.user.has_group('website.group_website_publisher')):
request.env['website']._force_website(request.httprequest.args.get('fw'))
context = {}
if not request.context.get('tz'):
context['tz'] = request.session.get('geoip', {}).get('time_zone')
try:
pytz.timezone(context['tz'] or '')
except pytz.UnknownTimeZoneError:
context.pop('tz')
request.website = request.env['website'].get_current_website() # can use `request.env` since auth methods are called
context['website_id'] = request.website.id
# This is mainly to avoid access errors in website controllers where there is no
# context (eg: /shop), and it's not going to propagate to the global context of the tab
# If the company of the website is not in the allowed companies of the user, set the main
# company of the user.
website_company_id = request.website._get_cached('company_id')
if website_company_id in request.env.user.company_ids.ids:
context['allowed_company_ids'] = [website_company_id]
else:
context['allowed_company_ids'] = request.env.user.company_id.ids
# modify bound context
request.context = dict(request.context, **context)
super(Http, cls)._add_dispatch_parameters(func)
if request.routing_iteration == 1:
request.website = request.website.with_context(request.context)
@classmethod
def _get_frontend_langs(cls):
if get_request_website():
return [code for code, *_ in request.env['res.lang'].get_available()]
# _get_frontend_langs() is used by @http_routing:IrHttp._match
# where is_frontend is not yet set and when no backend endpoint
# matched. We have to assume we are going to match a frontend
# route, hence the default True. Elsewhere, request.is_frontend
# is set.
if getattr(request, 'is_frontend', True):
website_id = request.env.get('website_id', request.website_routing)
res_lang = request.env['res.lang'].with_context(website_id=website_id)
return [code for code, *_ in res_lang.get_available()]
else:
return super()._get_frontend_langs()
@classmethod
def _get_default_lang(cls):
if getattr(request, 'website', False):
return request.env['res.lang'].browse(request.website._get_cached('default_lang_id'))
return super(Http, cls)._get_default_lang()
if getattr(request, 'is_frontend', True):
website = request.env['website'].sudo().get_current_website()
return request.env['res.lang'].browse([website._get_cached('default_lang_id')])
return super()._get_default_lang()
@classmethod
def _get_translation_frontend_modules_name(cls):
mods = super(Http, cls)._get_translation_frontend_modules_name()
installed = request.registry._init_modules | set(odoo.conf.server_wide_modules)
mods = super()._get_translation_frontend_modules_name()
installed = request.registry._init_modules.union(odoo.conf.server_wide_modules)
return mods + [mod for mod in installed if mod.startswith('website')]
@classmethod
@@ -289,7 +272,7 @@ class Http(models.AbstractModel):
page.cache_time # cache > 0
and request.httprequest.method == "GET"
and request.env.user._is_public() # only cache for unlogged user
and 'nocache' not in request.params # allow bypass cache / debug
and 'nocache' not in request.httprequest.args # allow bypass cache / debug
and not request.session.debug
and len(cache_key) and cache_key[-1] is not None # nocache via expr
):
@@ -333,15 +316,22 @@ class Http(models.AbstractModel):
return request.env['website.rewrite'].sudo().search(domain, limit=1)
@classmethod
def _serve_fallback(cls, exception):
def _serve_fallback(cls):
# serve attachment before
parent = super(Http, cls)._serve_fallback(exception)
parent = super()._serve_fallback()
if parent: # attachment
return parent
if not request.is_frontend:
return False
# minimal setup to serve frontend pages
if not request.uid:
cls._auth_method_public()
cls._frontend_pre_dispatch()
request.params = request.get_http_params()
website_page = cls._serve_page()
if website_page:
website_page.flatten()
cls._register_website_track(website_page)
return website_page
redirect = cls._serve_redirect()
@@ -351,11 +341,9 @@ class Http(models.AbstractModel):
code=redirect.redirect_type,
local=False) # safe because only designers can specify redirects
return False
@classmethod
def _get_exception_code_values(cls, exception):
code, values = super(Http, cls)._get_exception_code_values(exception)
code, values = super()._get_exception_code_values(exception)
if isinstance(exception, werkzeug.exceptions.NotFound) and request.website.is_publisher():
code = 'page_404'
values['path'] = request.httprequest.path[1:]
@@ -368,7 +356,7 @@ class Http(models.AbstractModel):
@classmethod
def _get_values_500_error(cls, env, values, exception):
View = env["ir.ui.view"]
values = super(Http, cls)._get_values_500_error(env, values, exception)
values = super()._get_values_500_error(env, values, exception)
if 'qweb_exception' in values:
try:
# exception.name might be int, string
@@ -398,7 +386,7 @@ class Http(models.AbstractModel):
def _get_error_html(cls, env, code, values):
if code in ('page_404', 'protected_403'):
return code.split('_')[1], env['ir.ui.view']._render_template('website.%s' % code, values)
return super(Http, cls)._get_error_html(env, code, values)
return super()._get_error_html(env, code, values)
def binary_content(self, xmlid=None, model='ir.attachment', id=None, field='datas',
unique=False, filename=None, filename_field='name', download=False,
@@ -433,7 +421,7 @@ class Http(models.AbstractModel):
if obj:
return obj[0]
return super(Http, cls)._xmlid_to_obj(env, xmlid)
return super()._xmlid_to_obj(env, xmlid)
@api.model
def get_frontend_session_info(self):
addons/website/models/res_lang.py
+3 -6
View File
@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
from odoo import api, models, tools, _
@@ -19,8 +18,6 @@ class Lang(models.Model):
@api.model
@tools.ormcache_context(keys=("website_id",))
def get_available(self):
website = ir_http.get_request_website()
if not website:
return super().get_available()
# Return the website-available ones in this case
return request.website.language_ids.get_sorted()
if request and getattr(request, 'is_frontend', True):
return self.env['website'].get_current_website().language_ids.get_sorted()
return super().get_available()
addons/website/models/website.py
+23 -20
View File
@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
import base64
@@ -16,7 +15,7 @@ from werkzeug.datastructures import OrderedMultiDict
from werkzeug.exceptions import NotFound
from odoo import api, fields, models, tools, http, release, registry
from odoo.addons.http_routing.models.ir_http import slugify, _guess_mimetype, url_for
from odoo.addons.http_routing.models.ir_http import RequestUID, slugify, _guess_mimetype, url_for
from odoo.addons.website.models.ir_http import sitemap_qs2dom
from odoo.addons.website.tools import similarity_score, text_from_html
from odoo.addons.portal.controllers.portal import pager
@@ -925,7 +924,7 @@ class Website(models.Model):
# The format of `httprequest.host` is `domain:port`
domain_name = request and request.httprequest.host or ''
country = request.session.geoip.get('country_code') if request and request.session.geoip else False
country = request.session.geoip.get('country_code') if request and 'geoip' in request.session else False
country_id = False
if country:
country_id = self.env['res.country'].search([('code', '=', country)], limit=1).id
@@ -1116,7 +1115,7 @@ class Website(models.Model):
return False
# dont't list routes without argument having no default value or converter
sign = inspect.signature(endpoint.method.original_func)
sign = inspect.signature(endpoint.original_endpoint)
params = list(sign.parameters.values())[1:] # skip self
supported_kinds = (inspect.Parameter.POSITIONAL_ONLY,
inspect.Parameter.POSITIONAL_OR_KEYWORD)
@@ -1138,7 +1137,6 @@ class Website(models.Model):
of the same.
:rtype: list({name: str, url: str})
"""
router = http.root.get_db_router(request.db)
# Force enumeration to be performed as public user
url_set = set()
@@ -1295,30 +1293,35 @@ class Website(models.Model):
"""Returns the canonical URL for the current request with translatable
elements appropriately translated in `lang`.
If `request.endpoint` is not true, returns the current `path` instead.
If it is not possible to rebuild a path, use the current one instead.
`url_quote_plus` is applied on the returned path.
"""
self.ensure_one()
if request.endpoint:
router = http.root.get_db_router(request.db).bind('')
arguments = dict(request.endpoint_arguments)
for key, val in list(arguments.items()):
try:
# Re-match the controller where the request path routes.
rule, args = self.env['ir.http']._match(request.httprequest.path)
for key, val in list(args.items()):
if isinstance(val, models.BaseModel):
if isinstance(val._uid, RequestUID):
args[key] = val = val.with_user(request.uid)
if val.env.context.get('lang') != lang.code:
arguments[key] = val.with_context(lang=lang.code)
path = router.build(request.endpoint, arguments)
else:
args[key] = val = val.with_context(lang=lang.code)
router = http.root.get_db_router(request.db).bind('')
path = router.build(rule.endpoint, args)
except (NotFound, AccessError):
# The build method returns a quoted URL so convert in this case for consistency.
path = urls.url_quote_plus(request.httprequest.path, safe='/')
lang_path = ('/' + lang.url_code) if lang != self.default_lang_id else ''
canonical_query_string = '?%s' % urls.url_encode(canonical_params) if canonical_params else ''
lang_path = f'/{lang.url_code}' if lang != self.default_lang_id else ''
canonical_query_string = f'?{urls.url_encode(canonical_params)}' if canonical_params else ''
return self.get_base_url() + lang_path + path + canonical_query_string
def _get_canonical_url(self, canonical_params):
"""Returns the canonical URL for the current request."""
self.ensure_one()
return self._get_canonical_url_localized(lang=request.lang, canonical_params=canonical_params)
lang = getattr(request, 'lang', self.env['ir.http']._get_default_lang())
return self._get_canonical_url_localized(lang=lang, canonical_params=canonical_params)
def _is_canonical_url(self, canonical_params):
"""Returns whether the current request URL is canonical."""
@@ -1329,10 +1332,10 @@ class Website(models.Model):
canonical_params = canonical_params or OrderedMultiDict()
if params != canonical_params:
return False
# Compare URL at the first rerouting iteration (if available) because
# it's the one with the language in the path.
# It is important to also test the domain of the current URL.
current_url = request.httprequest.url_root[:-1] + (hasattr(request, 'rerouting') and request.rerouting[0] or request.httprequest.path)
# Compare URL at the first routing iteration because it's the one with
# the language in the path. It is important to also test the domain of
# the current URL.
current_url = request.httprequest.url_root[:-1] + request.httprequest.environ['REQUEST_URI']
canonical_url = self._get_canonical_url_localized(lang=request.lang, canonical_params=None)
# A request path with quotable characters (such as ",") is never
# canonical because request.httprequest.base_url is always unquoted,
addons/website/tools.py
+58 -41
View File
@@ -1,26 +1,72 @@
# -*- encoding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
import contextlib
from lxml import etree
from unittest.mock import Mock, MagicMock, patch
import werkzeug
from werkzeug.exceptions import NotFound
from werkzeug.test import EnvironBuilder
import odoo
from odoo.tools.misc import DotDict
def werkzeugRaiseNotFound(*args, **kwargs):
raise werkzeug.exceptions.NotFound()
from odoo.tests.common import HttpCase, HOST
from odoo.tools.misc import DotDict, frozendict
@contextlib.contextmanager
def MockRequest(
env, *, routing=True, multilang=True,
context=None,
cookies=None, country_code=None, website=None, sale_order_id=None,
website_sale_current_pl=None,
env, *, path='/mockrequest/', routing=True, multilang=True,
context=frozendict(), cookies=frozendict(), country_code=None,
website=None, sale_order_id=None, website_sale_current_pl=None,
):
lang_code = context.get('lang', env.context.get('lang', 'en_US'))
env = env(context=dict(context, lang=lang_code))
request = Mock(
# request
httprequest=Mock(
host='localhost',
path=path,
app=odoo.http.root,
environ=dict(
EnvironBuilder(
path=path,
base_url=HttpCase.base_url()
).get_environ(),
REMOTE_ADDR=HOST,
),
cookies=cookies,
referrer='',
),
type='http',
future_response=odoo.http.FutureResponse(),
params={},
redirect=env['ir.http']._redirect,
session=DotDict(
odoo.http.DEFAULT_SESSION,
geoip={'country_code': country_code},
sale_order_id=sale_order_id,
website_sale_current_pl=website_sale_current_pl,
),
db=None,
env=env,
registry=env.registry,
cr=env.cr,
uid=env.uid,
context=env.context,
lang=env['res.lang']._lang_get(lang_code),
website=website,
)
if website:
request.website_routing = website.id
# The following code mocks match() to return a fake rule with a fake
# 'routing' attribute (routing=True) or to raise a NotFound
# exception (routing=False).
#
# router = odoo.http.root.get_db_router()
# rule, args = router.bind(...).match(path)
# # arg routing is True => rule.endpoint.routing == {...}
# # arg routing is False => NotFound exception
router = MagicMock()
match = router.return_value.bind.return_value.match
if routing:
@@ -30,36 +76,7 @@ def MockRequest(
'multilang': multilang
}
else:
match.side_effect = werkzeugRaiseNotFound
if context is None:
context = {}
lang_code = context.get('lang', env.context.get('lang', 'en_US'))
context.setdefault('lang', lang_code)
request = Mock(
context=context,
db=None,
endpoint=match.return_value[0] if routing else None,
env=env,
httprequest=Mock(
host='localhost',
path='/hello/',
app=odoo.http.root,
environ={'REMOTE_ADDR': '127.0.0.1'},
cookies=cookies or {},
referrer='',
),
lang=env['res.lang']._lang_get(lang_code),
redirect=env['ir.http']._redirect,
session=DotDict(
geoip={'country_code': country_code},
debug=False,
sale_order_id=sale_order_id,
website_sale_current_pl=website_sale_current_pl,
),
website=website
)
match.side_effect = NotFound
with contextlib.ExitStack() as s:
odoo.http._request_stack.push(request)