With the iot-box it is possible to download modules from an odoo server
into the iot-box. Those odoo modules are `exec`-like and exposed on the
box. With those modules come some controllers. Because they were
`exec`-like, they lack a python module name which makes
`_generate_routing_rules` to reject them.
With this fix, we authorise to declare controllers outside of odoo
addons. Those controllers cannot be extended, nor can they override
other controllers.
closesodoo/odoo#86275
Signed-off-by: Julien Castiaux <juc@odoo.com>
* http.py:docstring of odoo.http.route:19: WARNING: Unexpected
indentation.
* http.py:docstring of odoo.http.route:20: WARNING: Block quote ends
without a blank line; unexpected unindent.
* http.py:docstring of odoo.http.route:30: WARNING: undefined label:
csrf
-> No section in the doc exists with reference csrf
* Add docstring for the default_lang method.
closesodoo/odoo#85505
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
=> `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
=> `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.
Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.
Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.
PR: odoo#78857
Task: 2571224
This commit is the 12th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
The web module is twofold, on one side there are many controllers: /,
/web, /web/login, /web/database/selector, /web/dataset/call_kw, etc, on
the other side there is `session_info`: the method responsible to create
the web client's environ.
This module is kinda an exception as it is (with base) a server wide
module. In the case of the HTTP framework, it means that the controllers
of web are always accessible, i.e. going to / or /web/login will never
return a 404 Not Found even if the user is not connected to a database.
This is both a blessing and a curse. It is a blessing because the
controllers are always accessible it means that a new users can freely
access those routes. It is a curse because *any* user can access them,
even user who don't have a session yet thus who are not connected to a
database yet. From a developer standpoint, we have to put extra care to
correct serve users with and without a database. An example is the
/web/login route, the login/password pair is stored in a database,
without database it is impossible to validate a user login but users can
still access this route without db.
To solve this problem, there is the `ensure_db` function. This function
attempts to find a database using various sources (?db= query-string,
session db, mono db) and to save it on the user session. In case no db
is found, the user is redirected to the database selector. In a way,
this function grants a database to the user in a seamingly experience.
In a way, this function brings a welcome differentiation between
`auth='none'` with a database and `auth='none'` without a database. Such
differentiation only matters for the server wide modules as "regular"
module controllers are only accessible via the ir.http routing map, i.e.
it is not possible to declare a nodb controller outside of server wide
modules.
An important changement is the `session.authenticate` method, before it
was possible to call the method when the cursor was not yet initialized,
authenticate would open a cursor against the given database, setup a
registry and an environment and ultimately save everything on the
current request. Because the cursor is now greedily created, it is no
more possible to update the request environment when authenticating on
another database.
PR: odoo#78857
Task: 2571224
This commit is the 11th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals. See also [REF] core: HTTPocalypse (9) ORM initialization.
Where `request._serve_db` is reponsible for initializing the ORM,
`request._serve_ir_http` is responsible to do the match -> authenticate
-> pre dispatch -> dispatch -> post dispatch sequence. This sequence is
very much similar to `_serve_nodb` with the notable addition of the
authenticate step.
Where `_serve_nodb` was working in a db-free environment, calling the
methods of (Http|Json)Dispatcher right away, `_serve_ir_http` delegates
to the `ir.http` abstract model which in turn use the dispatchers.
The ir.http model is crucial for several modules, web, portal and
website on top. They override the various methods to add advenced
capabilities to the http framework.
An important changement to the ir.http model in this work is regarding
the `_dispatch` method. Before this work it was that method that was
responsible of doing the sequence match -> pre dispatch -> ... . It was
decided to move the responsibility to http.py in order to prevent
modules from taking over the entire framework. One such take-over was
done in the `http_routing` addon and resulted in arguably very poor and
fragile code.
All modules must now carefully override the correct method. You need to
do smart stuff on the URL prior to matching an endpoint? You should
override `_match`. You need to save a special option from the query-
strings to the session before calling the controller? Override
`_pre_dispatch`.
PR: odoo#78857
Task: 2571224
This commit is the 10th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
One of the objectives of this comprehensive refactor was to simplify the
error reporting, to show developers shorter and more informative
traceback upon errors.
We argue that the current error reporting is too much bloated:
1. There is a "dual" traceback, one traceback for the frames more recent
than `/base:ir.http_dispatch` (`@route` wrappers + controller) and a
second traceback for the former frames (http framework + web server).
2. We trace too far back, we argue that it is useless to trace back to
the thread/worker creation, that we should only trace back until the
WSGI entry point.
3. There is too much noise, many frames are either implementation
details or useless information. e.g. middlewares, `_dispatch`
overrides, wrappers, "checks".
To solve the above problem we did the following:
1. All errors bubble up to `request.__call__` where it is logged, any
wrapper that catch an error must log it or re-raise it. By logging
exceptions in `__call__`, we ensure the traceback at that function.
2. When one raises an HTTP error, e.g. 404/NotFound, the object is at a
same time an exception and a response. We don't log it and use the
response right way.
3. When one raises an exception, e.g. ValueError, the exception is sent
to `handle_error`, its job is to return the HTTP error corresponding
to the given exception, e.g. 500/InternalError in case of ValueError.
The given exception's `__traceback__` and `__clause__` are not
modified, we log the real exception but use the http error as
response.
Minimalistic example showing a traceback before and after this work:
https://gist.github.com/Julien00859/d6a48d523cac42d0df2e3136f5ba3e54
PR: odoo#78857
Task: 2571224
This commit is the 9th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals. See also [REF] core: HTTPocalypse (11) ir.http base model.
This is a two-part commit with "(11) ir.http base model". In this commit
we focus on the initialization of the various ORM objects, namely: the
registry, the cursor, the environment, the user and the context. In the
other n°11 commit we focus on the ir.http model and its relation to the
current http.py module.
One of the objectives of this comprehensive refactor was to ease the
cognitive complexity of the http framework, in other words to make it
simplier. One of the problem identified quite early during the
preparation of this work is the way the various ORM objects are
initialized, modified and cleaned during the request lifetime.
Before this work, all the ORM internals were lazily initialized via
properties. It is the first time one uses `request.cr` that a cursor is
opened to `request.db` and stored on `request._cr`. It is the first time
one uses `request.env` that an environment is create with the current
`request.user` and `request.context`. Upon user or context modification,
the current environment is discarded, the next usage of `request.env`
will create yet another environment on the fly using the modified user
and/or context.
Using this model, no ressource is initialized if not necessary. It is
possible for nodb-compatible endpoint to be served via the db-compatible
router and ir.http without ever opening a cursor to the database.
But this model is harder to reason about and ultimately to maintain.
In this work we propose to drop the lazy approach for a greedy one. In
this work the first steps of `_serve_db`, the db-compatible counter-part
of `_serve_nodb`, are dedicated to setup a registry, open a cursor to
the database and create an environment using the session's user and
context. In this work, when one wants to change the environ's user or
context, he must call `request.update_env` or `request.update_context`,
both method will recreate the environment *now* with the given values.
The downside of this approach is that resources are always allocated
even when it is not necessary. We argue that, in general, the
controllers that do not use the ORM are rare thus it is rare we allocate
unecessary resources. We also argue that the cognitive benefits are more
than welcome and that the new APIs will help at writing more robust
applications.
PR: odoo#78857
Task: 2571224
This commit is the 8th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
PR: odoo#78857
Task: 2571224
See also: [REF] core: HTTPocalypse (6) serve db-free routes
This commit is the 7th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
PR: odoo#78857
Task: 2571224
See also: [REF] core: HTTPocalypse (6) serve db-free routes
This commit is the 6th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
There are some routes that must work even when the user is not connected
to a database yet. Two obvious examples are the index (/) and the
database selector (/web/database/selector). Accessing those routes
should be possible even when the user is not connected to a database yet,
the `nodb_routing_map` and the `_serve_nodb` method fulfill this need.
Along with the introduction of the `_serve_nodb` method, we also
introduce the match -> pre dispatch -> dispatch -> post dispatch
sequence. `match` matches an endpoint using the http path,
`pre_dispatch` prepares the system, `dispatch` is the actual endpoint
call, `post_dispatch` cleanups the system and add some http headers to
the http response.
At the beginning, when an endpoint is matched, we retrieve the `routing`
dictionnary that is attached to the endpoint method. This dictionnary
contains the information set by the `@route` decorator, among other
values is the routing type `http` or `json`. This routing type is used
to specialize the request via one of the dispatchers: HttpDispatcher for
http and JsonDispatcher for json. Via the dispatchers, it is possible to
add custom behavior in the form of (pre_|post_)dispatch override. One
example of such custom behavior is the way the request body will be
loaded: urllib.parse.parse_qs (http) vs json.loads (json).
Before this work, it was not possible for multiple dispatchers to be
compatible with a same request mimetype. Because `JsonRequest` was
implementing the support for `application/json` (and like) mimetypes,
all requests having a body's mimetype `application/json` were considered
JSON-RPC2 (the protocol implemented by `JsonRequest`) even if the
request was actually bare json data. With the introduction of the
`is_compatible_with` classmethod, multiple dispatchers can be compatible
with a same mimetype and it is up to the `@route(type=...)` param to
define which dispatcher should be used with this route.
PR: odoo#78857
Task: 2571224
This commit is the 5th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
A session is a file that contains user data that must be preserved
across several requests. It is a small (=at most a few kb) file that is
stored on the filesystem in the session store. Each filename is a random
sha-1 string, the same sha-1 is saved in the request's cookies. The
browser has no access to the session, it only possesses the random sha-1
cookie which the server use to retrieve the session upon each request.
It is in the session that the database that the user is connected to is
stored. The first time the user accesses the server the database is
determined using the HTTP Host header of the request plus the list of
available databases and is saved into the user's session. Later requests
merely ensure the database stored in the session is still enabled.
The session "feeling" is the same as before the httpocalypse. The
session is still a DotDict-like structure with methods such as
`authenticate` and `logout`. The difference is the implementation, in
master we were using the werkzeug session, with this work we decided to
get rid of those off-the-shelf sessions to implement our own object.
The major difference between the two implementations is the way we
determine when a session is "dirty", when the session should be written
back on the filesystem. In werkzeug it is a `modified` flag attribute
that is set as soon as `__setitem__` is called, in our implemention we
keep an immutable copy of the original session, at the end the of
request lifecycle we compare the live session to the copy to determine
if it was changed. The former was setting the `modified` flag even when
one was setting the same value as existing in the session. The later
approach is not affected by this limitation.
PR: odoo#78857
Task: 2571224
This commit is the 4th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
Complete refactor of the "registry" of controllers. The `ControllerType`
metaclass have been replaced by an abstract class with a py3.7
`__init_subclass__`. The `Endpoint` class is gone too, replaced by a
clever usage of `functools.partial`. This refactor is "pure", it doesn't
add any new feature, it only merely adds a few warnings.
The four `@route`, `Controller`, `_generate_routing_rule`, `routing_map`
work as follow:
1. A reference to each immediate child class of `Controller` (not grand-
children) is registered in a global list indexed by module (thus a
dictionnary) everytime the server starts. Remember that every first-
child (not grand-children) of `Controller` is the primary controller,
the one that can later be extended by other controllers (the grand-
children) in other modules. Remember that it is possible to get each
class's children via the `__subclasses__` dunder method.
2. Every controller method that is decorated with `@route` is granted an
attribute: `original_routing`, a dictionnary containing the `@route`
arguments. When a controller method has the `original_routing`
attribute, this method is called an `endpoint`.
3. `_generate_routing_rules` receives the list of installed module
names, this list is topologicaly sorted according to the modules
dependencies. That is `base` comes before `web` in this list. The
objectif of this function is to pair each route to an endpoint whoose
class's MRO respect the above-mentioned order. Whe achieve this by
carefully crafting classes at runtime, classes inheritating from the
correct "source code" controllers in accordance to the topology. This
method is also responsible of merging each method's `original_routing`
into one `routing` dictionnary, the very `rule.endpoint.routing` dict
that is used through the rest of the http framework.
4. Each route-endpoint pair is saved into a `routing_map`, an object
that bind each route to its endpoint. This object exposes a `match`
method used to find back the endpoint given its route (=http path).
In addition to this refactor, we added some new helpers in our tools,
among them `submap` that implement a kind of `dict() - set() -> dict()`
operator. Filtering a dict on a set of keys is a common operation but
the python standard library lacks a dedicated operator.
PR: odoo#78857
Task: 2571224
This commit is the 3rd commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
Static files were historically served thanks to a werkzeug middleware,
the usage of that middleware prevented easy modification of the response
headers. Using `http.send_file` and `request._serve_static` we achieve
the same result as the middleware with less code and less frames in the
callstack.
The historic `http.send_file` have been splitted in two. One to send
a file via its path on the filesystem (prefered function) and one to
send a file already open. It is preferred to use `http.send_filepath`.
During development we tried to remove the fallback from the
`_serve_static` function into the more capables (and resource hungry)
db-compatible router. The purpose was to quickly respond to all requests
requesting static files at the cost of reduced customization. We have
been prooved wrong, e.g. the website themes all depends on the fallback
mechanism to deliver static files.
PR: odoo#78857
Task: 2571224
This commit is the 2nd commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
This commit poses the global structure of the new HTTP framework. Major
classes are present (yet empty) with a minimal docstring explaining
their general purpose.
PR: odoo#78857
Task: 2571224
This commit is the 1st commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
Our application framework is quite complicated. There are a few explicit
middlewares: `ProxyFix`, `DisableCacheMiddleware`, `SharedDataMiddleware`.
There are many (13, in total) implicit ones in the form of
`ir.http._dispatch` overrides. There are a few frames that don't have
much values by themselves and are very much implementation details
there are: `application_unproxied`, `_call_function`, `checked_call` and
`EndPoint.__call__`.
The ORM initialisation is quite magical, the cursor, registry and
environment are all setup lazily thanks to class properties. The
context and uid of the environment are aliased on the request. The
abuse of properties makes it impossible to reason about where and when
the environment is actually instantiated and modified.
The dispatch of a request follow the next scheme:
`odoo.http/Root.dispatch` -> `odoo.addons.base.models/ir.http._dispatch`
-> `odoo.http/(Http|Json)Request.dispatch` -> `odoo.http/@route` ->
controller. Because the request becomes http or json specialized quite
late, the many ir.http override all need to parsimony try/except their
code in order to manually call `_handle_exception()` uppon error, they
cannot let the error bubble-up. This back-and-forth between odoo.http
and ir.http is source of some headaches.
Speaking about error, the `odoo.http/WebRequest._handle_exception`
implementation is quite complicated, it basically craft a new exception
out of the passed exception object in order to correct its traceback.
If the error had been bubbled-up instead of handled by
`_handle_exception()` such python hack would not have been necessary.
The objectives of this refactor are about refounding the technical dept:
* We want shorter error reporting;
* We want simpler request and ir.http APIs;
* We want better integration of all the ir.http extensions.
PR: odoo#78857
Task: 2571224
Core client remains incompatible with CSP, however it can't hurt to
CSP the sub-resources.
Current scheme is simplistic, however if useful of necessary it could
be made more flexible e.g. there could be a map of mimetypes to CSP
configuration, that sort of things.
Access /base/static/img/country_flags/fr.png, 404 file not found but it
exists.
The conditional was wrong. We should load static files when the addon is
installable OR that it contains assets.
Code before 2e29a93:
expr = not manifest or (
not manifest.get('installable', True)
and 'asset' not in manifest
)
if expr:
continue
...
Code after 2e29a93:
expr = manifest and (manifest['installable'] or manifest['assets']
if not expr:
...
Proof:
a is manifest
b is manifest.get('installable', True)
c is 'asset' in manifest
expr = ~a | (~b & ~c)
~expr = ~(~a | (~b & ~c)
= a & ~(~b & ~c)
= a & (b | c)
Fine-tuning of 2e29a93closesodoo/odoo#81509
Signed-off-by: Raphael Collet <rco@odoo.com>
The http.addons_manifest is a map {module: manifest_dict} that is
populated upon the first http request. This map is basically a module
manifest cache with an extra `addons_path` key, the path of the module
on the file-system. This cache is eagerly populated upon the first http
request, the map is empty in non-http contextes (e.g. cron) which have
been a source of bugs (e.g. 50c8eb1).
A manifest cache is necessary because reading and parsing python files
from the file-system is not that cheap but there is no reason that cache
is located in `odoo.http`. A thin cache layer now wraps
`load_information_from_description_file()`/`load_manifest()` and is
lazily populated.
The `http.addons_manifest` have been removed. The extra `addons_path`
key is now present in the "normal" manifest. The `read_manifest()` was
hardly used so it has been deprecated. The only way to retrieve a
manifest is now `load_information_from_description_file()` which was
renamed `load_manifest()` (no cache) and `get_manifest()` (cache).
Side note about performances, the cache is necessary. Addons manifest
are read-only and reading + parsing python files from the file system is
not a cheap operation. Running the e-commerce tour
`@website_sale.test_04_admin_website_sale_tour` without cache on
`load_manifest()` requires 68,29 secs to complete on my laptop,
exceeding the default 1-minute time frame allowed in tests. Using a
cache the time is down to 36,53 secs. The performance impact is huge.
Part-of: odoo/odoo#79977
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
fields) are out of scope for the lint but my editor catches
closesodoo/odoo#74604
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
With the introduction of the guest feature, a lot of fields are now
accessed through a controller so that they can be available to guests,
with additional checking in the controllers.
Avatar is one such field, however the extra checks were too restrictive:
we only allowed people to see the avatar if the user whose avatar was
requested was also a member of the channel. This breaks the avatar on
previous messages if a user leaves a channel.
This commit relaxes this restriction for internal users (they can now
see all avatars through this route), and adds a fallback to the avatar
placeholder for people who do not have acces (ie: guests) so that the
UI doesn't look broken.
closesodoo/odoo#76341
X-original-commit: b849fd426806332c1ecc1bbba7b568384a021c65
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Despite that Werkzeug documentation specify:
'location (str) – the location the response should redirect to.'
Werkzeug support URL as location.
So now Odoo will support URL for request.redirect as argument too.
This commit closes#73729
Introduce the env variable ODOO_DISABLE_SESSION_GC to disable session_gc
on high-volume systems and avoid random slow calls as discussed in
https://github.com/odoo/odoo/pull/70063closesodoo/odoo#73376
X-original-commit: a61d459a200a56fba74f655f73b10d1ddd8e3a1e
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
This branch adds request.redirect on all requests.
In case of a front end request, we do an url_for to the location.
We removed redirect_with_hash that was only for retro compatibility
local_redirect has been renamed to redirect_query, and param keep_hash has been
removed and moved.
Default code for redirect is 303 now instead of 302.
Now redirect and redirect_query make local redirect by default, you need to
pass local=False to make external redirect.
All werkeug.utils.redirect has been replaced by request.redirect.
Http.redirect now use an http.Response type, and it become easy to add an
override like 'set_cookies' e.g.
Dispatch of a website.page return an http.response too, so we first need to
check if it is a cached version before to check if it is an Odoo Response.
Migrate your code:
http.redirect -> request.redirect(location, code, local)
http.local_redirect -> request.redirect_query(location, query, code, local)
http.redirect_with_hash -> request.redirect
Courtesy of odony for help and review ;)
closesodoo/odoo#72599
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
The usage of an additional optional context manager leads to the usage
of ExitStack. Unfortunately, this changes the initial format quite a
lot, decreasing readability and adding a loop on context managers, even
if there should be only one most of the time (request).
This commit proposes to a nesting utility for the profiler, allowing to
nest another context manager inside a profiler.
This solution allows an easier integration into http.py, avoids to
manage the "enter_context" case when recovering the init stack and can
be useful in other cases.
Improvement of get_dispatch_context_managers conditions and styling:
- Check profile expiration very first.
- Add an additional check for odoo.evented. Profiling will crash on an
evented server. Only longpolling should use evented servers, it but
this is more future proof.
The common mistake of method instead of method leads to the parameter being ignored.
Add a warning to ensure no other controllers are defined this way
closesodoo/odoo#71571
X-original-commit: d0048ef115985a24eec1e47ceb146f4a98f71b38
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
The profiling tools can be useful to profile a test of some execution
point but this is not convenient to identify a problem on a running
instance.
With this commit, an option available in the debug menu allows to add a
flag on the user sessions to enable profiling of all requests. Each
request will be saved in a different 'ir.profile' entry, but will be
grouped under the same session.
The profiling can be activated on all sessions, even for a public user,
but only if profiling is enabled on the database globally.
This commits also adds a speedscope view to visualize saved results in
the web client.
closesodoo/odoo#66590
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Following the recent reorganisation of the documentation in 12.0+,
the majority of the documents have been moved and their old links are no longer valid.
Some redirection rules will soon be deployed, but those rules might be dropped in some years
and we want the links to still work, which is why we still replace the links to the new ones.
FW-Port of odoo/odoo#70675 (13.0)
closesodoo/odoo#70920
X-original-commit: bc9c1eef538ba6095e74c19d5d9ed9e01625ec7c
Related: odoo/enterprise#18361
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
This commit changes the way assets are declared in Odoo modules.
Before: assets were declared in template files. Template bundles were
generated from primary templates, so technically any qweb template could
have been called as an asset bundle, with the 't-call-assets' directive.
Being standard qweb templates, they had access to standard HTML tags
(script, link, with or without raw scripts or style definition), qweb
directives (t-call, t-raw, etc.) and could be inherited by other
templates.
Now: assets are defined in the module's manifest and generated by the
't-call-assets' directive.
More information on the new system can be found on the updated user
documentation (see the "JavaScript Reference" section).
Task: 2352566
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
Improve the development experience in debug=assets mode by reducing the
number of requests to the server. We are adapting the solution used for
the JS files to the CSS files. This solution consists of no longer
sending all the files separately, but sending only the bundles
associated with their sourcemap. This allows us to keep the same
debugging experience while drastically reducing the number of requests
to the server.
Benchmark:
saas 14.2 917 requests domcontentloaded after 3.76s
master (bundling du js) 299 requests domcontentloaded after 2.03s
branch (bundling js+css) 36 requests domcontentloaded after 1.01s
Task id : 2463840
closesodoo/odoo#66169
Related: odoo/design-themes#453
Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
Because of the way Odoo works at its core, we do not know before hand
which files will be loaded as an asset in the browser, because it
depends on the installed Odoo addons. This is why it is historically
difficult to integrate Odoo with standard JS tooling, and this is why
Odoo needs to use a custom javascript module system.
However, there is a way to use native JS modules (and gain all the
benefits from it: IDE autocompletion, ease of refactoring, intellisense,
...): we can write JS as native JS modules, but convert them at runtime
into Odoo custom modules. This is exactly the strategy applied by this
PR.
This has a lot of benefits, but there is a downside: we can no longer
serve statically JS files in debug=assets. This would be a dealbreaker,
if we did not have sourcemaps (implemented in the next commit).
This commit introduces the python code that will transpile native JS
modules into odoo JS modules.
Task ID: 2414902
PR: 63177
Co-authored-by: Francois (fge) <fge@odoo.com>
The mobile app's authentication bases its feedback to the user on the
error message returned by the server.
Sadly, in case of multi-databases, no error message are properly
returned when the user provides wrong credentials (presenting only an
empty Snackbar).
This difference of behaviour was introduced in the commit
odoo/odoo@07d5ea3 which goals was to give more power for JsonRequest's
error handling customization. To do so, the catching of the exception
(if one happens) was delegated to the caller instead of the JsonRequest
itself.
This change indirectly introduced a difference about the treatment of an
exception between mono and multi-database :
* mono-database was already catching the exception and handling it
properly (see in
https://github.com/odoo/odoo/blob/f05f8ef647a4eaba49d8a20eb8068f1c7a702297/odoo/addons/base/models/ir_http.py#L235-L241).
* multi-databases didn't catch it.
This commit restores the parity between both use cases by applying the
same behaviour (as described in the commit odoo/odoo@07d5ea3):
```python
ir_http.dispatch():
try:
request.dispatch()
catch Exception as e:
ir_http._handle_exception(e)
ir_http.handle_exception(e)
request._handle_exception(e)
```
Steps to reproduce:
1. Open the mobile app;
2. Try to register an account with a wrong password with multiple
databases;
3. The snackbar is empty and no error is shown => bug;
Task ID: 2287176
closesodoo/odoo#64965
X-original-commit: f54c97f6214c235bac931184c6e533a7fa72fbe2
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
XML loading always raises a ParseError, however when an XML loading error is triggered from an HTTP request (e.g. a button) the final ParseError logged & shown to the user would not be properly chained to its ancestor, leading the original cause to be lost and issues being much harder to diagnose.
This cause is lost in _handle_exception where we duplicate the exception in order to chain it correctly, the __cause__ of the source exception was not properly copied over, and would thus break the chain.
The fix also requires explicitly chaining the ParseError to its cause, this should not be necessary but the cause is also lost if this is missing, it is not clear why.
opw-2381776
closesodoo/odoo#62117
X-original-commit: 650476812ee7d57f1d954be5557a3856448ffe5e
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this commit, links to the documentation were referenced the
previous version, 13.0, instead of the current one, 14.0.
Eventhough there is a redirection done by NGINX of a "versionless" URL
to the latest one (e.g. /documentation/user/general/auth/google.html
-> /documentation/user/14.0/general/auth/google.html as of today), the
goal is to keep links owrking for users that will still be using the
14.0 in three years (and should not endup on the 17.0 doc).
closesodoo/odoo#60228
X-original-commit: 7ac08486d91d0ff0151abeeda057ffa6beda72e8
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
1af543a399 removed the default time limit
for CSRF tokens, because of the usability issues and the limited
security benefit.
However the timestamp that was used to implement the limit also served as
a salt, making the CSRF token variable for each request. This is a
desirable property that can help mitigate some attacks, such as BREACH.
This patch re-introduces the variability by including a distant expiry
(1 year) when no specific time limit is passed. The purpose isn't to
expire the token, but simply to serve as a salt.
closesodoo/odoo#57395
X-original-commit: 136e4f66cd5cafe7df450514937c7218c7216c93
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Python 3.8 changed the equality rules for bound methods to be based on
the *identity* of the receiver (`__self__`) rather than its *equality*.
This means that in 3.7, methods from different instances will compare
(and hash) equal, thereby landing in the same map "slot", but that isn't
the case in 3.8.
While it's usually not relevant, it's an issue for `GroupCalls` which is
indexed by a function: in 3.7, that being a method from recordsets
comparing equal will deduplicate them, but not anymore in 3.8, leading
to duplicated callbacks (exactly the thing GroupCalls aims to avoid).
Also, the API of `GroupCalls` turned out to be unusual and weird. The
bug above is fixed by using a plain list for callbacks, thereby avoiding
comparisons between registered functions. The API is now:
callbacks.add(func) # add func to callbacks
callbacks.run() # run all callbacks in addition order
callbacks.clear() # remove all callbacks
In order to handle aggregated data, the `callbacks` object provides a
dictionary `callbacks.data` that any callback function can freely use.
For the sake of consistency, the `callbacks.data` dict is automatically
cleared upon execution of callbacks.
Discovered by @william-andre
Related to odoo#56583
References:
* https://bugs.python.org/issue1617161
* python/cpython#7848
* https://docs.python.org/3/whatsnew/changelog.html#python-3-8-0-alpha-1
(no direct link because individual entries are not linkable, look for
bpo-1617161)
X-original-commit: d4b2e9224839aed8fc160ebe5a89e0f7d4c6a5bb
This module provide routes to manage people, companies, and
leads from the outlook add-on. It could eventually be accessed by
add-ons for other mail clients.
closesodoo/odoo#44936
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Odoo provides basic handling of CORS preflight requests: if an
endpoint is marked as `cors=<truthy value>` then it'll automatically
reply allowing the request.
*However* this is performed in `HttpRequest.dispatch` (likely in order
to correctly handle the nodb case), which means it's executed after
the auth handler has run... which means custom auth handlers will be
called on preflight requests.
This is a problem because they are missing relevant
information (e.g. which endpoint they're invoked for), plus having to
deal with preflight requests in every custom auth handler is annoying,
and simply allowing preflights could cause issues if the decision
diverges between the auth handler and the automatic
handling.
To fix this issue, extract the preflight *decision* into a separate
method so we get the same decision-making process everywhere, and in
case of CORS preflight set the auth to none to limit the eventual
capacity for nuisance in the span between the bypassed auth and the
automated preflight handling.
Also change the signature of IrHttp._authenticate so it's clearer if a
callsite was forgotten somehow (and this makes for less changes and
duplication at the callsites).
closesodoo/odoo#56029
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Missed while merging the 2FA: in a mono-db scenario, the request is
bound to a database more or less immediately; however in a multi-db
scenario the current request *may* not be bound to a database by the
time we reach session authentication, this binding is performed *by*
the authentication step[0].
By trying to access an environment (requiring a db) before this
binding is performed, the authentication procedure broke.
We could create the environment by hand, but it seems completely
unnecessary: while there is a bundle of operations to perform in all
cases and a bundle to perform during finalization, they don't seem to
strictly depend on one another (and indeed this separation is already
a reordering of the operations from before), so the finalization can
be performed *after* having bound the current session to a database.
In fact that is closer to the older order: in the pre-totp iteration,
only the binding of the uid was performed before that of the login,
and even it was done after binding the database.
[0] this really is mostly a concern when using this step
programmatically, as interactively the database selector should
require selecting (and binding) a database before it's possible
to log-in
closesodoo/odoo#56093
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
New module for supporting two-factor authentication via time-base
one-time-password (TOTP).
Users (including portal users) can choose to enable two-factor auth in
their user account settings, by scanning a QR code and adding it to an
authenticator app, such as Google Auth, 1Password, etc.
When two-factor is enabled, password-based non-interactive RPC is only
possible by using API keys.
Co-authored-by: Olivier Dony <odo@odoo.com>
Allows accessing various keys, especially whether this is an
interactive login or not.
Also have the xml-rpc `login` delegate to `authenticate` instead of
having its own half-assed implementation.
And remove some dead code: as far as I can tell, Session.authenticate
is never called with a uid.
Our CSRF tokens are based on the current user session, and automatically
expire as soon as the session does.
However, they also come with a default 1h expiration delay. This proves
to be a frequent annoyance for users who pause more than 1h on a form
before submitting it (e.g. user logs out and browser sits on login page
until the next day).
It can even lead to blocking bugs, e.g. when the 1h expiration occurs in the
middle of taking a survey exam, and the user is never able to post the
answers that are only present in the state of the form they need to
post.
More generally, users have a hard time understanding those CSRF expiration
errors, and don't know how to react.
Longer default expiration times have been considered (e.g. 1 day or
1 week) but those would not bring any identified benefit in terms of
security, while still giving a chance that some users would experience
the incomprehensible HTTP 400 errors).
Attacks that can typically compromise the CSRF token (XSS, RCE)
can achieve as much, or more, on the system or user account than what is
possible with the token. And nothing generally prevents the attacker
from using the token immediately after capturing it, during the initial
attack, making the expiration delay rather irrelevant.
Given there seems to be no significant benefit in expiring the tokens
before the session itself, let's just keep them valid as long as the
session.
Note: sessions are GC'd automatically after 7 days of inactivity,
which gives an effective 1 week expiry for abandoned web forms anyway,
as the token expires with the session.
Additionally, fix `survey` module tests, that were using an incorrect
regex for extracting CSRF tokens.
closesodoo/odoo#51499
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
DeferredException was removed in
ab4000fb3c but this specific use was
apparently missed.
closesodoo/odoo#53657
X-original-commit: 0de46bc231622d5b7e4c3e3afeaece6767426af2
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This reverts commit c43647f085a7f62c9c81db6553be6a6e402943d0.
That change was not tested properly and can cause unforeseen errors
because it has far-reaching consequences, modifying the fallback
language on all requests.
One of the consequences is an alteration of the behavior of the
translation function `_()` due to the absence of a default language. For
users with no language set, it will now translate False/None values as
False/None, rather than the empty string fallback. Code that was not
prepared to deal with those non-str translations will now crash.
Besides, 'en_US' is a hardcoded default used in many areas of the code,
and we cannot get rid of it like this, especially in a stable series.
Cfr #52758closesodoo/odoo#53273
X-original-commit: e220c5a28ecbd160be86b0ce00a7f17bcfaa816e
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Julien Castiaux