Before this commit, links to the documentation were referenced the
previous version, 13.0, instead of the current one, 14.0.
Eventhough there is a redirection done by NGINX of a "versionless" URL
to the latest one (e.g. /documentation/user/general/auth/google.html
-> /documentation/user/14.0/general/auth/google.html as of today), the
goal is to keep links owrking for users that will still be using the
14.0 in three years (and should not endup on the 17.0 doc).
closesodoo/odoo#60228
X-original-commit: 7ac08486d91d0ff0151abeeda057ffa6beda72e8
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
1af543a399 removed the default time limit
for CSRF tokens, because of the usability issues and the limited
security benefit.
However the timestamp that was used to implement the limit also served as
a salt, making the CSRF token variable for each request. This is a
desirable property that can help mitigate some attacks, such as BREACH.
This patch re-introduces the variability by including a distant expiry
(1 year) when no specific time limit is passed. The purpose isn't to
expire the token, but simply to serve as a salt.
closesodoo/odoo#57395
X-original-commit: 136e4f66cd5cafe7df450514937c7218c7216c93
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Python 3.8 changed the equality rules for bound methods to be based on
the *identity* of the receiver (`__self__`) rather than its *equality*.
This means that in 3.7, methods from different instances will compare
(and hash) equal, thereby landing in the same map "slot", but that isn't
the case in 3.8.
While it's usually not relevant, it's an issue for `GroupCalls` which is
indexed by a function: in 3.7, that being a method from recordsets
comparing equal will deduplicate them, but not anymore in 3.8, leading
to duplicated callbacks (exactly the thing GroupCalls aims to avoid).
Also, the API of `GroupCalls` turned out to be unusual and weird. The
bug above is fixed by using a plain list for callbacks, thereby avoiding
comparisons between registered functions. The API is now:
callbacks.add(func) # add func to callbacks
callbacks.run() # run all callbacks in addition order
callbacks.clear() # remove all callbacks
In order to handle aggregated data, the `callbacks` object provides a
dictionary `callbacks.data` that any callback function can freely use.
For the sake of consistency, the `callbacks.data` dict is automatically
cleared upon execution of callbacks.
Discovered by @william-andre
Related to odoo#56583
References:
* https://bugs.python.org/issue1617161
* python/cpython#7848
* https://docs.python.org/3/whatsnew/changelog.html#python-3-8-0-alpha-1
(no direct link because individual entries are not linkable, look for
bpo-1617161)
X-original-commit: d4b2e9224839aed8fc160ebe5a89e0f7d4c6a5bb
This module provide routes to manage people, companies, and
leads from the outlook add-on. It could eventually be accessed by
add-ons for other mail clients.
closesodoo/odoo#44936
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Odoo provides basic handling of CORS preflight requests: if an
endpoint is marked as `cors=<truthy value>` then it'll automatically
reply allowing the request.
*However* this is performed in `HttpRequest.dispatch` (likely in order
to correctly handle the nodb case), which means it's executed after
the auth handler has run... which means custom auth handlers will be
called on preflight requests.
This is a problem because they are missing relevant
information (e.g. which endpoint they're invoked for), plus having to
deal with preflight requests in every custom auth handler is annoying,
and simply allowing preflights could cause issues if the decision
diverges between the auth handler and the automatic
handling.
To fix this issue, extract the preflight *decision* into a separate
method so we get the same decision-making process everywhere, and in
case of CORS preflight set the auth to none to limit the eventual
capacity for nuisance in the span between the bypassed auth and the
automated preflight handling.
Also change the signature of IrHttp._authenticate so it's clearer if a
callsite was forgotten somehow (and this makes for less changes and
duplication at the callsites).
closesodoo/odoo#56029
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Missed while merging the 2FA: in a mono-db scenario, the request is
bound to a database more or less immediately; however in a multi-db
scenario the current request *may* not be bound to a database by the
time we reach session authentication, this binding is performed *by*
the authentication step[0].
By trying to access an environment (requiring a db) before this
binding is performed, the authentication procedure broke.
We could create the environment by hand, but it seems completely
unnecessary: while there is a bundle of operations to perform in all
cases and a bundle to perform during finalization, they don't seem to
strictly depend on one another (and indeed this separation is already
a reordering of the operations from before), so the finalization can
be performed *after* having bound the current session to a database.
In fact that is closer to the older order: in the pre-totp iteration,
only the binding of the uid was performed before that of the login,
and even it was done after binding the database.
[0] this really is mostly a concern when using this step
programmatically, as interactively the database selector should
require selecting (and binding) a database before it's possible
to log-in
closesodoo/odoo#56093
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
New module for supporting two-factor authentication via time-base
one-time-password (TOTP).
Users (including portal users) can choose to enable two-factor auth in
their user account settings, by scanning a QR code and adding it to an
authenticator app, such as Google Auth, 1Password, etc.
When two-factor is enabled, password-based non-interactive RPC is only
possible by using API keys.
Co-authored-by: Olivier Dony <odo@odoo.com>
Allows accessing various keys, especially whether this is an
interactive login or not.
Also have the xml-rpc `login` delegate to `authenticate` instead of
having its own half-assed implementation.
And remove some dead code: as far as I can tell, Session.authenticate
is never called with a uid.
Our CSRF tokens are based on the current user session, and automatically
expire as soon as the session does.
However, they also come with a default 1h expiration delay. This proves
to be a frequent annoyance for users who pause more than 1h on a form
before submitting it (e.g. user logs out and browser sits on login page
until the next day).
It can even lead to blocking bugs, e.g. when the 1h expiration occurs in the
middle of taking a survey exam, and the user is never able to post the
answers that are only present in the state of the form they need to
post.
More generally, users have a hard time understanding those CSRF expiration
errors, and don't know how to react.
Longer default expiration times have been considered (e.g. 1 day or
1 week) but those would not bring any identified benefit in terms of
security, while still giving a chance that some users would experience
the incomprehensible HTTP 400 errors).
Attacks that can typically compromise the CSRF token (XSS, RCE)
can achieve as much, or more, on the system or user account than what is
possible with the token. And nothing generally prevents the attacker
from using the token immediately after capturing it, during the initial
attack, making the expiration delay rather irrelevant.
Given there seems to be no significant benefit in expiring the tokens
before the session itself, let's just keep them valid as long as the
session.
Note: sessions are GC'd automatically after 7 days of inactivity,
which gives an effective 1 week expiry for abandoned web forms anyway,
as the token expires with the session.
Additionally, fix `survey` module tests, that were using an incorrect
regex for extracting CSRF tokens.
closesodoo/odoo#51499
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
DeferredException was removed in
ab4000fb3c but this specific use was
apparently missed.
closesodoo/odoo#53657
X-original-commit: 0de46bc231622d5b7e4c3e3afeaece6767426af2
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This reverts commit c43647f085a7f62c9c81db6553be6a6e402943d0.
That change was not tested properly and can cause unforeseen errors
because it has far-reaching consequences, modifying the fallback
language on all requests.
One of the consequences is an alteration of the behavior of the
translation function `_()` due to the absence of a default language. For
users with no language set, it will now translate False/None values as
False/None, rather than the empty string fallback. Code that was not
prepared to deal with those non-str translations will now crash.
Besides, 'en_US' is a hardcoded default used in many areas of the code,
and we cannot get rid of it like this, especially in a stable series.
Cfr #52758closesodoo/odoo#53273
X-original-commit: e220c5a28ecbd160be86b0ce00a7f17bcfaa816e
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Steps to reproduce the bug:
- Let's consider a new instance with default installed language 'en_US'
- Install CRM
- Activate a second language (e.g. en_GB)
- Set that language in all users
- Inactivate default language 'en_US'
- Reset the language of your current user (no value)
- Go to contact and try to create a new one
Bug:
A traceback was raised because the lang en_US did not exist.
opw:2267711
closesodoo/odoo#53064
X-original-commit: c43647f085a7f62c9c81db6553be6a6e402943d0
Signed-off-by: Simon Goffin (sig) <sig@openerp.com>
render, render_template, load, activity_schedule_with_view,
get_website_pages should all be private:
It should not be possible to render an aribtrary template only with
its name or id
Still need to render some qweb views from js so the method
render_template is kept public.
This explains why the website editor still need read access on
ir.ui.view as we want to allow any snippet to be rendered.
Werkzeug 1.0 adds a `merge_slashes` feature which calls rule.build()
*during match*[0] (= during dispatch).
However at this point our environment still contains an invalid /
placeholder UID, so `display_name` / `name_get()` fails dramatically.
One possibility would be to update the slugifying to not access the
record (outside of the id) if the environment is not "proper", an
other alternative is to just disable the feature since it seems to not
have been necessary so far.
The latter seems less hacky so do that for now, we can always swap the
solution later if we need to.
[0] https://github.com/pallets/werkzeug/blame/048cdfd9b969c0c3a133d7ff43b8ad1ad6a673ec/src/werkzeug/routing.py#L904
Those were deprecations implemented in 0.15
* all middlewares have been moved from `werkzeug.wsgi` to
`werkzeug.middleware`, including the `SharedDataMiddleware` we use
* ProxyFix was moved to werkzeug.middleware.proxy_fix, this had
already been fixed but I forgot the import
* sessions support was moved to a separate package
(`pallets/secure-cookies`), however while distros are starting to
update werkzeug to 1.0 (e.g. done on Arch, and in Debian
Experimental) they're not bundling secure-cookies so using a
vendored version seems like the least bad thing we can do, even more
so as conditional dependencies are not really a thing (e.g. even
with just pip we can't depend on secure-cookie iff werkzeug >= 1.0)
Before this commit if the loading of the registry failed because of an
AttributeError or a psycopg2 error the http dispatcher would redirect
the user to the database manager.
This can be problematic because integrators (e.g. odoo.sh) may choose to
disable / forbid access to the database manager, and when the registry
crashes because of e.g. a migration, the real error will be overshadowed
by an AccessDenied error or somesuch depending on the path taken to
forbid access to the database manager.
With this commit, the real exception is simply reraised
closesodoo/odoo#49238
X-original-commit: de4e67dcc52916337251370387aea6aea893a60e
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
TL;DR: remember `osv` and `except_orm` ? You can forget about them.
* Deprecated `except_orm` dropped.
* `UserError` elevated as super type of all user-related
errors.
* Unused `DeferredException` dropped.
* Unused `QWebException` dropped (real one is in `qweb.py`).
* `MailDeliveryException` made a python exception.
* `name` legacy exception attribute made an alias of the python standard
`args[0]` attribute and deprecated.
* `value` legacy exception attribute dropped.
* `exception_type` RPC error response key dropped.
* Deprecated `osv` module dropped.
* `--osv-memory-age-limit` cli option made an alias of
`--transient-age-limit` and deprecated.
The `odoo.exceptions.Warning` have long been a deprecated alias to
`UserError`. It is going to be removed in a future version but first we
explicitly deprecate it with a warning.
The `odoo.exceptions.DeferredException` was a very old internal
exception, it has been removed without deprecation notice as it is never
raised.
The `odoo.exceptions.except_orm` has been a deprecated exception type
with deprecation warning for 5 years, it has been removed in favor of
UserError which becomes the super class of all user-related errors.
The `odoo.base.models.ir_mail_server.MailDeliveryException` was
inheriting `except_orm`. As it is not related to a user error but is
more of a problem an admin much take care of, the exception has been
made a Python error.
The `exception_type` JSON key in RPC error responses was holding an
hardcoded value derived from the exception type. Its usage has been
dropped in favor of the `name` JSON key that holds the precise exception
name. Again as it was hardly used in the source code (beside the crash
manager) it has been dropped without deprecation warning.
Since we are here trying to clean odoo custom exceptions, we are also
deprecating the `name` exception attribute in favor of the more standard
`args[0]` attribute.
The `name` (along with `value`) were two attributes used to raise
`except_orm` exceptions before the introduction of `UserError`,
`AccessError` and related exceptions. The `name` attribute, at the time,
was holding the exception type/title. Nowadays it contains the error
message. The `value` attribute, at the time, was holding the error
message. Nowadays it is no more used.
The `osv` module contains very old deprecated aliases. There is no
simple way to log a deprecation warning for osv, osv_memory and
osv_abstract but as they have not been in use for ages, they have been
removed too. To be consistent, the `--osv-memory-age-limit` cli option
has been made a deprecated alias to the `--transient-age-limit`.
closesodoo/odoo#45723
Task: 2187728
Related: odoo/enterprise#9162
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When a serialization error occurs in a given code context, the call is
retried. However, currently the environment is flushed outside this
context, which may cause a serialization error that is not handled by a
retry. Move the flush inside the checked call to catch all
serialization errors, and handle them properly.
X-original-commit: 6197bfb3b2d51873416780c96cd4cb538a1f219c
The idea is to make the dispatching and error-handling consistent for
JsonRequest and HTTPRequest.
In 8809c77f60 we introduced a way for
request-specific error-handling, but JsonRequest.dispatch() was
still catching all errors internally, instead of letting them bubble
up to ir_http._handle_exception()
By removing the internal try except in JsonRequest.dispatch(), we do not
change the behavior much as there are only a couple of "if" in
ir_http._handle_exception() before returning to the request-specific
handle_exception. However we give the opportunity to modules to
customize exception handling even for JsonRPC, the same way it is
possible for HTTP requests.
Here is a simple pseudo code of the flow
ir_http.dispatch():
try:
request.dispatch()
catch Exception as e:
ir_http._handle_exception(e)
ir_http.handle_exception(e)
request._handle_exception(e)
closesodoo/odoo#45639
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
In 0.15 accessing werkzeug.urls functions directly through werkzeug
is deprecated, the shortcut will be removed in the eventual werkzeug
1.0.
Fix existing uses of these shortcuts. Also cleanup some imports when
they're not far from a werkzeug* import being altered.
If an automated action raises an exception, in the traceback modal:
- For admins, display Disable & Edit automated action buttons
to be able to directly know with wihch automated action the error occurred,
and to give the possibility to edit or disable it quickly,
- For regular users, just add a paragraph to tell with which automated action the error occurred,
so they can give this useful information to their administrator
This is specially useful for databases which have just been
upgraded to a newer version, and for which the server action
is failing because its code is no longer supported.
closesodoo/odoo#44513
X-original-commit: 8f3940c132bbbc99e47fa3f0cba0e768159d9af2
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Some code modify return of res.users().context_get, but this is a
cached method so this will unexpectedly affects totally unrelated code.
For example, changing the company with the company switcher could add
`allowed_company_ids` inside the cache, then it will be cached until the
server is restarted, even if we change company again inbetween.
Added test failed with:
"NotImplementedError: '__setitem__' not supported on frozendict"
on the line with `User = User.with_context(context)` where User already
contained `allowed_company_ids` in its context.
note:
in this forward-port, context_get is also changed to return frozendict
and prevent being able to have an unexpected issue by code that modify
context_get returns.
opw-2158340
closes#42465closesodoo/odoo#42723
X-original-commit: 5d69885c1cd6921b3de00aae7e0ed6fff243ff95
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
odoo/odoo#28519 removed large parts of pycompat, but left reraise
despite that not having much value.
Remove that helper and replace it by just a `raise` in most cases:
when raising from an except block, the old exception is automatically
chained to the new one, no need to mess around.
There is one exception: in http we have to re-raise an existing
exception explicitly (aka `raise exc` rather than just `raise).
This is less than ideal as Python *concatenates* stacks: the
previously reified stack (from the except clause) is stacked on top of
the new stack (from this raises), this leads to tracebacks "jumping
around" at the break point of the handler and is somewhat confusing.
So we want to use explicit chaining (`raise a from b`) with the
"source" providing the caught exception's original traceback and the
child providing the rest.
However since callers rely on the exception making sense, we need the
re-raised exception to be the original[0]. Copying the exception
doesn't work (see [0]), chaining an exception to itself doesn't
do anything useful, and while we could probably copy exceptions using
the pickle method[1] that's still risky.
So the most reliable option seems to be to create a new "cause"
exception, move the old traceback over to it, then re-raise the
original exception having cleared its traceback, chained to new the
cause.
[0] or a copy thereof but Odoo exceptions don't all work properly with
copy.copy and we don't want that to fail so not really an option,
we can't rely / bet on every new exception being cleanly copy-able
[1] create an "empty" instance using __new__ (or an instance of
something else onto which we re-set the __class__ in case the exctype
actually overrides __new__) then copy the __dict__
closesodoo/odoo#39709
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Activate developer mode, generate a report (print an invoice)
The report will have missing pieces, like the footer or some part of the
header. This is probably caused by wkhtmltopdf not loading properly some
resources
Wkhtmtopdf generate the same warning message for every problematic resource:
"Warning: Received createRequest signal on a disposed ResourceObject's
NetworkAccessManager. This might be an indication of an iframe taking
too long to load."
Related issue on wkhtmltopdf project page:
wkhtmltopdf/wkhtmltopdf#1865wkhtmltopdf/wkhtmltopdf#3933wkhtmltopdf/wkhtmltopdf#2565
The problem is located in the response that wkhtmltopdf receive:
in debug mode the header of the response contains
'Cache-Control: no-cache' which probably create a race condition during
the rendering while a second request is attempted to verify the
resources.
Adding a raw user agent check to not include this header directive
fix the problem
Notes from odony:
We've considered some alternative solutions to preserve the purpose of the
DisableCacheMiddleware without having to explicitly test for wkhtmltopdf.
* 'Cache-Control: no-cache' (current behavior) breaks wkhtmltopdf rendering
* 'Cache-Control: no-store' breaks wkhtmltopdf rendering too
* 'Cache-Control: max-age=0' breaks wkhtmltopdf rendering too. It works
when increasing the delay to a few seconds, but no magic value will work
for very long documents, or it will stop serving its purpose, so it's not a
viable option.
* 'Cache-Control: must-revalidate' does not break wkhtmltopdf rendering (no
duplicate requests at all), but it is not clear from the RFC
(https://tools.ietf.org/html/rfc7234#section-5.2.2.1) that it
will have the intended effect for our middlewar
opw-2086708
Closes#38394closesodoo/odoo#39634
X-original-commit: 8cac60be37133cabef46dab016a5692876da9e5e
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
After this commit, you will be able (in technical mode) to update the url for
the python controllers.
Eg.
You can now rename /shop in /garden and /shop/product/ in /garden/vegetable/
Most of urls will be replaced at fly in the renderd qweb, with the function
url_for but all old urls will keep available. So if you access url /shop you
will be automatically redirected to /garden (308 Permanent Redirect).
As for cdn and other post-process of att, the automatically replacement in the
rendered qweb is only done when you will be not website editor. But the new
dispatch of URL will be applied in all cases.
For developper, since it is Permanent Redirect, don't forget to clear cache or
open chrome debug tool (with option 'Disable cache while DevTools is Open) to
see your lasts changes.
closesodoo/odoo#36555
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Section 7.1.2 of RFC 7231 requires preservation of URL fragment
through redirects. Our old JS redirection code was necessary when
browsers did not consistently implement it. Apart from a few odd
and marginal exceptions they all do it now, so we can stop that.
* base, web
Google now recommends a TTL of one year for static contents. We used to
use 1 week in almost every case. This commit increases that value to one
year for safe resources, like assets bundles which contain a specific
hash in the URL which changes if the bundle is recomputed anyway.
Note: this commit refactors the code so that both the one week and one
year durations are defined in http.py and used by others apps. Loading
the library "locale" file used to be done with 10-hours-cache, this has
been increased to 1-week-cache by using the http.py STATIC_CACHE var.
closesodoo/odoo#37402
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The rollback clears the cache, which lose all data that have been fetched before
arriving in the route method.
This lost cache includes some website data that was used during the dispatch and
that will be used again in the route.
By keeping it we reduce the number of queries on every request by at least 2.
Part of task-2061122
closesodoo/odoo#36245
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
* = stock, test_website, web, website_forum, website_slides, base
Replace KarmaError with AccessError and remove the related override made
on crash_manager and ir_http.
task-2069890
closesodoo/odoo#36655
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
With this commit it is now possible to change the lang displayed in the URL.
Eg, you could use `/fr` instead of `/fr_BE`, or even a fancier `/french`.
Task-32838
Courtesy of pla@odoo.comclosesodoo/odoo#35135
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
* = gamification, test_website, stock
- KarmaError is now handled as a 400 exception.
- test_website has been updated.
- NO_POSTMORTEM is now clean it was referencing duplicates as most the
exceptions inherit from except_orm.
- serialize_exception from http.py has been moved to ir_http
to take advantage of the odoo inheritance system. We can then
extend ir_http serialize_exception method to add the KarmaError
logic if and only if gamification is installed.
Places where serialize_exception was previously used are updated.
Part of https://github.com/odoo/odoo/pull/32132
task-1894820
* = account, iap, point_of_sale
Removing the CrashManager templates from base.xml and moving
it to a new file (crash_manager.xml) to allow lazy loading the required
xml only when a crash occurs (necessary for the frontend as no xml is
pre-loaded).
All rpc errors/warnings are now displayed using the same modal system as
the backend (crash_manager.js).
CrashManager will now return the WarningDialog and ErrorDialog
in addition to the CrashManager itself.
In some cases an instance of CrashManager was created just to
display a warning/error dialog.
In the backend all rpc errors but sessionExpired will be logged.
If the exception is a warning, NotFound (403) or except_orm, it will be
a warning log. The rest will be exception log.
In the frontend all rpc errors will be logged using console.debug
instead of console.warn and console.error. This way we can trigger
rpc errors in tests without failing them. If a real error happend
during a test it will fall back on the backend logger to fail the test.
Part of https://github.com/odoo/odoo/pull/32132
task-1894820
On `WebRequest` `__exit__`, when an exception occured,
(in `self.registry.signal_changes` or `self.registry.reset_changes`)
cursor were left unclosed as `self._cr.close` was not called
in such cases.
Having exceptions in the above mentioned method do not happen
often, but when it does it left unclosed and unusable cursors
in the connection pool, and in the extreme case explained below,
it left the connection pool with only unclosed and unusable cursors.
The entire server was then unusable as it no longer had working cursors.
Case:
- Start a multi-thread server with db_maxconn set to 5
- Ensure you do not send any request to the server,
not even with a left open tab on `http://localhost:8069` in your browser
- Send 6 parallel HTTP requests to `/web/login`
thanks to an external thread python script
(See below, at the end of this long commit message)
According to your registry state (if you have a lot of modules installed or not),
and the native Python Garbage Collecting state,
you might end with
- either warnings telling some unclosed cursor were garbage collected,
and therefore closed (by a kind of luck thanks to the Python garbage collecting),
- either, a server completely blocked not accepting any other request
(you can try for instance `curl http://localhost:8069`
and you end up with a `500 Internal Server Error`
This observed issue looks to appear only in 11.0. Not 10.0 or 12.0.
This is because only 11.0 clear the cache during registry loading:
`https://github.com/odoo/odoo/blob/f1706c848d41c47646dabca771996e9b9f788241/odoo/modules/loading.py#L236`
This cache clearing doesn't happen in 10.0 nor 12.0
(in 12.0, thanks to e181f592f3)
When sending the 6 parallel requests,
it uses instantly all the 5 available cursors of the connection pool to handle these requests,
and when each request exits, in `__exit__`, it calls `self.registry.signal_changes()`
which tries to open a new cursor because of
- `self.cache_invalidated` which is True, for all the 6 requests, thanks to the call to `clear_caches`
explained above during the registry loading and the fact all requests have been treated in parallel,
- `with closing(self.cursor()) as cr:`, `self.cursor()` attempting to use a new cursor
(the `closing(...)` does not have any incidence on this issue, despite it could look like guilty)
The attempt to use a new cursor fails, as there is no more available (`db_maxconn` is reached),
raising a `PoolError('The Connection Pool Is Full')` exception.
In the request `__exit__` method, because of this exception raised when calling `signal_changes`,
`self._cr.close` is never reached, and the parallel request therefore left only unclosed
cursors in the connection pool,
therefore leaving the server in a state where it only has unusable cursors
and therefore can't do anything more.
This might look like really bad luck to land in such a state,
but we observed multiple actual case on Odoo.sh,
the one referenced in this commit (opw-2008340) was because of an Outlook client
which launched 18 parallel requests to fetch the email images,
and the server wasn't spawned, therefore neither was the registry.
The server registry was therefore just loaded when it received the 18 parallel requests,
and it therefore triggered this extreme use case.
The server was left unusable for several minutes, until a forced restart.
For reference, here is the script that has been used to trigger the 6 parallel requests:
```
import requests
import threading
threads = []
for i in range(6):
threads.append(threading.Thread(target=lambda: requests.get('http://localhost:8069/web/login')))
for thread in threads:
thread.start()
```
opw-2008340
closesodoo/odoo#34071
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
It seems to be a better solution to keep starting tests without being in debug
mode to avoid having new fields and menus appearing.
This commit simply drop the debug mode in test mode and retrieve the test
assets by checking if test mode is enabled directly.
closesodoo/odoo#34012
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Before this commit, call a controller defined as:
```
@http.route('/route', type='http', auth='public')
def controller_func(self, foo):
do_it()
```
and called with url like /route?foo=1&bar=2
will crash with an exception:
`TypeError: controller_func() got an unexpected keyword argument 'bar'`
Now, we remove the extra parameters if the controller doesn't support it.
This case is not uncommon, you can easily arrive in this case with utm or
debug as extra parameter.
closesodoo/odoo#33962
Signed-off-by: Christophe Simonis <chs@odoo.com>
This commit goal is to encapsulate every tour-test into a separate assets
bundle that would be called only during tests (command line) or by URL
(debug=tests). That way, a lot of .js files would not be loaded anymore
uselessly outside test mode and will speed up the page loads (especially in the
frontend as the backend do not reload the page anyway).
In order to do that, we needed to propagate the `debug` state from page to
page.
Otherwise, every page change during a test would simply lose the debug mode and
the test would stop as the test assets would not be loaded.
As we could not add the `&debug=tests` on every link (either hardcoded or
preprocess during the rendering), it has been decided to store it in session.
Technical summary:
1. `debug` state (currently only stored in URL) will be stored in session.
Either when adding the `debug` param in URL (handled with _dispatch) or by
starting Odoo with `test-enable` or `test-file` (handled by session init).
2. Once activated (and so set in session), debug mode will remain activated
even if not visible in URL (after a page navigation eg).
To deactivate it, set its value to nothing, `debug=`. That will exit debug mode
(whatever mode it is: debug, assets, tests).
3. As tour-test files are now in a separate bundle, every layout (when needed)
should `t-call="web.conditional_assets_tests"`.
As it would be redundant and verbose, no `t-if` is needed on the t-call to
load it only in test debug mode. That will be handled by
`compiled_assets_tests` that will actually do the conditionnal t-call-assets
to web.assets_tests, if tests debug mode is activated.
4. In addition to separating the tour-tests files in a separate bundle, we also
moved those files to a specific folder under /static/tests/tours next to
QUnit tests.
5. Also, tour files will be moved in a specific folder /static/src/js/tours for
cleanness purpose (those files will still be kept in their 'normal' assets
as needed outside test mode since it is tours).
This will be done in the next commit.
6. It is possible to enable multiple debug mode, such as 'tests' and 'assets'
together. Simply separate debug modes with a comma, eg '?debug=assets,tests'
task-1934445
Comes with https://github.com/odoo/enterprise/pull/4281Closes#33213
In the HTTP pipeline, there is a middleware called DisableCacheMiddleware
that is called on every request to check if we are in debug mode, to remove
the cache headers and replace them with a static 'no-cache' so the browser
has to contact the server and check for a new version of every ressouce.
There were 2 problems with it:
1. it was doing more work than strictly needed when not in debug, like
rebuilding a list of headers for every request
2. it was removing Etag (case sensitive) with a typo and other headers
that could have been left in place.
This commit fixes those 2 issues by only modifying the headers if we are
in debug mode, and only removing the header 'cache-control' and replacing
it with 'cache-control: no-cache'
* = website_profile, website_slides
Before this commit, the returned Content-Type was not always correct, for
example if the image tool was changing the format, which happens when given
a BMP (converted to PNG), or other types being converted to JPEG.
The previous method `force_contenttype` was too specific and it wasn't available
in every controller. It didn't even need to be a controller method because it
didn't use self.
Now we create a generic helper to ease updating headers.
The Content-Type is only updated when it is safe to do. It is especially unsafe
for example for SVG files.
task-1958000
PR: #31811
JSONP can be entirely replaced by the CORS mechanism, which is simpler.
We support CORS in our routes since 8.0 (odoo/odoo@9cce88a), so it's about time
to get rid of JSONP.
Swapnesh Shah