Some code modify return of res.users().context_get, but this is a
cached method so this will unexpectedly affects totally unrelated code.
For example, changing the company with the company switcher could add
`allowed_company_ids` inside the cache, then it will be cached until the
server is restarted, even if we change company again inbetween.
Added test failed with:
"NotImplementedError: '__setitem__' not supported on frozendict"
on the line with `User = User.with_context(context)` where User already
contained `allowed_company_ids` in its context.
note:
in this forward-port, context_get is also changed to return frozendict
and prevent being able to have an unexpected issue by code that modify
context_get returns.
opw-2158340
closes#42465closesodoo/odoo#42723
X-original-commit: 5d69885c1cd6921b3de00aae7e0ed6fff243ff95
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
odoo/odoo#28519 removed large parts of pycompat, but left reraise
despite that not having much value.
Remove that helper and replace it by just a `raise` in most cases:
when raising from an except block, the old exception is automatically
chained to the new one, no need to mess around.
There is one exception: in http we have to re-raise an existing
exception explicitly (aka `raise exc` rather than just `raise).
This is less than ideal as Python *concatenates* stacks: the
previously reified stack (from the except clause) is stacked on top of
the new stack (from this raises), this leads to tracebacks "jumping
around" at the break point of the handler and is somewhat confusing.
So we want to use explicit chaining (`raise a from b`) with the
"source" providing the caught exception's original traceback and the
child providing the rest.
However since callers rely on the exception making sense, we need the
re-raised exception to be the original[0]. Copying the exception
doesn't work (see [0]), chaining an exception to itself doesn't
do anything useful, and while we could probably copy exceptions using
the pickle method[1] that's still risky.
So the most reliable option seems to be to create a new "cause"
exception, move the old traceback over to it, then re-raise the
original exception having cleared its traceback, chained to new the
cause.
[0] or a copy thereof but Odoo exceptions don't all work properly with
copy.copy and we don't want that to fail so not really an option,
we can't rely / bet on every new exception being cleanly copy-able
[1] create an "empty" instance using __new__ (or an instance of
something else onto which we re-set the __class__ in case the exctype
actually overrides __new__) then copy the __dict__
closesodoo/odoo#39709
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Activate developer mode, generate a report (print an invoice)
The report will have missing pieces, like the footer or some part of the
header. This is probably caused by wkhtmltopdf not loading properly some
resources
Wkhtmtopdf generate the same warning message for every problematic resource:
"Warning: Received createRequest signal on a disposed ResourceObject's
NetworkAccessManager. This might be an indication of an iframe taking
too long to load."
Related issue on wkhtmltopdf project page:
wkhtmltopdf/wkhtmltopdf#1865wkhtmltopdf/wkhtmltopdf#3933wkhtmltopdf/wkhtmltopdf#2565
The problem is located in the response that wkhtmltopdf receive:
in debug mode the header of the response contains
'Cache-Control: no-cache' which probably create a race condition during
the rendering while a second request is attempted to verify the
resources.
Adding a raw user agent check to not include this header directive
fix the problem
Notes from odony:
We've considered some alternative solutions to preserve the purpose of the
DisableCacheMiddleware without having to explicitly test for wkhtmltopdf.
* 'Cache-Control: no-cache' (current behavior) breaks wkhtmltopdf rendering
* 'Cache-Control: no-store' breaks wkhtmltopdf rendering too
* 'Cache-Control: max-age=0' breaks wkhtmltopdf rendering too. It works
when increasing the delay to a few seconds, but no magic value will work
for very long documents, or it will stop serving its purpose, so it's not a
viable option.
* 'Cache-Control: must-revalidate' does not break wkhtmltopdf rendering (no
duplicate requests at all), but it is not clear from the RFC
(https://tools.ietf.org/html/rfc7234#section-5.2.2.1) that it
will have the intended effect for our middlewar
opw-2086708
Closes#38394closesodoo/odoo#39634
X-original-commit: 8cac60be37133cabef46dab016a5692876da9e5e
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
After this commit, you will be able (in technical mode) to update the url for
the python controllers.
Eg.
You can now rename /shop in /garden and /shop/product/ in /garden/vegetable/
Most of urls will be replaced at fly in the renderd qweb, with the function
url_for but all old urls will keep available. So if you access url /shop you
will be automatically redirected to /garden (308 Permanent Redirect).
As for cdn and other post-process of att, the automatically replacement in the
rendered qweb is only done when you will be not website editor. But the new
dispatch of URL will be applied in all cases.
For developper, since it is Permanent Redirect, don't forget to clear cache or
open chrome debug tool (with option 'Disable cache while DevTools is Open) to
see your lasts changes.
closesodoo/odoo#36555
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Section 7.1.2 of RFC 7231 requires preservation of URL fragment
through redirects. Our old JS redirection code was necessary when
browsers did not consistently implement it. Apart from a few odd
and marginal exceptions they all do it now, so we can stop that.
* base, web
Google now recommends a TTL of one year for static contents. We used to
use 1 week in almost every case. This commit increases that value to one
year for safe resources, like assets bundles which contain a specific
hash in the URL which changes if the bundle is recomputed anyway.
Note: this commit refactors the code so that both the one week and one
year durations are defined in http.py and used by others apps. Loading
the library "locale" file used to be done with 10-hours-cache, this has
been increased to 1-week-cache by using the http.py STATIC_CACHE var.
closesodoo/odoo#37402
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The rollback clears the cache, which lose all data that have been fetched before
arriving in the route method.
This lost cache includes some website data that was used during the dispatch and
that will be used again in the route.
By keeping it we reduce the number of queries on every request by at least 2.
Part of task-2061122
closesodoo/odoo#36245
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
* = stock, test_website, web, website_forum, website_slides, base
Replace KarmaError with AccessError and remove the related override made
on crash_manager and ir_http.
task-2069890
closesodoo/odoo#36655
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
With this commit it is now possible to change the lang displayed in the URL.
Eg, you could use `/fr` instead of `/fr_BE`, or even a fancier `/french`.
Task-32838
Courtesy of pla@odoo.comclosesodoo/odoo#35135
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
* = gamification, test_website, stock
- KarmaError is now handled as a 400 exception.
- test_website has been updated.
- NO_POSTMORTEM is now clean it was referencing duplicates as most the
exceptions inherit from except_orm.
- serialize_exception from http.py has been moved to ir_http
to take advantage of the odoo inheritance system. We can then
extend ir_http serialize_exception method to add the KarmaError
logic if and only if gamification is installed.
Places where serialize_exception was previously used are updated.
Part of https://github.com/odoo/odoo/pull/32132
task-1894820
* = account, iap, point_of_sale
Removing the CrashManager templates from base.xml and moving
it to a new file (crash_manager.xml) to allow lazy loading the required
xml only when a crash occurs (necessary for the frontend as no xml is
pre-loaded).
All rpc errors/warnings are now displayed using the same modal system as
the backend (crash_manager.js).
CrashManager will now return the WarningDialog and ErrorDialog
in addition to the CrashManager itself.
In some cases an instance of CrashManager was created just to
display a warning/error dialog.
In the backend all rpc errors but sessionExpired will be logged.
If the exception is a warning, NotFound (403) or except_orm, it will be
a warning log. The rest will be exception log.
In the frontend all rpc errors will be logged using console.debug
instead of console.warn and console.error. This way we can trigger
rpc errors in tests without failing them. If a real error happend
during a test it will fall back on the backend logger to fail the test.
Part of https://github.com/odoo/odoo/pull/32132
task-1894820
On `WebRequest` `__exit__`, when an exception occured,
(in `self.registry.signal_changes` or `self.registry.reset_changes`)
cursor were left unclosed as `self._cr.close` was not called
in such cases.
Having exceptions in the above mentioned method do not happen
often, but when it does it left unclosed and unusable cursors
in the connection pool, and in the extreme case explained below,
it left the connection pool with only unclosed and unusable cursors.
The entire server was then unusable as it no longer had working cursors.
Case:
- Start a multi-thread server with db_maxconn set to 5
- Ensure you do not send any request to the server,
not even with a left open tab on `http://localhost:8069` in your browser
- Send 6 parallel HTTP requests to `/web/login`
thanks to an external thread python script
(See below, at the end of this long commit message)
According to your registry state (if you have a lot of modules installed or not),
and the native Python Garbage Collecting state,
you might end with
- either warnings telling some unclosed cursor were garbage collected,
and therefore closed (by a kind of luck thanks to the Python garbage collecting),
- either, a server completely blocked not accepting any other request
(you can try for instance `curl http://localhost:8069`
and you end up with a `500 Internal Server Error`
This observed issue looks to appear only in 11.0. Not 10.0 or 12.0.
This is because only 11.0 clear the cache during registry loading:
`https://github.com/odoo/odoo/blob/f1706c848d41c47646dabca771996e9b9f788241/odoo/modules/loading.py#L236`
This cache clearing doesn't happen in 10.0 nor 12.0
(in 12.0, thanks to e181f592f3)
When sending the 6 parallel requests,
it uses instantly all the 5 available cursors of the connection pool to handle these requests,
and when each request exits, in `__exit__`, it calls `self.registry.signal_changes()`
which tries to open a new cursor because of
- `self.cache_invalidated` which is True, for all the 6 requests, thanks to the call to `clear_caches`
explained above during the registry loading and the fact all requests have been treated in parallel,
- `with closing(self.cursor()) as cr:`, `self.cursor()` attempting to use a new cursor
(the `closing(...)` does not have any incidence on this issue, despite it could look like guilty)
The attempt to use a new cursor fails, as there is no more available (`db_maxconn` is reached),
raising a `PoolError('The Connection Pool Is Full')` exception.
In the request `__exit__` method, because of this exception raised when calling `signal_changes`,
`self._cr.close` is never reached, and the parallel request therefore left only unclosed
cursors in the connection pool,
therefore leaving the server in a state where it only has unusable cursors
and therefore can't do anything more.
This might look like really bad luck to land in such a state,
but we observed multiple actual case on Odoo.sh,
the one referenced in this commit (opw-2008340) was because of an Outlook client
which launched 18 parallel requests to fetch the email images,
and the server wasn't spawned, therefore neither was the registry.
The server registry was therefore just loaded when it received the 18 parallel requests,
and it therefore triggered this extreme use case.
The server was left unusable for several minutes, until a forced restart.
For reference, here is the script that has been used to trigger the 6 parallel requests:
```
import requests
import threading
threads = []
for i in range(6):
threads.append(threading.Thread(target=lambda: requests.get('http://localhost:8069/web/login')))
for thread in threads:
thread.start()
```
opw-2008340
closesodoo/odoo#34071
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
It seems to be a better solution to keep starting tests without being in debug
mode to avoid having new fields and menus appearing.
This commit simply drop the debug mode in test mode and retrieve the test
assets by checking if test mode is enabled directly.
closesodoo/odoo#34012
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Before this commit, call a controller defined as:
```
@http.route('/route', type='http', auth='public')
def controller_func(self, foo):
do_it()
```
and called with url like /route?foo=1&bar=2
will crash with an exception:
`TypeError: controller_func() got an unexpected keyword argument 'bar'`
Now, we remove the extra parameters if the controller doesn't support it.
This case is not uncommon, you can easily arrive in this case with utm or
debug as extra parameter.
closesodoo/odoo#33962
Signed-off-by: Christophe Simonis <chs@odoo.com>
This commit goal is to encapsulate every tour-test into a separate assets
bundle that would be called only during tests (command line) or by URL
(debug=tests). That way, a lot of .js files would not be loaded anymore
uselessly outside test mode and will speed up the page loads (especially in the
frontend as the backend do not reload the page anyway).
In order to do that, we needed to propagate the `debug` state from page to
page.
Otherwise, every page change during a test would simply lose the debug mode and
the test would stop as the test assets would not be loaded.
As we could not add the `&debug=tests` on every link (either hardcoded or
preprocess during the rendering), it has been decided to store it in session.
Technical summary:
1. `debug` state (currently only stored in URL) will be stored in session.
Either when adding the `debug` param in URL (handled with _dispatch) or by
starting Odoo with `test-enable` or `test-file` (handled by session init).
2. Once activated (and so set in session), debug mode will remain activated
even if not visible in URL (after a page navigation eg).
To deactivate it, set its value to nothing, `debug=`. That will exit debug mode
(whatever mode it is: debug, assets, tests).
3. As tour-test files are now in a separate bundle, every layout (when needed)
should `t-call="web.conditional_assets_tests"`.
As it would be redundant and verbose, no `t-if` is needed on the t-call to
load it only in test debug mode. That will be handled by
`compiled_assets_tests` that will actually do the conditionnal t-call-assets
to web.assets_tests, if tests debug mode is activated.
4. In addition to separating the tour-tests files in a separate bundle, we also
moved those files to a specific folder under /static/tests/tours next to
QUnit tests.
5. Also, tour files will be moved in a specific folder /static/src/js/tours for
cleanness purpose (those files will still be kept in their 'normal' assets
as needed outside test mode since it is tours).
This will be done in the next commit.
6. It is possible to enable multiple debug mode, such as 'tests' and 'assets'
together. Simply separate debug modes with a comma, eg '?debug=assets,tests'
task-1934445
Comes with https://github.com/odoo/enterprise/pull/4281Closes#33213
In the HTTP pipeline, there is a middleware called DisableCacheMiddleware
that is called on every request to check if we are in debug mode, to remove
the cache headers and replace them with a static 'no-cache' so the browser
has to contact the server and check for a new version of every ressouce.
There were 2 problems with it:
1. it was doing more work than strictly needed when not in debug, like
rebuilding a list of headers for every request
2. it was removing Etag (case sensitive) with a typo and other headers
that could have been left in place.
This commit fixes those 2 issues by only modifying the headers if we are
in debug mode, and only removing the header 'cache-control' and replacing
it with 'cache-control: no-cache'
* = website_profile, website_slides
Before this commit, the returned Content-Type was not always correct, for
example if the image tool was changing the format, which happens when given
a BMP (converted to PNG), or other types being converted to JPEG.
The previous method `force_contenttype` was too specific and it wasn't available
in every controller. It didn't even need to be a controller method because it
didn't use self.
Now we create a generic helper to ease updating headers.
The Content-Type is only updated when it is safe to do. It is especially unsafe
for example for SVG files.
task-1958000
PR: #31811
JSONP can be entirely replaced by the CORS mechanism, which is simpler.
We support CORS in our routes since 8.0 (odoo/odoo@9cce88a), so it's about time
to get rid of JSONP.
- This commit fixes a crash that happens when the server receive a
JSON-P call done in two requests (a POST followed by a GET).
The issue is due to the fact that when the first request is done (POST
one) the member `params` is never initialized.
This parameter is then used in the module `auth_signup` on an override
of the method `dispatch` thus crashing the code.
To avoid the crash, we now initialize `params`.
closesodoo/odoo#27369
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
reuse current_thread result
No needed to call threading.current_thread() on evry arg setting
closesodoo/odoo#32000
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
This revision aims to support the memory limits
on Linux, Windows and MacOSX according to their own spefication
regarding their memory management.
Among others, it brings the possibility to
use the multi-workers mode on Windows and MacOSX.
e.g.
- Windows does not support `resource`,
and therefore we skip to set the hard limit
- MacOSX allocates a large virtual memory for each process
even if the memory is not actually used,
and therefore using the VMS to limit the memory is pointless
as it will always exceeds the default soft memory limit.
We therefore choose to use the RSS to limit the memory
closesodoo/odoo#27848
Session rotation was introduced a long time ago, but deactivated at
login due to obscure side-effects related to #6949 (aka the "BBQ" PR).
This commit reinstates the rotation, which is better from a security
standpoint.
In order to also prevent session ID reuse, we force the renewal of
deleted sessions, at the SessionStore level (via `renew_missing`).
When there is a performance issue, it's sometimes difficult to discover
which request increased the query count or its duration.
With this commit, the query count, the query time and "python and io" time are displayed
in the logs at the end of each werkzeug request line.
Co-authored-by: Christophe Monniez <moc@odoo.com>
From this commit onwards, Date fields will return datetime.date objects and Datetime fields will return datetime.datetime objects, this implies a number of things that are clearly explained both in the ORM API for master.
This commit also introduces a number of helper functions for dates and datetimes that are exposed in tools.date_utils and fields.Date[time], explained in the documentation as well.
Task-ID: 47189
* Make Users._login and session.authenticate always raise AccessDenied
on authentication failure instead of only sometimes (cf
Session.authenticate calling security.check() which raises and not
catching the exception)
* Alter AccessDenied such that it's possible to add a custom access
message, for use with login rate limiting instead of smuggling the
information via the session
* Alter the RPC endpoints to catch and convert AccessDenied back to
a boolean sentinel
Browsers accept it as they are wont to do, but the Content-Disposition
lib (introduced in 35d452cffb) does
not. Simply don't mark them as safe when %-escaping the filename so
they do get %-escaped properly.
RFC5987 states
ext-parameter = parmname "*" LWSP "=" LWSP ext-value
ext-value = charset "'" [ language ] "'" value-chars
value-chars = *( pct-encoded / attr-char )
attr-char = ALPHA / DIGIT
/ "!" / "#" / "$" / "&" / "+" / "-" / "."
/ "^" / "_" / "`" / "|" / "~"
; token except ( "*" / "'" / "%" )
Neither : nor / are in attr-char. This is further confirmed by
checking out RFC2616:
CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)>
token = 1*<any CHAR except CTLs or separators>
separators = "(" | ")" | "<" | ">" | "@"
| "," | ";" | ":" | "\" | <">
| "/" | "[" | "]" | "?" | "="
| "{" | "}" | SP | HT
Here we can see "/" and ":" are both in "separators", which are
specifically *not* in token. attr-char restricts token further, so an
invalid token char can't be a valid attr-char.
- Each time we check if a session is valid we create a new cursor.
This could lead to issues with db_maxconn that limits the number of
connections to the postgresql server.
In a perfect world, a worker should use a single connection to
postgres to process the request.
The only known side effect is that the cursor is created earlier in
the execution of the code.
- This commit fixes issues with the longpolling raising
Psycopg2.PoolError exceptions on databases with a lot of clients.
Nicolas Lempereur