On `WebRequest` `__exit__`, when an exception occured,
(in `self.registry.signal_changes` or `self.registry.reset_changes`)
cursor were left unclosed as `self._cr.close` was not called
in such cases.
Having exceptions in the above mentioned method do not happen
often, but when it does it left unclosed and unusable cursors
in the connection pool, and in the extreme case explained below,
it left the connection pool with only unclosed and unusable cursors.
The entire server was then unusable as it no longer had working cursors.
Case:
- Start a multi-thread server with db_maxconn set to 5
- Ensure you do not send any request to the server,
not even with a left open tab on `http://localhost:8069` in your browser
- Send 6 parallel HTTP requests to `/web/login`
thanks to an external thread python script
(See below, at the end of this long commit message)
According to your registry state (if you have a lot of modules installed or not),
and the native Python Garbage Collecting state,
you might end with
- either warnings telling some unclosed cursor were garbage collected,
and therefore closed (by a kind of luck thanks to the Python garbage collecting),
- either, a server completely blocked not accepting any other request
(you can try for instance `curl http://localhost:8069`
and you end up with a `500 Internal Server Error`
This observed issue looks to appear only in 11.0. Not 10.0 or 12.0.
This is because only 11.0 clear the cache during registry loading:
`https://github.com/odoo/odoo/blob/f1706c848d41c47646dabca771996e9b9f788241/odoo/modules/loading.py#L236`
This cache clearing doesn't happen in 10.0 nor 12.0
(in 12.0, thanks to e181f592f3)
When sending the 6 parallel requests,
it uses instantly all the 5 available cursors of the connection pool to handle these requests,
and when each request exits, in `__exit__`, it calls `self.registry.signal_changes()`
which tries to open a new cursor because of
- `self.cache_invalidated` which is True, for all the 6 requests, thanks to the call to `clear_caches`
explained above during the registry loading and the fact all requests have been treated in parallel,
- `with closing(self.cursor()) as cr:`, `self.cursor()` attempting to use a new cursor
(the `closing(...)` does not have any incidence on this issue, despite it could look like guilty)
The attempt to use a new cursor fails, as there is no more available (`db_maxconn` is reached),
raising a `PoolError('The Connection Pool Is Full')` exception.
In the request `__exit__` method, because of this exception raised when calling `signal_changes`,
`self._cr.close` is never reached, and the parallel request therefore left only unclosed
cursors in the connection pool,
therefore leaving the server in a state where it only has unusable cursors
and therefore can't do anything more.
This might look like really bad luck to land in such a state,
but we observed multiple actual case on Odoo.sh,
the one referenced in this commit (opw-2008340) was because of an Outlook client
which launched 18 parallel requests to fetch the email images,
and the server wasn't spawned, therefore neither was the registry.
The server registry was therefore just loaded when it received the 18 parallel requests,
and it therefore triggered this extreme use case.
The server was left unusable for several minutes, until a forced restart.
For reference, here is the script that has been used to trigger the 6 parallel requests:
```
import requests
import threading
threads = []
for i in range(6):
threads.append(threading.Thread(target=lambda: requests.get('http://localhost:8069/web/login')))
for thread in threads:
thread.start()
```
opw-2008340
closesodoo/odoo#34071
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
- This commit fixes a crash that happens when the server receive a
JSON-P call done in two requests (a POST followed by a GET).
The issue is due to the fact that when the first request is done (POST
one) the member `params` is never initialized.
This parameter is then used in the module `auth_signup` on an override
of the method `dispatch` thus crashing the code.
To avoid the crash, we now initialize `params`.
closesodoo/odoo#27369
This revision aims to support the memory limits
on Linux, Windows and MacOSX according to their own spefication
regarding their memory management.
Among others, it brings the possibility to
use the multi-workers mode on Windows and MacOSX.
e.g.
- Windows does not support `resource`,
and therefore we skip to set the hard limit
- MacOSX allocates a large virtual memory for each process
even if the memory is not actually used,
and therefore using the VMS to limit the memory is pointless
as it will always exceeds the default soft memory limit.
We therefore choose to use the RSS to limit the memory
closesodoo/odoo#27848
Session rotation was introduced a long time ago, but deactivated at
login due to obscure side-effects related to #6949 (aka the "BBQ" PR).
This commit reinstates the rotation, which is better from a security
standpoint.
In order to also prevent session ID reuse, we force the renewal of
deleted sessions, at the SessionStore level (via `renew_missing`).
When there is a performance issue, it's sometimes difficult to discover
which request increased the query count or its duration.
With this commit, the query count, the query time and "python and io" time are displayed
in the logs at the end of each werkzeug request line.
Co-authored-by: Christophe Monniez <moc@odoo.com>
From this commit onwards, Date fields will return datetime.date objects and Datetime fields will return datetime.datetime objects, this implies a number of things that are clearly explained both in the ORM API for master.
This commit also introduces a number of helper functions for dates and datetimes that are exposed in tools.date_utils and fields.Date[time], explained in the documentation as well.
Task-ID: 47189
* Make Users._login and session.authenticate always raise AccessDenied
on authentication failure instead of only sometimes (cf
Session.authenticate calling security.check() which raises and not
catching the exception)
* Alter AccessDenied such that it's possible to add a custom access
message, for use with login rate limiting instead of smuggling the
information via the session
* Alter the RPC endpoints to catch and convert AccessDenied back to
a boolean sentinel
Browsers accept it as they are wont to do, but the Content-Disposition
lib (introduced in 35d452cffb) does
not. Simply don't mark them as safe when %-escaping the filename so
they do get %-escaped properly.
RFC5987 states
ext-parameter = parmname "*" LWSP "=" LWSP ext-value
ext-value = charset "'" [ language ] "'" value-chars
value-chars = *( pct-encoded / attr-char )
attr-char = ALPHA / DIGIT
/ "!" / "#" / "$" / "&" / "+" / "-" / "."
/ "^" / "_" / "`" / "|" / "~"
; token except ( "*" / "'" / "%" )
Neither : nor / are in attr-char. This is further confirmed by
checking out RFC2616:
CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)>
token = 1*<any CHAR except CTLs or separators>
separators = "(" | ")" | "<" | ">" | "@"
| "," | ";" | ":" | "\" | <">
| "/" | "[" | "]" | "?" | "="
| "{" | "}" | SP | HT
Here we can see "/" and ":" are both in "separators", which are
specifically *not* in token. attr-char restricts token further, so an
invalid token char can't be a valid attr-char.
- Each time we check if a session is valid we create a new cursor.
This could lead to issues with db_maxconn that limits the number of
connections to the postgresql server.
In a perfect world, a worker should use a single connection to
postgres to process the request.
The only known side effect is that the cursor is created earlier in
the execution of the code.
- This commit fixes issues with the longpolling raising
Psycopg2.PoolError exceptions on databases with a lot of clients.
Endpoints can be explicitly marked as `save_session=False` (default is
true across the board). In that case they will have an in-memory session
(either the existing one or a brand new one) but the session won't be
persisted to disk.
Currently used for non-browser RPC endpoints: the APIs don't use
cookies/sessions and we can't assume the RPC libraries keep cookies
across calls. This means a new session is created and saved to disk for
each RPC calls, for no useful reason.
Before this patch, early connections made to a 11.0 Odoo server running
on Python 3 and deployed in threaded mode with socket activation would
generate invalid registries.
With this patch it is now possible, when this deployment mode is used,
to opt-out addons preload by setting the following environment variable:
ODOO_PRELOAD_ADDONS=no
Note: this environment variable is only available for v11.0 as later
versions does not preload anymore (cf: 1a39c9b)
This way, XMLRPC calls can get request details form the standard
`odoo.http.request` system.
Move jsonrpc to the same controller while at it, for coherence.
Fix#24183
Replace `httprequest.stream.read()` by `httprequest.get_data()` so the
payload content remains available: get_data stores the request body (by
default) so it remains available for alternative processing or checks
(MAC checks for webhook validations for instance). With `stream.read()`,
once the data is read if it's not stored separately it is lost.
- Install Website
- Load the 'Norwegian Bokmål' translation, and choose to translate the
website
- Logout
When accessing the website, the language displayed is not consistent to
the browser language.
1. `no` shows homepage in `nb_NO` (Firefox only) => expected since
Bokmål is the main language in Norway
2. `nb` shows homepage in `en_US` => unexpected
3. `nn` shows homepage in `en_US` => expected since 'Norwegian Nynorsk'
is not a language available (this will be the topic of another PR)
The issues comes from Babel's side, since `nb` is not in the
`LOCALE_ALIASES` while `no` is. We monkey-patch the value to avoid this
while Babel is corrected.
opw-1827258
Clean method _add_dispatch_parameters
Remove unused code for caching
Call super before to have the correct lang when we browse website.
Without it, menu was not loaded in correct language.
Purpose
=======
If an external dependency is unmet on a module that is not installed, it's actually impossible to launch an odoo server and load the web client without getting a traceback.
This commit removes this constraint and allow to use odoo in that case. Obviously the install will crash if the external dependency is still unmet.
Close https://github.com/odoo/odoo/pull/17790
Coming from https://github.com/odoo/odoo/pull/14850
Christophe Simonis