80098df3dddd65295988221477aea8a5dba7bc5c
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Denis Ledoux
|
536e670f8c |
[FIX] http: ensure values of session are serializable when stored
closes odoo/odoo#86015 Signed-off-by: Julien Castiaux <juc@odoo.com> |
||
|
Julien Castiaux
|
ca5ca24bc6 |
[FIX] core: check browser lang is installed
A visitor could visit a web page having a lang in its context that is not installed in the databased he is connected to. The problem is that visitors are not logged-in thus it is not possible to determine their lang via their `res.users` preferences. The lang used instead is the lang set in the `Accept-Language` header of the incoming request, that header is set by various browsers in accordance to the user system preferences or browser settings. The browser lang (`Request.best`) is only parsed according to the `babel` database, it is a lang syntactically speaking but not necessary a lang that is installed in the database. At the moment the browser lang is set in the context (inside of `Request._get_dbname_and_session`), it is not possible to verify it is installed in the database as no connection to any database as been established yet. Instead the lang is validated inside of `ir.http._pre_dispatch` which is the method responsible to prepare/fix various stuff on the request/session/context. In regard to |
||
|
Julien Castiaux
|
c59750d824 |
[IMP] core: smarter geoip
Maxmind offers multiple ip-geolocalization databases, historically we have been using the City database which contains records on a city-basis. Many years later it turns out we are primary using geoip to know the country of the user. Geolocalization in the City database is considered slow by our standard and we have been clever in order not to geolocate each request by saving the info in the session. On the other hand, the Country database that is offered by Maxmind is much more lightweight and geoip using that country is considered a fast operation by our standard. In this work we make Odoo compatible with both the City and the Country databases. Using multiple database at the same time, we can be smart and only query each of the two on-demand. If a user ask for its country, we'll use the fast Country db. If a user ask for its city/timezone we'll use the slower City db. By default it loads both database from the `/usr/share/GeoIP/` folder, respectively the files `GeoLite2-City.mmdb` and `GeoLite2-Country.mmdb`, you can provide alternative paths using the `--geoip-city-db` and `--geoip-country-db` CLI options. In the same mindset as #86015, geoip is still lazy. It is done on-demand and the result is cached on the current request. The different with the related PR is that as we know consider geoip to be fast, we no longer cache the result in the session. Task: 2848206 Part-of: odoo/odoo#91337 |
||
|
Julien Castiaux
|
5502313853 |
[FIX] web, *: multi-db /web/session/authenticate
*: base_setup, hr_timesheet, mail, partner_autocomplete, web_tour Start odoo without -d and with a --dbfilter that allows multiple databases. Via JSON-RPC access the /web/session/authenticate route providing a non-filtered database and valid credentials. Traceback, `request.env` is None. Since httpocalypse the initialization of the ORM (cursor, registry, environment) is greedy. It means that the connection to the database is established very early during the request routing or skip altogether in case no dbname was known at that time. This contrast with prepocalypse where the various ORM thingies were lazily setup the first time they were accessed. This changement has an important implication regarding authentication. In prepocalypse, thanks to the lazy approache, a cursor/registry/env would be setup on the database you just login upon using the `request.env` for the first time. This was very nice in this regard but had other problems. Since httpocalypse such operation is no more possible. Devs must initialize and use their own cursor/registry/env in case they authenticate on another database than the one `request.cr` is (maybe) connected to. The `/web/session/authenticate` controller is an example of such case. It crates its own cr/registry/environment after authentication. The problem the controller uses `ir.http.session_info` and that not all overrides were updated to use `self.env` (=the env created in the web controller) instead of `request.env` (=the missing env of the request). closes odoo/odoo#108063 X-original-commit: 7b9bd9d37731fae724dc5d91da656dab70aa9ad4 Related: odoo/enterprise#35012 Signed-off-by: Julien Castiaux <juc@odoo.com> |
||
|
Julien Castiaux
|
ec4826ef5b |
[FIX] core: session logout after 16.0 migration
Create a 15.0 database with website, access the home page via your
browser. Stop the server and migrate the database to 16.0. Restart the
server with a `--dbfilter` that rejects the database you created and
refresh your browser. 500 Internal server error, attribute error:
the `request` object as no `session`.
An error could occurs after a migration to 16.0 due to the presence of
the `geoip` key in the session. `request.session.geoip` has been made a
deprecated alias to `request.geoip` between 15.0 and 16.0, see
|
||
|
Julien Castiaux
|
da8def8e41 |
[IMP] core, web: Delegate delivery of static files
Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closes odoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
|