Web lacked a default route for /robots.txt meaning that if you didn't
installed website (which comes with a full fledged robots.txt) you would
get a 404 page not found.
closesodoo/odoo#127402
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Before this commit:
When uploading a file, if the transaction fails due to a serialization
failure, Odoo will retry the request. However, if a file upload is read
during the transaction, the file pointer will be at the end of the file,
and calling `.read()` again returns an empty bytes object.
After this commit:
Upon retrying the request, rewind uploads to the beginning of the file,
if the file supports it.
opw-3228200
closesodoo/odoo#120180
X-original-commit: ac59ef0668122ad71dffbb5575250c767a0a56ec
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Maxmind offers multiple ip-geolocalization databases, historically we
have been using the City database which contains records on a
city-basis. Many years later it turns out we are primary using geoip to
know the country of the user. Geolocalization in the City database is
considered slow by our standard and we have been clever in order not to
geolocate each request by saving the info in the session.
On the other hand, the Country database that is offered by Maxmind is
much more lightweight and geoip using that country is considered a fast
operation by our standard.
In this work we make Odoo compatible with both the City and the Country
databases. Using multiple database at the same time, we can be smart and
only query each of the two on-demand. If a user ask for its country,
we'll use the fast Country db. If a user ask for its city/timezone we'll
use the slower City db.
By default it loads both database from the `/usr/share/GeoIP/` folder,
respectively the files `GeoLite2-City.mmdb` and `GeoLite2-Country.mmdb`,
you can provide alternative paths using the `--geoip-city-db` and
`--geoip-country-db` CLI options.
In the same mindset as #86015, geoip is still lazy. It is done on-demand
and the result is cached on the current request. The different with the
related PR is that as we know consider geoip to be fast, we no longer
cache the result in the session.
Task: 2848206
Part-of: odoo/odoo#91337
Move code to support only the redirect from url containing double '/' in
the middle of the path.
Keep same behavior than v15 and default Apache behavior.
domain.com//shop/product/1 -> 404
domain.com/shop//product/1 -> 301 -> /shop/product/1
opw-3063387
closesodoo/odoo#106137
X-original-commit: fdd6bf9942e2807b6d1460dba1ca5f61404c7b06
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Jérémy Kersten <jke@odoo.com>
This fix allows generating the sitemap (via `website.search_pages`) via
RPC. The changes are necessary as since #99667 the `request` object is
no more available to RPC-executed functions.
See also: the `test_search` test case of `website.tests.test_page`.
closesodoo/odoo#104567
X-original-commit: 8f4e214417cfb1aed7e595b8d6ab71f33a347b29
Signed-off-by: Julien Castiaux <juc@odoo.com>
Install website then render a qweb template via xmlrpc, attribute error:
`request` has no `is_frontend` attribute.
Inside the qweb's `_prepare_environment` override of the http_routing
module (a dependency of website) is the following code snippet:
if (not irQweb.env.context.get('minimal_qcontext') and
request and request.is_frontend):
return irQweb._prepare_frontend_environment(values)
The conditionnal is about injecting extra informations in the qweb's
context in case we are serving a frontend request. By default, there is
no `is_frontend` attribute on the request object. That attribute is set
by the ir.http's `_match` override of the http_routing module (a website
dependency): it is set True when we don't match any endpoint or that we
match an endpoint that is `website=True`, it is set False otherwise.
The ir.http's `_match` method is called whenever we are serving a http
request whoose session is bound to a specific database. i.e. when there
is a valid database saved in the request's session. When there is not
database in the request's session (or that it is invalid) the matched
endpoint is directly called without going throught ir.http. Most
endpoints are only accessible via ir.http.
The two `/xmlrpc` and `/jsonrpc` endpoints are examples of endpoint that
do not require an established database connection to work. They perform
the request authentication and database connection themselves. It is
possible to call those two endpoints with no database saved in the
session, thus it is possible to call those two endpoints without going
throught ir.http. This is expected.
The two endpoints's duty is to execute public model methods and return
the xml/json serialized result. To do so, a registry is loaded on the
database with all the installed modules, including http_routing.
We fall in a situation where (1) there is a request, (2) we are using a
registry where http_routing is loaded, (3) there is no `is_frontend`
attribute on `request` as we didn't serve the endpoint via ir.http. This
situation is illegale.
To solve the problem, instead of working on the `if request.is_frontend`
bit of the above conditional, we decided to work on the `if request`
bit. ISO-model wise, RPC is an extra 8th layer built on top of HTTP.
HTTP is merely a transparent transport between a RPC client and a RPC
server, any other request-response capable procotol could fit. The
method executed via RPC must be independant from the usage of HTTP as
mean of transportation thus it should not be capable of using the
current request.
The proposed change is to temporary un-expose the current request from
the local-stack during the execution of the RPC method. This fixes the
problem as the code now run like it was executed from the shell or from
a cron. The other benefit is that the pattern used inside the condition:
`if request and request.is_frontend` doesn't need to change.
X-original-commit: f28863bfdaac3f644fcb274c856e0770783ed75c
Part-of: odoo/odoo#104567
It allows to always have a OdooResponse Object and don't allow redirect
to external except when you allow it explicitly with local=False.
Always return to a local url:
/website/add
/slides/slide/<model("slide.slide"):slide>
/microsoft_outlook/confirm
Allow previously external redirect without reason, now blocked
/website/lang/<lang> -> open redirect
Allow external redirect for good reason and url is controlled by code.
/social_facebook/redirect_to_profile/
PS: HTTP Code 303 is a better default for generic redirects. It's not
historically the default for werkzeug.utils, but it is what we want in
general. Contrary to 302, there is no browser-dependent behavior, and
no risk of asking the user whether they want to accept the redirect if
the original method wasn't GET. It's always a non-permanent GET on the
target location.
closesodoo/odoo#95019
X-original-commit: 77f8d9c5d96a9274785ffc2ad83b95ea157d26ad
Related: odoo/enterprise#29000
Signed-off-by: Olivier Dony <odo@odoo.com>
Signed-off-by: Jérémy Kersten <jke@odoo.com>
Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closesodoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
Christophe Monniez