[IMP] core, web: Delegate delivery of static files

Rationnals
----------

Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.

Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.

X-Sendfile
----------

In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.

Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.

Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:

    location /web/filestore {  # custom path, hardcoded within Odoo
        # Prevent access from the outside world, i.e. makes this
        # route only accessible via X-Accel. MANDATORY!!!
        internal;

        # Give access to the filestore using this server's
        # permissions. Odoo is in charge of verifying the access
        # rights.
        alias /path/to/odoo/data-dir/filestore;
    }

The Odoo [deployment documentation] has been updated accordingly.

[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments

Changes to the API
------------------

To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.

Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.

I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.

A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.

Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:

- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`

The new `ir.binary` abstract model exposes the following utilities:

**`_find_record`**

Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.

**`_get_stream_from`**

Create a Stream from an attachment or a record with a binary-field.

**`_get_image_stream_from`**

Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.

**`_placeholder`**

Get the image placeholder blob.

Testing
-------

It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.

    odoo-bin -i test_http --stop-after-init
    WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init

closes odoo/odoo#88134

Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>

addons/account/tests/test_portal_attachment.py

@@ -89,11 +89,11 @@ class TestPortalAttachment(AccountTestInvoicingHttpCommon):
         self.assertEqual(create_res['mimetype'], 'text/plain')
 
         res_binary = self.url_open('/web/content/%d?access_token=%s' % (create_res['id'], create_res['access_token']))
-        self.assertEqual(res_binary.headers['Content-Type'], 'text/plain')
+        self.assertEqual(res_binary.headers['Content-Type'], 'text/plain; charset=utf-8')
         self.assertEqual(res_binary.content, b'<svg></svg>')
 
         res_image = self.url_open('/web/image/%d?access_token=%s' % (create_res['id'], create_res['access_token']))
-        self.assertEqual(res_image.headers['Content-Type'], 'text/plain')
+        self.assertEqual(res_image.headers['Content-Type'], 'text/plain; charset=utf-8')
         self.assertEqual(res_image.content, b'<svg></svg>')
 
         # Test attachment can't be removed without valid token

addons/im_livechat/controllers/main.py

@@ -20,10 +20,9 @@ class LivechatController(http.Controller):
         mock_attachment = getattr(asset, ext)()
         if isinstance(mock_attachment, list):  # suppose that CSS asset will not required to be split in pages
             mock_attachment = mock_attachment[0]
-        # can't use /web/content directly because we don't have attachment ids (attachments must be created)
-        _, headers, content = request.env['ir.http'].binary_content(id=mock_attachment.id, unique=asset.checksum)
-        headers.append(('Content-Length', len(content)))
-        return request.make_response(content, headers)
+
+        stream = request.env['ir.binary']._get_stream_from(mock_attachment)
+        return stream.get_response()
 
     @http.route('/im_livechat/load_templates', type='json', auth='none', cors="*")
     def load_templates(self, **kwargs):
@@ -100,21 +99,11 @@ class LivechatController(http.Controller):
                 ('operator_partner_id', 'in', operator.ids)
             ]))
 
-        status = 200
-        headers = []
-        # custom placeholer (a smiley face instead of the infamous camera)
-        image_placeholder = 'mail/static/src/img/smiley/avatar.jpg'
-
-        if is_livechat_member:
-            status, headers, image_base64 = request.env['ir.http'].sudo().binary_content(
-                model='res.partner', id=operator_id, field='avatar_128', default_mimetype='image/png')
-
-            if status in [301, 304]:
-                image_base64 = request.env['ir.http']._placeholder(image_placeholder)
-        else:
-            image_base64 = request.env['ir.http']._placeholder(image_placeholder)
-
-        return request.env['ir.http']._content_image_get_response(status, headers, image_base64)
+        return request.env['ir.binary']._get_image_stream_from(
+            operator if is_livechat_member else None,
+            field_name='avatar_128',
+            placeholder='mail/static/src/img/smiley/avatar.jpg',
+        ).get_response()
 
     @http.route('/im_livechat/get_session', type="json", auth='public', cors="*")
     def get_session(self, channel_id, anonymous_name, previous_operator_id=None, chatbot_script_id=None, **kwargs):

addons/mail/controllers/discuss.py

@@ -136,29 +136,36 @@ class DiscussController(http.Controller):
     @http.route('/mail/channel/<int:channel_id>/partner/<int:partner_id>/avatar_128', methods=['GET'], type='http', auth='public')
     def mail_channel_partner_avatar_128(self, channel_id, partner_id, **kwargs):
         channel_partner_sudo = request.env['mail.channel.partner']._get_as_sudo_from_request(request=request, channel_id=channel_id)
-        if not channel_partner_sudo or not channel_partner_sudo.env['mail.channel.partner'].search([('channel_id', '=', channel_id), ('partner_id', '=', partner_id)], limit=1):
-            if request.env.user.share:
-                placeholder = channel_partner_sudo.env['res.partner'].browse(partner_id).exists()._avatar_get_placeholder()
-                return channel_partner_sudo.env['ir.http']._placeholder_image_get_response(placeholder)
-            return channel_partner_sudo.sudo(False).env['ir.http']._content_image(model='res.partner', res_id=partner_id, field='avatar_128')
-        return channel_partner_sudo.env['ir.http']._content_image(model='res.partner', res_id=partner_id, field='avatar_128')
+        partner_sudo = channel_partner_sudo.env['res.partner'].browse(partner_id).exists()
+        placeholder = partner_sudo._avatar_get_placeholder_path()
+        if channel_partner_sudo and channel_partner_sudo.env['mail.channel.partner'].search([('channel_id', '=', channel_id), ('partner_id', '=', partner_id)], limit=1):
+            return request.env['ir.binary']._get_image_stream_from(partner_sudo, field_name='avatar_128', placeholder=placeholder).get_response()
+        if request.env.user.share:
+            return request.env['ir.binary']._get_placeholder_stream(placeholder)
+        return request.env['ir.binary']._get_image_stream_from(partner_sudo.sudo(False), field_name='avatar_128', placeholder=placeholder).get_response()
 
     @http.route('/mail/channel/<int:channel_id>/guest/<int:guest_id>/avatar_128', methods=['GET'], type='http', auth='public')
     def mail_channel_guest_avatar_128(self, channel_id, guest_id, **kwargs):
         channel_partner_sudo = request.env['mail.channel.partner']._get_as_sudo_from_request(request=request, channel_id=channel_id)
-        if not channel_partner_sudo or not channel_partner_sudo.env['mail.channel.partner'].search([('channel_id', '=', channel_id), ('guest_id', '=', guest_id)], limit=1):
-            if request.env.user.share:
-                placeholder = channel_partner_sudo.env['mail.guest'].browse(guest_id).exists()._avatar_get_placeholder()
-                return channel_partner_sudo.env['ir.http']._placeholder_image_get_response(placeholder)
-            return channel_partner_sudo.sudo(False).env['ir.http']._content_image(model='mail.guest', res_id=guest_id, field='avatar_128')
-        return channel_partner_sudo.env['ir.http']._content_image(model='mail.guest', res_id=guest_id, field='avatar_128')
+        guest_sudo = channel_partner_sudo.env['mail.guest'].browse(guest_id).exists()
+        placeholder = guest_sudo._avatar_get_placeholder_path()
+        if channel_partner_sudo and channel_partner_sudo.env['mail.channel.partner'].search([('channel_id', '=', channel_id), ('guest_id', '=', guest_id)], limit=1):
+            return request.env['ir.binary']._get_image_stream_from(guest_sudo, field_name='avatar_128', placeholder=placeholder).get_response()
+        if request.env.user.share:
+            return request.env['ir.binary']._get_placeholder_stream(placeholder)
+        return request.env['ir.binary']._get_image_stream_from(guest_sudo.sudo(False), field_name='avatar_128', placeholder=placeholder).get_response()
 
     @http.route('/mail/channel/<int:channel_id>/attachment/<int:attachment_id>', methods=['GET'], type='http', auth='public')
     def mail_channel_attachment(self, channel_id, attachment_id, download=None, **kwargs):
         channel_partner_sudo = request.env['mail.channel.partner']._get_as_sudo_from_request_or_raise(request=request, channel_id=int(channel_id))
-        if not channel_partner_sudo.env['ir.attachment'].search([('id', '=', int(attachment_id)), ('res_id', '=', int(channel_id)), ('res_model', '=', 'mail.channel')], limit=1):
+        attachment_sudo = channel_partner_sudo.env['ir.attachment'].search([
+                ('id', '=', int(attachment_id)),
+                ('res_id', '=', int(channel_id)),
+                ('res_model', '=', 'mail.channel')
+            ], limit=1)
+        if not attachment_sudo:
             raise NotFound()
-        return channel_partner_sudo.env['ir.http']._get_content_common(res_id=int(attachment_id), download=download)
+        return request.env['ir.binary']._get_stream_from(attachment_sudo).get_response(as_attachment=download)
 
     @http.route([
         '/mail/channel/<int:channel_id>/image/<int:attachment_id>',
@@ -166,9 +173,18 @@ class DiscussController(http.Controller):
     ], methods=['GET'], type='http', auth='public')
     def fetch_image(self, channel_id, attachment_id, width=0, height=0, **kwargs):
         channel_partner_sudo = request.env['mail.channel.partner']._get_as_sudo_from_request_or_raise(request=request, channel_id=int(channel_id))
-        if not channel_partner_sudo.env['ir.attachment'].search([('id', '=', int(attachment_id)), ('res_id', '=', int(channel_id)), ('res_model', '=', 'mail.channel')], limit=1):
+        attachment_sudo = channel_partner_sudo.env['ir.attachment'].search([
+                ('id', '=', int(attachment_id)),
+                ('res_id', '=', int(channel_id)),
+                ('res_model', '=', 'mail.channel'),
+            ], limit=1)
+
+        if not attachment_sudo:
             raise NotFound()
-        return channel_partner_sudo.env['ir.http']._content_image(res_id=int(attachment_id), height=int(height), width=int(width))
+
+        return request.env['ir.binary']._get_image_stream_from(
+            attachment_sudo, width=width, height=height
+        ).get_response(as_attachment=kwargs.get('download'))
 
     # --------------------------------------------------------------------------
     # Client Initialization

addons/point_of_sale/static/src/js/Chrome.js

@@ -417,12 +417,12 @@ odoo.define('point_of_sale.Chrome', function(require) {
         _preloadImages() {
             for (let product of this.env.pos.db.get_product_by_category(0)) {
                 const image = new Image();
-                image.src = `/web/image?model=product.product&field=image_128&id=${product.id}&write_date=${product.write_date}&unique=1`;
+                image.src = `/web/image?model=product.product&field=image_128&id=${product.id}&unique=${product.write_date}`;
             }
             for (let category of Object.values(this.env.pos.db.category_by_id)) {
                 if (category.id == 0) continue;
                 const image = new Image();
-                image.src = `/web/image?model=pos.category&field=image_128&id=${category.id}&write_date=${category.write_date}&unique=1`;
+                image.src = `/web/image?model=pos.category&field=image_128&id=${category.id}&unique=${category.write_date}`;
             }
             const staticImages = ['backspace.png', 'bc-arrow-big.png'];
             for (let imageName of staticImages) {

addons/point_of_sale/static/src/js/Screens/PartnerListScreen/PartnerDetailsEdit.js

@@ -33,7 +33,7 @@ odoo.define('point_of_sale.PartnerDetailsEdit', function(require) {
             if (this.changes.image_1920) {
                 return this.changes.image_1920;
             } else if (partner.id) {
-                return `/web/image?model=res.partner&id=${partner.id}&field=avatar_128&write_date=${partner.write_date}&unique=1`;
+                return `/web/image?model=res.partner&id=${partner.id}&field=avatar_128&unique=${partner.write_date}`;
             } else {
                 return false;
             }

addons/point_of_sale/static/src/js/Screens/ProductScreen/CategoryButton.js

@@ -7,7 +7,7 @@ odoo.define('point_of_sale.CategoryButton', function(require) {
     class CategoryButton extends PosComponent {
         get imageUrl() {
             const category = this.props.category
-            return `/web/image?model=pos.category&field=image_128&id=${category.id}&write_date=${category.write_date}&unique=1`;
+            return `/web/image?model=pos.category&field=image_128&id=${category.id}&unique=${category.write_date}`;
         }
     }
     CategoryButton.template = 'CategoryButton';

addons/point_of_sale/static/src/js/Screens/ProductScreen/ProductItem.js

@@ -18,7 +18,7 @@ odoo.define('point_of_sale.ProductItem', function(require) {
         }
         get imageUrl() {
             const product = this.props.product;
-            return `/web/image?model=product.product&field=image_128&id=${product.id}&write_date=${product.write_date}&unique=1`;
+            return `/web/image?model=product.product&field=image_128&id=${product.id}&unique=${product.write_date}`;
         }
         get pricelist() {
             const current_order = this.env.pos.get_order();

addons/point_of_sale/static/src/js/models.js

@@ -581,7 +581,7 @@ class PosGlobalState extends PosModel {
         if (order) {
             order.get_orderlines().forEach(function (orderline) {
                 var product = orderline.product;
-                var image_url = `/web/image?model=product.product&field=image_128&id=${product.id}&write_date=${product.write_date}&unique=1`;
+                var image_url = `/web/image?model=product.product&field=image_128&id=${product.id}&unique=${product.write_date}`;
 
                 // only download and convert image if we haven't done it before
                 if (!(product.id in PRODUCT_ID_TO_IMAGE_CACHE)) {

addons/purchase/controllers/portal.py

@@ -93,7 +93,7 @@ class CustomerPortal(portal.CustomerPortal):
         #
         def resize_to_48(source):
             if not source:
-                source = request.env['ir.http']._placeholder()
+                source = request.env['ir.binary']._placeholder()
             else:
                 source = base64.b64decode(source)
             return base64.b64encode(image_process(source, size=(48, 48)))

addons/survey/controllers/main.py

@@ -401,7 +401,9 @@ class Survey(http.Controller):
                 type='http', auth="public", website=True, sitemap=False)
     def survey_get_background(self, survey_token):
         survey_sudo, dummy = self._fetch_from_access_token(survey_token, False)
-        return self._get_background_image(survey_sudo._name, survey_sudo.id)
+        return request.env['ir.binary']._get_image_stream_from(
+            survey_sudo, 'background_image'
+        ).get_response()
 
     @http.route('/survey/<string:survey_token>/<int:section_id>/get_background_image',
                 type='http', auth="public", website=True, sitemap=False)
@@ -413,13 +415,9 @@ class Survey(http.Controller):
             # trying to access a question that is not in this survey
             raise werkzeug.exceptions.Forbidden()
 
-        return self._get_background_image(section._name, section.id)
-
-    def _get_background_image(self, model_name, res_id):
-        status, headers, image_base64 = request.env['ir.http'].sudo().binary_content(
-            model=model_name, id=res_id, field='background_image',
-            default_mimetype='image/png')
-        return request.env['ir.http']._content_image_get_response(status, headers, image_base64)
+        return request.env['ir.binary']._get_image_stream_from(
+            section, 'background_image'
+        ).get_response()
 
     @http.route('/survey/get_question_image/<string:survey_token>/<string:answer_token>/<int:question_id>/<int:suggested_answer_id>', type='http', auth="public", website=True, sitemap=False)
     def survey_get_question_image(self, survey_token, answer_token, question_id, suggested_answer_id):
@@ -429,15 +427,20 @@ class Survey(http.Controller):
 
         survey_sudo, answer_sudo = access_data['survey_sudo'], access_data['answer_sudo']
 
-        if not survey_sudo.question_ids.filtered(lambda q: q.id == question_id)\
-                          .suggested_answer_ids.filtered(lambda a: a.id == suggested_answer_id):
+        suggested_answer = False
+        if int(question_id) in survey_sudo.question_ids.ids:
+            suggested_answer = request.env['survey.question.answer'].sudo().search([
+                ('id', '=', int(suggested_answer_id)),
+                ('question_id', '=', int(question_id)),
+                ('question_id.survey_id', '=', survey_sudo.id),
+            ])
+
+        if not suggested_answer:
             return werkzeug.exceptions.NotFound()
 
-        status, headers, image_base64 = request.env['ir.http'].sudo().binary_content(
-            model='survey.question.answer', id=suggested_answer_id, field='value_image',
-            default_mimetype='image/png')
-
-        return request.env['ir.http']._content_image_get_response(status, headers, image_base64)
+        return request.env['ir.binary']._get_image_stream_from(
+            suggested_answer, 'value_image'
+        ).get_response()
 
     # ----------------------------------------------------------------
     # JSON ROUTES to begin / continue survey (ajax navigation) + Tools

addons/web/__manifest__.py

@@ -20,6 +20,7 @@ This module provides the core of the Odoo Web Client.
         'views/base_document_layout_views.xml',
         'views/speedscope_template.xml',
         'views/lazy_assets.xml',
+        'data/ir_attachment.xml',
         'data/report_layout.xml',
     ],
     'assets': {

addons/web/controllers/binary.py

@@ -8,20 +8,41 @@ import logging
 import os
 import unicodedata
 
+try:
+    from werkzeug.utils import send_file
+except ImportError:
+    from odoo.tools._vendor.send_file import send_file
+
 import odoo
 import odoo.modules.registry
-from odoo import http
-from odoo.exceptions import AccessError
+from odoo import http, _
+from odoo.exceptions import AccessError, UserError
 from odoo.http import request
 from odoo.modules import get_resource_path
-from odoo.tools import pycompat
+from odoo.tools import file_open, file_path, replace_exceptions
 from odoo.tools.mimetypes import guess_mimetype
-from odoo.tools.misc import file_open, file_path
-from odoo.tools.translate import _
+from odoo.tools.image import image_guess_size_from_field_name
 
 
 _logger = logging.getLogger(__name__)
 
+BAD_X_SENDFILE_ERROR = """\
+Odoo is running with --x-sendfile but is receiving /web/filestore requests.
+
+With --x-sendfile enabled, NGINX should be serving the
+/web/filestore route, however Odoo is receiving the
+request.
+
+This usually indicates that NGINX is badly configured,
+please make sure the /web/filestore location block exists
+in your configuration file and that it is similar to:
+
+    location /web/filestore {{
+        internal;
+        alias {data_dir}/filestore;
+    }}
+"""
+
 
 def clean(name):
     return name.replace('\x3c', '')
@@ -29,6 +50,15 @@ def clean(name):
 
 class Binary(http.Controller):
 
+    @http.route('/web/filestore/<path:_path>', type='http', auth='none')
+    def content_filestore(self, _path):
+        if odoo.tools.config['x_sendfile']:
+            # pylint: disable=logging-format-interpolation
+            _logger.error(BAD_X_SENDFILE_ERROR.format(
+                data_dir=odoo.tools.config['data_dir']
+            ))
+        raise http.request.not_found()
+
     @http.route(['/web/content',
         '/web/content/<string:xmlid>',
         '/web/content/<string:xmlid>/<string:filename>',
@@ -36,25 +66,45 @@ class Binary(http.Controller):
         '/web/content/<int:id>/<string:filename>',
         '/web/content/<string:model>/<int:id>/<string:field>',
         '/web/content/<string:model>/<int:id>/<string:field>/<string:filename>'], type='http', auth="public")
-    def content_common(self, xmlid=None, model='ir.attachment', id=None, field='datas',
-                       filename=None, filename_field='name', unique=None, mimetype=None,
-                       download=None, data=None, token=None, access_token=None, **kw):
+    # pylint: disable=redefined-builtin,invalid-name
+    def content_common(self, xmlid=None, model='ir.attachment', id=None, field='raw',
+                       filename=None, filename_field='name', mimetype=None, unique=False,
+                       download=False, access_token=None, nocache=False):
+        with replace_exceptions(UserError, by=request.not_found()):
+            record = request.env['ir.binary']._find_record(xmlid, model, id and int(id), access_token)
+            stream = request.env['ir.binary']._get_stream_from(record, field, filename, filename_field, mimetype)
+        send_file_kwargs = {'as_attachment': download}
+        if unique:
+            send_file_kwargs['max_age'] = http.STATIC_CACHE_LONG
+        if nocache:
+            send_file_kwargs['max_age'] = None
 
-        return request.env['ir.http']._get_content_common(xmlid=xmlid, model=model, res_id=id, field=field, unique=unique, filename=filename,
-            filename_field=filename_field, download=download, mimetype=mimetype, access_token=access_token, token=token)
+        return stream.get_response(**send_file_kwargs)
 
     @http.route(['/web/assets/debug/<string:filename>',
         '/web/assets/debug/<path:extra>/<string:filename>',
         '/web/assets/<int:id>/<string:filename>',
         '/web/assets/<int:id>-<string:unique>/<string:filename>',
         '/web/assets/<int:id>-<string:unique>/<path:extra>/<string:filename>'], type='http', auth="public")
-    def content_assets(self, id=None, filename=None, unique=None, extra=None, **kw):
-        id = id or request.env['ir.attachment'].sudo().search_read(
-            [('url', '=like', f'/web/assets/%/{extra}/{filename}' if extra else f'/web/assets/%/{filename}')],
-             fields=['id'], limit=1)[0]['id']
+    # pylint: disable=redefined-builtin,invalid-name
+    def content_assets(self, id=None, filename=None, unique=False, extra=None, nocache=False):
+        if not id:
+            domain = [('url', '=like', '/web/assets/%/' + (f'{extra}/{filename}' if extra else filename))]
+            attachments = request.env['ir.attachment'].sudo().search_read(domain, fields=['id'], limit=1)
+            if not attachments:
+                raise request.not_found()
+            id = attachments[0]['id']
+        with replace_exceptions(UserError, by=request.not_found()):
+            record = request.env['ir.binary']._find_record(res_id=int(id))
+            stream = request.env['ir.binary']._get_stream_from(record, 'raw', filename)
 
-        return request.env['ir.http']._get_content_common(xmlid=None, model='ir.attachment', res_id=id, field='datas', unique=unique, filename=filename,
-            filename_field='name', download=None, mimetype=None, access_token=None, token=None)
+        send_file_kwargs = {'as_attachment': False}
+        if unique:
+            send_file_kwargs['max_age'] = http.STATIC_CACHE_LONG
+        if nocache:
+            send_file_kwargs['max_age'] = None
+
+        return stream.get_response(as_attachment=False)
 
     @http.route(['/web/image',
         '/web/image/<string:xmlid>',
@@ -73,39 +123,35 @@ class Binary(http.Controller):
         '/web/image/<int:id>-<string:unique>/<string:filename>',
         '/web/image/<int:id>-<string:unique>/<int:width>x<int:height>',
         '/web/image/<int:id>-<string:unique>/<int:width>x<int:height>/<string:filename>'], type='http', auth="public")
+    # pylint: disable=redefined-builtin,invalid-name
     def content_image(self, xmlid=None, model='ir.attachment', id=None, field='raw',
-                      filename_field='name', unique=None, filename=None, mimetype=None,
-                      download=None, width=0, height=0, crop=False, access_token=None,
-                      **kwargs):
-        # other kwargs are ignored on purpose
-        return request.env['ir.http']._content_image(xmlid=xmlid, model=model, res_id=id, field=field,
-            filename_field=filename_field, unique=unique, filename=filename, mimetype=mimetype,
-            download=download, width=width, height=height, crop=crop,
-            quality=int(kwargs.get('quality', 0)), access_token=access_token)
-
-    # backward compatibility
-    @http.route(['/web/binary/image'], type='http', auth="public")
-    def content_image_backward_compatibility(self, model, id, field, resize=None, **kw):
-        width = None
-        height = None
-        if resize:
-            width, height = resize.split(",")
-        return request.env['ir.http']._content_image(model=model, res_id=id, field=field, width=width, height=height)
-
-    @http.route('/web/binary/upload', type='http', auth="user")
-    def upload(self, ufile, callback=None):
-        # TODO: might be useful to have a configuration flag for max-length file uploads
-        out = """<script language="javascript" type="text/javascript">
-                    var win = window.top.window;
-                    win.jQuery(win).trigger(%s, %s);
-                </script>"""
+                      filename_field='name', filename=None, mimetype=None, unique=False,
+                      download=False, width=0, height=0, crop=False, access_token=None,
+                      nocache=False):
         try:
-            data = ufile.read()
-            args = [len(data), ufile.filename,
-                    ufile.content_type, pycompat.to_text(base64.b64encode(data))]
-        except Exception as e:
-            args = [False, str(e)]
-        return out % (json.dumps(clean(callback)), json.dumps(args)) if callback else json.dumps(args)
+            record = request.env['ir.binary']._find_record(xmlid, model, id and int(id), access_token)
+            stream = request.env['ir.binary']._get_image_stream_from(
+                record, field, filename=filename, filename_field=filename_field,
+                mimetype=mimetype, width=int(width), height=int(height), crop=crop,
+            )
+        except UserError as exc:
+            if download:
+                raise request.not_found() from exc
+            # Use the ratio of the requested field_name instead of "raw"
+            if (int(width), int(height)) == (0, 0):
+                width, height = image_guess_size_from_field_name(field)
+            record = request.env.ref('web.image_placeholder').sudo()
+            stream = request.env['ir.binary']._get_image_stream_from(
+                record, 'raw', width=width, height=height, crop=crop,
+            )
+
+        send_file_kwargs = {'as_attachment': download}
+        if unique:
+            send_file_kwargs['max_age'] = http.STATIC_CACHE_LONG
+        if nocache:
+            send_file_kwargs['max_age'] = None
+
+        return stream.get_response(**send_file_kwargs)
 
     @http.route('/web/binary/upload_attachment', type='http', auth="user")
     def upload_attachment(self, model, id, ufile, callback=None):
@@ -161,7 +207,7 @@ class Binary(http.Controller):
         uid = (request.session.uid if dbname else None) or odoo.SUPERUSER_ID
 
         if not dbname:
-            response = http.send_file(placeholder(imgname + imgext))
+            response = http.Stream.from_path(placeholder(imgname + imgext)).get_response()
         else:
             try:
                 # create an empty registry
@@ -188,11 +234,11 @@ class Binary(http.Controller):
                         imgext = '.' + mimetype.split('/')[1]
                         if imgext == '.svg+xml':
                             imgext = '.svg'
-                        response = http.send_file(image_data, filename=imgname + imgext, mimetype=mimetype, mtime=row[1])
+                        response = send_file(image_data, filename=imgname + imgext, mimetype=mimetype, mtime=row[1])
                     else:
-                        response = http.send_file(placeholder('nologo.png'))
+                        response = http.Stream.from_path(placeholder('nologo.png')).get_response()
             except Exception:
-                response = http.send_file(placeholder(imgname + imgext))
+                response = http.Stream.from_path(placeholder(imgname + imgext)).get_response()
 
         return response
 

addons/web/data/ir_attachment.xml

@@ -0,0 +1,10 @@
+<odoo>
+    <data>
+        <record id="image_placeholder" model="ir.attachment">
+            <field name="name">placeholder.png</field>
+            <field name="type">url</field>
+            <field name="url">/web/static/img/placeholder.png</field>
+            <field name="public">True</field>
+        </record>
+    </data>
+</odoo>

addons/web/models/ir_http.py

@@ -170,70 +170,3 @@ class Http(models.AbstractModel):
         Currency = request.env['res.currency']
         currencies = Currency.search([]).read(['symbol', 'position', 'decimal_places'])
         return {c['id']: {'symbol': c['symbol'], 'position': c['position'], 'digits': [69,c['decimal_places']]} for c in currencies}
-
-    @api.model
-    def _get_content_common(self, xmlid=None, model='ir.attachment', res_id=None, field='datas',
-            unique=None, filename=None, filename_field='name', download=None, mimetype=None,
-            access_token=None, token=None):
-        status, headers, content = self.binary_content(
-            xmlid=xmlid, model=model, id=res_id, field=field, unique=unique, filename=filename,
-            filename_field=filename_field, download=download, mimetype=mimetype, access_token=access_token
-        )
-        if status != 200:
-            return self._response_by_status(status, headers, content)
-        else:
-            headers.append(('Content-Length', len(content)))
-            response = request.make_response(content, headers)
-        return response
-
-    @api.model
-    def _content_image(self, xmlid=None, model='ir.attachment', res_id=None, field='raw',
-            filename_field='name', unique=None, filename=None, mimetype=None, download=None,
-            width=0, height=0, crop=False, quality=0, access_token=None, **kwargs):
-        status, headers, image = self.binary_content(
-            xmlid=xmlid, model=model, id=res_id, field=field, unique=unique, filename=filename,
-            filename_field=filename_field, download=download, mimetype=mimetype,
-            default_mimetype='image/png', access_token=access_token
-        )
-        return self._content_image_get_response(
-            status, headers, image, model=model, field=field, download=download,
-            width=width, height=height, crop=crop, quality=quality)
-
-    @api.model
-    def _content_image_get_response(self, status, headers, image, model='ir.attachment',
-            field='raw', download=None, width=0, height=0, crop=False, quality=0):
-        if status in [301, 304] or (status != 200 and download):
-            return self._response_by_status(status, headers, image)
-        if not image:
-            placeholder_filename = False
-            if model in self.env:
-                placeholder_filename = self.env[model]._get_placeholder_filename(field)
-            image = self._placeholder(image=placeholder_filename)
-            # Since we set a placeholder for any missing image, the status must be 200. In case one
-            # wants to configure a specific 404 page (e.g. though nginx), a 404 status will cause
-            # troubles.
-            status = 200
-            if not (width or height):
-                width, height = odoo.tools.image_guess_size_from_field_name(field)
-        try:
-            content = image_process(image, size=(int(width), int(height)), crop=crop, quality=int(quality))
-        except Exception:
-            return request.not_found()
-        headers = http.set_safe_image_headers(headers, content)
-        response = request.make_response(content, headers)
-        response.status_code = status
-        return response
-
-    @api.model
-    def _placeholder_image_get_response(self, content):
-        headers = http.set_safe_image_headers([], content)
-        response = request.make_response(content, headers)
-        response.status_code = 200
-        return response
-
-    @api.model
-    def _placeholder(self, image=False):
-        if not image:
-            image = 'web/static/img/placeholder.png'
-        with file_open(image, 'rb', filter_ext=('.png', '.jpg')) as fd:
-            return fd.read()

addons/web/tests/test_image.py

@@ -18,26 +18,31 @@ class TestImage(HttpCase):
 
         # CASE: resize placeholder, given size but original ratio is always kept
         response = self.url_open('/web/image/0/200x150')
+        response.raise_for_status()
         image = Image.open(io.BytesIO(response.content))
         self.assertEqual(image.size, (150, 150))
 
         # CASE: resize placeholder to 128
         response = self.url_open('/web/image/fake/0/image_128')
+        response.raise_for_status()
         image = Image.open(io.BytesIO(response.content))
         self.assertEqual(image.size, (128, 128))
 
         # CASE: resize placeholder to 256
         response = self.url_open('/web/image/fake/0/image_256')
+        response.raise_for_status()
         image = Image.open(io.BytesIO(response.content))
         self.assertEqual(image.size, (256, 256))
 
         # CASE: resize placeholder to 1024 (but placeholder image is too small)
         response = self.url_open('/web/image/fake/0/image_1024')
+        response.raise_for_status()
         image = Image.open(io.BytesIO(response.content))
         self.assertEqual(image.size, (256, 256))
 
         # CASE: no size found, use placeholder original size
         response = self.url_open('/web/image/fake/0/image_no_size')
+        response.raise_for_status()
         image = Image.open(io.BytesIO(response.content))
         self.assertEqual(image.size, (256, 256))
 
@@ -51,12 +56,14 @@ class TestImage(HttpCase):
             'mimetype': 'image/gif',
         })
         response = self.url_open('/web/image/%s' % attachment.id, timeout=None)
+        response.raise_for_status()
         self.assertEqual(response.status_code, 200)
         self.assertEqual(base64.b64encode(response.content), attachment.datas)
 
         etag = response.headers.get('ETag')
 
         response2 = self.url_open('/web/image/%s' % attachment.id, headers={"If-None-Match": etag})
+        response2.raise_for_status()
         self.assertEqual(response2.status_code, 304)
         self.assertEqual(len(response2.content), 0)
 
@@ -72,12 +79,15 @@ class TestImage(HttpCase):
 
         # CASE: no filename given
         res = self.url_open('/web/image/%s/0x0/?download=true' % att.id)
-        self.assertEqual(res.headers['Content-Disposition'], content_disposition('testFilename.gif'))
+        res.raise_for_status()
+        self.assertEqual(res.headers['Content-Disposition'], 'attachment; filename=testFilename.gif')
 
         # CASE: given filename without extension
         res = self.url_open('/web/image/%s/0x0/custom?download=true' % att.id)
-        self.assertEqual(res.headers['Content-Disposition'], content_disposition('custom.gif'))
+        res.raise_for_status()
+        self.assertEqual(res.headers['Content-Disposition'], 'attachment; filename=custom.gif')
 
         # CASE: given filename and extention
         res = self.url_open('/web/image/%s/0x0/custom.png?download=true' % att.id)
-        self.assertEqual(res.headers['Content-Disposition'], content_disposition('custom.png'))
+        res.raise_for_status()
+        self.assertEqual(res.headers['Content-Disposition'], 'attachment; filename=custom.png')

addons/web_editor/controllers/main.py

@@ -1,11 +1,12 @@
-# -*- coding: utf-8 -*-
 # Part of Odoo. See LICENSE file for full copyright and licensing details.
+import contextlib
 import io
 import json
 import logging
 import re
 import time
 import requests
+import werkzeug.exceptions
 import werkzeug.urls
 import werkzeug.wrappers
 from PIL import Image, ImageFont, ImageDraw
@@ -16,7 +17,7 @@ from odoo.http import request
 from odoo import http, tools, _, SUPERUSER_ID
 from odoo.addons.http_routing.models.ir_http import slug, unslug
 from odoo.addons.web_editor.tools import get_video_url_data
-from odoo.exceptions import UserError
+from odoo.exceptions import UserError, MissingError
 from odoo.modules.module import get_resource_path
 from odoo.tools import file_open
 from odoo.tools.mimetypes import guess_mimetype
@@ -262,15 +263,17 @@ class Web_Editor(http.Controller):
         it can be used as a base to modify it again (crop/optimization/filters).
         """
         attachment = None
-        id_match = re.search('^/web/image/([^/?]+)', src)
-        if id_match:
-            url_segment = id_match.group(1)
-            number_match = re.match('^(\d+)', url_segment)
-            if '.' in url_segment: # xml-id
-                attachment = request.env['ir.http']._xmlid_to_obj(request.env, url_segment)
-            elif number_match: # numeric id
-                attachment = request.env['ir.attachment'].browse(int(number_match.group(1)))
-        else:
+        if src.startswith('/web/image'):
+            with contextlib.suppress(werkzeug.exceptions.NotFound, MissingError):
+                _, args = request.env['ir.http']._match(src)
+                record = request.env['ir.binary']._find_record(
+                    xmlid=args.get('xmlid'),
+                    res_model=args.get('model'),
+                    res_id=args.get('id'),
+                )
+                if record._name == 'ir.attachment':
+                    attachment = record
+        if not attachment:
             # Find attachment by url. There can be multiple matches because of default
             # snippet images referencing the same image in /static/, so we limit to 1
             attachment = request.env['ir.attachment'].search([
@@ -617,10 +620,18 @@ class Web_Editor(http.Controller):
     @http.route(['/web_editor/image_shape/<string:img_key>/<module>/<path:filename>'], type='http', auth="public", website=True)
     def image_shape(self, module, filename, img_key, **kwargs):
         svg = self._get_shape_svg(module, 'image_shapes', filename)
-        _, _, image = request.env['ir.http'].binary_content(
-            xmlid=img_key, model='ir.attachment', field='datas', default_mimetype='image/png')
-        if not image:
-            image = request.env['ir.http']._placeholder()
+
+        record = request.env['ir.binary']._find_record(img_key)
+        stream = request.env['ir.binary']._get_image_stream_from(record)
+        if stream.type == 'url':
+            return stream.get_response()
+
+        if stream.type == 'path':
+            with file_open(stream.path, 'rb') as file:
+                image = file.read()
+        else:
+            image = stream.data
+
         img = binary_to_image(image)
         width, height = tuple(str(size) for size in img.size)
         root = etree.fromstring(svg)

addons/website/controllers/main.py

@@ -764,33 +764,7 @@ class Website(Home):
         return request.redirect('/')
 
 
-# ------------------------------------------------------
-# Retrocompatibility routes
-# ------------------------------------------------------
 class WebsiteBinary(http.Controller):
-
-    @http.route([
-        '/website/image',
-        '/website/image/<xmlid>',
-        '/website/image/<xmlid>/<int:width>x<int:height>',
-        '/website/image/<xmlid>/<field>',
-        '/website/image/<xmlid>/<field>/<int:width>x<int:height>',
-        '/website/image/<model>/<id>/<field>',
-        '/website/image/<model>/<id>/<field>/<int:width>x<int:height>'
-    ], type='http', auth="public", website=False, multilang=False)
-    def content_image(self, id=None, max_width=0, max_height=0, **kw):
-        if max_width:
-            kw['width'] = max_width
-        if max_height:
-            kw['height'] = max_height
-        if id:
-            id, _, unique = id.partition('_')
-            kw['id'] = int(id)
-            if unique:
-                kw['unique'] = unique
-        kw['res_id'] = kw.pop('id', None)
-        return request.env['ir.http']._content_image(**kw)
-
     # if not icon provided in DOM, browser tries to access /favicon.ico, eg when opening an order pdf
     @http.route(['/favicon.ico'], type='http', auth='public', website=True, multilang=False, sitemap=False)
     def favicon(self, **kw):

addons/website/models/__init__.py

@@ -5,6 +5,7 @@ from . import assets
 from . import ir_actions
 from . import ir_asset
 from . import ir_attachment
+from . import ir_binary
 from . import ir_http
 from . import ir_model
 from . import ir_model_data

addons/website/models/ir_attachment.py

@@ -27,9 +27,8 @@ class Attachment(models.Model):
     def get_serving_groups(self):
         return super(Attachment, self).get_serving_groups() + ['website.group_website_designer']
 
-    @api.model
-    def get_serve_attachment(self, url, extra_domain=None, extra_fields=None, order=None):
+    def _get_serve_attachment(self, url, extra_domain=None, order=None):
         website = self.env['website'].get_current_website()
         extra_domain = (extra_domain or []) + website.website_domain()
         order = ('website_id, %s' % order) if order else 'website_id'
-        return super(Attachment, self).get_serve_attachment(url, extra_domain, extra_fields, order)
+        return super()._get_serve_attachment(url, extra_domain, order)

addons/website/models/ir_binary.py

@@ -0,0 +1,28 @@
+from odoo import models
+
+
+class IrBinary(models.AbstractModel):
+    _inherit = 'ir.binary'
+
+    def _find_record(
+            self, xmlid=None, res_model='ir.attachment', res_id=None,
+            access_token=None,
+    ):
+        record = None
+        if xmlid:
+            website = self.env['website'].get_current_website()
+            if website.theme_id:
+                domain = [('key', '=', xmlid), ('website_id', '=', website.id)]
+                Attachment = self.env['ir.attachment']
+                if self.env.user.share:
+                    domain.append(('public', '=', True))
+                    Attachment = Attachment.sudo()
+                record = Attachment.search(domain, limit=1)
+
+        if not record:
+            record = super()._find_record(xmlid, res_model, res_id, access_token)
+
+        if 'website_published' in record and record.sudo().website_published:
+            record = record.sudo()
+
+        return record

addons/website/models/ir_http.py

@@ -398,41 +398,6 @@ class Http(models.AbstractModel):
             return code.split('_')[1], env['ir.ui.view']._render_template('website.%s' % code, values)
         return super()._get_error_html(env, code, values)
 
-    def binary_content(self, xmlid=None, model='ir.attachment', id=None, field='datas',
-                       unique=False, filename=None, filename_field='name', download=False,
-                       mimetype=None, default_mimetype='application/octet-stream',
-                       access_token=None):
-        obj = None
-        if xmlid:
-            obj = self._xmlid_to_obj(self.env, xmlid)
-        elif id and model in self.env:
-            obj = self.env[model].browse(int(id))
-        if obj and 'website_published' in obj._fields:
-            try:
-                if obj.sudo().website_published:
-                    self = self.sudo()
-            except MissingError:
-                pass
-        return super(Http, self).binary_content(
-            xmlid=xmlid, model=model, id=id, field=field, unique=unique, filename=filename,
-            filename_field=filename_field, download=download, mimetype=mimetype,
-            default_mimetype=default_mimetype, access_token=access_token)
-
-    @classmethod
-    def _xmlid_to_obj(cls, env, xmlid):
-        website_id = env['website'].get_current_website()
-        if website_id and website_id.theme_id:
-            domain = [('key', '=', xmlid), ('website_id', '=', website_id.id)]
-            Attachment = env['ir.attachment']
-            if request.env.user.share:
-                domain.append(('public', '=', True))
-                Attachment = Attachment.sudo()
-            obj = Attachment.search(domain)
-            if obj:
-                return obj[0]
-
-        return super()._xmlid_to_obj(env, xmlid)
-
     @api.model
     def get_frontend_session_info(self):
         session_info = super(Http, self).get_frontend_session_info()

addons/website/tests/test_performance.py

@@ -43,16 +43,16 @@ class TestStandardPerformance(UtilPerf):
         self.authenticate('demo', 'demo')
         self.env['res.users'].sudo().browse(2).website_published = True
         url = '/web/image/res.users/2/image_256'
-        self.assertEqual(self._get_url_hot_query(url), 5)
-        self.assertEqual(self._get_url_hot_query(url, cache=False), 5)
+        self.assertEqual(self._get_url_hot_query(url), 6)
+        self.assertEqual(self._get_url_hot_query(url, cache=False), 6)
 
     def test_20_perf_sql_img_controller_bis(self):
         url = '/web/image/website/1/favicon'
-        self.assertEqual(self._get_url_hot_query(url), 4)
-        self.assertEqual(self._get_url_hot_query(url, cache=False), 4)
+        self.assertEqual(self._get_url_hot_query(url), 5)
+        self.assertEqual(self._get_url_hot_query(url, cache=False), 5)
         self.authenticate('portal', 'portal')
-        self.assertEqual(self._get_url_hot_query(url), 4)
-        self.assertEqual(self._get_url_hot_query(url, cache=False), 4)
+        self.assertEqual(self._get_url_hot_query(url), 5)
+        self.assertEqual(self._get_url_hot_query(url, cache=False), 5)
 
 
 class TestWebsitePerformance(UtilPerf):
@@ -138,5 +138,5 @@ class TestWebsitePerformance(UtilPerf):
         # assets route /web/assets/..
         self.url_open('/')  # create assets attachments
         assets_url = self.env['ir.attachment'].search([('url', '=like', '/web/assets/%/web.assets_common%.js')], limit=1).url
-        self.assertEqual(self._get_url_hot_query(assets_url), 2)
-        self.assertEqual(self._get_url_hot_query(assets_url, cache=False), 2)
+        self.assertEqual(self._get_url_hot_query(assets_url), 4)
+        self.assertEqual(self._get_url_hot_query(assets_url, cache=False), 4)

addons/website_profile/controllers/main.py

@@ -35,10 +35,6 @@ class WebsiteProfile(http.Controller):
             return user.website_published and user.karma > 0
         return False
 
-    def _get_default_avatar(self):
-        with tools.file_open("web/static/img/placeholder.png", 'rb') as f:
-            return f.read()
-
     def _check_user_profile_access(self, user_id):
         user_sudo = request.env['res.users'].sudo().browse(user_id)
         # User can access - no matter what - his own profile
@@ -79,30 +75,14 @@ class WebsiteProfile(http.Controller):
         if field not in ('image_128', 'image_256', 'avatar_128', 'avatar_256'):
             return werkzeug.exceptions.Forbidden()
 
-        can_sudo = self._check_avatar_access(user_id, **post)
-        if can_sudo:
-            status, headers, image = request.env['ir.http'].sudo().binary_content(
-                model='res.users', id=user_id, field=field,
-                default_mimetype='image/png')
-        else:
-            status, headers, image = request.env['ir.http'].binary_content(
-                model='res.users', id=user_id, field=field,
-                default_mimetype='image/png')
-        if status == 301:
-            return request.env['ir.http']._response_by_status(status, headers, image)
-        if status == 304:
-            return werkzeug.wrappers.Response(status=304)
+        if (int(width), int(height)) == (0, 0):
+            width, height = tools.image_guess_size_from_field_name(field)
 
-        if not image:
-            image = self._get_default_avatar()
-            if not (width or height):
-                width, height = tools.image_guess_size_from_field_name(field)
-
-        content = tools.image_process(image, size=(int(width), int(height)), crop=crop)
-        headers = http.set_safe_image_headers(headers, content)
-        response = request.make_response(content, headers)
-        response.status_code = status
-        return response
+        can_sudo = self._check_avatar_access(int(user_id), **post)
+        return request.env['ir.binary']._get_image_stream_from(
+            request.env['res.users'].sudo(can_sudo).browse(int(user_id)),
+            field_name=field, width=int(width), height=int(height), crop=crop
+        ).get_response()
 
     @http.route(['/profile/user/<int:user_id>'], type='http', auth="public", website=True)
     def view_user_profile(self, user_id, **post):

addons/website_slides/controllers/main.py

@@ -819,25 +819,9 @@ class WebsiteSlides(WebsiteProfile):
         if not slide:
             raise werkzeug.exceptions.NotFound()
 
-        status, headers, image = request.env['ir.http'].sudo().binary_content(
-            model='slide.slide', id=slide.id, field=field,
-            default_mimetype='image/png')
-        if status == 301:
-            return request.env['ir.http']._response_by_status(status, headers, image)
-        if status == 304:
-            return werkzeug.wrappers.Response(status=304)
-
-        if not image:
-            image = self._get_default_avatar()
-            if not (width or height):
-                width, height = tools.image_guess_size_from_field_name(field)
-
-        content = tools.image_process(image, size=(int(width), int(height)), crop=crop)
-
-        headers = http.set_safe_image_headers(headers, content)
-        response = request.make_response(content, headers)
-        response.status_code = status
-        return response
+        return request.env['ir.binary']._get_image_stream_from(
+            slide, field, width=width, height=int(height), crop=int(crop)
+        ).get_response()
 
     # SLIDE.SLIDE UTILS
     # --------------------------------------------------

addons/website_slides/models/__init__.py

@@ -1,7 +1,5 @@
-# -*- coding: utf-8 -*-
 # Part of Odoo. See LICENSE file for full copyright and licensing details.
 
-from . import ir_http
 from . import gamification_challenge
 from . import slide_slide
 from . import slide_question

addons/website_slides/models/ir_http.py

@@ -1,27 +0,0 @@
-# -*- coding: utf-8 -*-
-# Part of Odoo. See LICENSE file for full copyright and licensing details.
-
-from odoo import models
-
-
-class Http(models.AbstractModel):
-    _inherit = 'ir.http'
-
-    def binary_content(self, xmlid=None, model='ir.attachment', id=None, field='datas',
-                       unique=False, filename=None, filename_field='name', download=False,
-                       mimetype=None, default_mimetype='application/octet-stream',
-                       access_token=None):
-        obj = None
-        if xmlid:
-            obj = self._xmlid_to_obj(self.env, xmlid)
-            if obj and obj._name != 'slide.slide':
-                obj = None
-        elif id and model == 'slide.slide':
-            obj = self.env[model].browse(int(id))
-        if obj:
-            obj.check_access_rights('read')
-            obj.check_access_rule('read')
-        return super(Http, self).binary_content(
-            xmlid=xmlid, model=model, id=id, field=field, unique=unique, filename=filename,
-            filename_field=filename_field, download=download, mimetype=mimetype,
-            default_mimetype=default_mimetype, access_token=access_token)

odoo/addons/base/models/__init__.py

@@ -1,4 +1,3 @@
-# -*- coding: utf-8 -*-
 # Part of Odoo. See LICENSE file for full copyright and licensing details.
 
 from . import assetsbundle
@@ -11,6 +10,7 @@ from . import ir_asset
 from . import ir_actions
 from . import ir_actions_report
 from . import ir_attachment
+from . import ir_binary
 from . import ir_cron
 from . import ir_filters
 from . import ir_default

odoo/addons/base/models/assetsbundle.py

@@ -757,9 +757,9 @@ class WebAsset(object):
                 return
             try:
                 # Test url against ir.attachments
-                attach = self.bundle.env['ir.attachment'].sudo().get_serve_attachment(self.url)
-                self._ir_attach = attach[0]
-            except Exception:
+                self._ir_attach = self.bundle.env['ir.attachment'].sudo()._get_serve_attachment(self.url)
+                self._ir_attach.ensure_one()
+            except ValueError:
                 raise AssetNotFound("Could not find %s" % self.name)
 
     def to_node(self):
@@ -791,7 +791,7 @@ class WebAsset(object):
                 with closing(file_open(self._filename, 'rb', filter_ext=EXTENSIONS)) as fp:
                     return fp.read().decode('utf-8')
             else:
-                return base64.b64decode(self._ir_attach['datas']).decode('utf-8')
+                return self._ir_attach.raw.decode()
         except UnicodeDecodeError:
             raise AssetError('%s is not utf-8 encoded.' % self.name)
         except IOError:

odoo/addons/base/models/ir_attachment.py

@@ -1,5 +1,5 @@
-# -*- coding: utf-8 -*-
 # Part of Odoo. See LICENSE file for full copyright and licensing details.
+
 import base64
 import hashlib
 import io
@@ -15,7 +15,7 @@ from PIL import Image
 
 from odoo import api, fields, models, tools, _
 from odoo.exceptions import AccessError, ValidationError, UserError
-from odoo.tools import config, human_size, ImageProcess, str2bool
+from odoo.tools import config, human_size, ImageProcess, str2bool, consteq
 from odoo.tools.mimetypes import guess_mimetype
 from odoo.osv import expression
 
@@ -673,12 +673,33 @@ class IrAttachment(models.Model):
     def _generate_access_token(self):
         return str(uuid.uuid4())
 
+    def validate_access(self, access_token):
+        self.ensure_one()
+        record_sudo = self.sudo()
+
+        if access_token:
+            tok = record_sudo.with_context(prefetch_fields=False).access_token
+            valid_token = consteq(tok or '', access_token)
+            if not valid_token:
+                raise AccessError("Invalid access token")
+            return record_sudo
+
+        if record_sudo.with_context(prefetch_fields=False).public:
+            return record_sudo
+
+        if self.env.user.has_group('base.group_portal'):
+            # Check the read access on the record linked to the attachment
+            # eg: Allow to download an attachment on a task from /my/tasks/task_id
+            self.check('read')
+            return record_sudo
+
+        return self
+
     @api.model
     def action_get(self):
         return self.env['ir.actions.act_window']._for_xml_id('base.action_attachment')
 
     @api.model
-    def get_serve_attachment(self, url, extra_domain=None, extra_fields=None, order=None):
+    def _get_serve_attachment(self, url, extra_domain=None, order=None):
         domain = [('type', '=', 'binary'), ('url', '=', url)] + (extra_domain or [])
-        fieldNames = ['__last_update', 'datas', 'mimetype'] + (extra_fields or [])
-        return self.search_read(domain, fieldNames, order=order, limit=1)
+        return self.search(domain, order=order, limit=1)

odoo/addons/base/models/ir_binary.py

@@ -0,0 +1,247 @@
+import logging
+import werkzeug.http
+from datetime import datetime
+from mimetypes import guess_extension
+
+from odoo import models
+from odoo.exceptions import MissingError, UserError
+from odoo.http import Stream, request
+from odoo.tools import file_open, replace_exceptions
+from odoo.tools.image import image_process, image_guess_size_from_field_name
+from odoo.tools.mimetypes import guess_mimetype, get_extension
+
+
+DEFAULT_PLACEHOLDER_PATH = 'web/static/img/placeholder.png'
+_logger = logging.getLogger(__name__)
+
+
+class IrBinary(models.AbstractModel):
+    _name = 'ir.binary'
+    _description = "File streaming helper model for controllers"
+
+    def _find_record(
+            self, xmlid=None, res_model='ir.attachment', res_id=None,
+            access_token=None,
+    ):
+        """
+        Find and return a record either using an xmlid either a model+id
+        pair. This method is an helper for the ``/web/content`` and
+        ``/web/image`` controllers and should not be used in other
+        contextes.
+
+        :param Optional[str] xmlid: xmlid of the record
+        :param Optional[str] res_model: model of the record,
+            ir.attachment by default.
+        :param Optional[id] res_id: id of the record
+        :param Optional[str] access_token: access token to use instead
+            of the access rights and access rules.
+        :returns: single record
+        :raises MissingError: when no record was found.
+        """
+        record = None
+        if xmlid:
+            record = self.env.ref(xmlid, False)
+        elif res_id is not None and res_model in self.env:
+            record = self.env[res_model].browse(res_id).exists()
+        if not record:
+            raise MissingError(f"No record found for xmlid={xmlid}, res_model={res_model}, id={res_id}")
+
+        if record._name == 'ir.attachment':
+            record = record.validate_access(access_token)
+
+        return record
+
+    def _record_to_stream(self, record, field_name):
+        """
+        Low level method responsible for the actual conversion from a
+        model record to a stream. This method is an extensible hook for
+        other modules. It is not meant to be directly called from
+        outside or the ir.binary model.
+
+        :param record: the record where to load the data from.
+        :param str field_name: the binary field where to load the data
+            from.
+        :rtype: odoo.http.Stream
+        """
+        if record._name == 'ir.attachment' and field_name in ('raw', 'datas', 'db_datas'):
+            return Stream.from_attachment(record)
+
+        field_def = record._fields[field_name]
+
+        # fields.Binary(attachment=False) or compute/related
+        if not field_def.attachment or field_def.compute or field_def.related:
+            return Stream.from_binary_field(record, field_name)
+
+        # fields.Binary(attachment=True)
+        field_attachment = self.env['ir.attachment'].sudo().search(
+            domain=[('res_model', '=', record._name),
+                    ('res_id', '=', record.id),
+                    ('res_field', '=', field_name)],
+            limit=1)
+        if not field_attachment:
+            raise MissingError("The related attachment does not exist.")
+        return Stream.from_attachment(field_attachment)
+
+    def _get_stream_from(
+        self, record, field_name='raw', filename=None, filename_field='name',
+        mimetype=None, default_mimetype='application/octet-stream',
+    ):
+        """
+        Create a :class:odoo.http.Stream: from a record's binary field.
+
+        :param record: the record where to load the data from.
+        :param str field_name: the binary field where to load the data
+            from.
+        :param Optional[str] filename: when the stream is downloaded by
+            a browser, what filename it should have on disk. By default
+            it is ``{model}-{id}-{field}.{extension}``, the extension is
+            determined thanks to mimetype.
+        :param Optional[str] filename_field: like ``filename`` but use
+            one of the record's char field as filename.
+        :param Optional[str] mimetype: the data mimetype to use instead
+            of the stored one (attachment) or the one determined by
+            magic.
+        :param str default_mimetype: the mimetype to use when the
+            mimetype couldn't be determined. By default it is
+            ``application/octet-stream``.
+        :rtype: odoo.http.Stream
+        """
+        with replace_exceptions(ValueError, by=UserError(f'Expected singleton: {record}')):
+            record.ensure_one()
+
+        try:
+            field_def = record._fields[field_name]
+        except KeyError:
+            raise UserError(f"Record has no field {field_name!r}.")
+        if field_def.type != 'binary':
+            raise UserError(
+                f"Field {field_def!r} is type {field_def.type!r} but "
+                f"it is only possible to stream Binary or Image fields."
+            )
+
+        stream = self._record_to_stream(record, field_name)
+
+        if stream.type in ('data', 'path'):
+            if mimetype:
+                stream.mimetype = mimetype
+            elif not stream.mimetype:
+                if stream.type == 'data':
+                    head = stream.data[:1024]
+                else:
+                    with open(stream.path, 'rb') as file:
+                        head = file.read(1024)
+                stream.mimetype = guess_mimetype(head, default=default_mimetype)
+
+            if filename:
+                stream.download_name = filename
+            elif filename_field in record:
+                stream.download_name = record[filename_field]
+            if not stream.download_name:
+                stream.download_name = f'{record._table}-{record.id}-{field_name}'
+
+            if (not get_extension(stream.download_name)
+                and stream.mimetype != 'application/octet-stream'):
+                stream.download_name += guess_extension(stream.mimetype) or ''
+
+        return stream
+
+    def _get_image_stream_from(
+        self, record, field_name='raw', filename=None, filename_field='name',
+        mimetype=None, default_mimetype='image/png', placeholder=None,
+        width=0, height=0, crop=False, quality=0,
+    ):
+        """
+        Create a :class:odoo.http.Stream: from a record's binary field,
+        equivalent of :meth:`~get_stream_from` but for images.
+
+        In case the record does not exist or is not accessible, the
+        alternative ``placeholder`` path is used instead. If not set,
+        a path is determined via
+        :meth:`~odoo.models.BaseModel._get_placeholder_filename` which
+        ultimately fallbacks on ``web/static/img/placeholder.png``.
+
+        In case the arguments ``width``, ``height``, ``crop`` or
+        ``quality`` are given, the image will be post-processed and the
+        ETags (the unique cache http header) will be updated
+        accordingly. See also :func:`odoo.tools.image.image_process`.
+
+        :param record: the record where to load the data from.
+        :param str field_name: the binary field where to load the data
+            from.
+        :param Optional[str] filename: when the stream is downloaded by
+            a browser, what filename it should have on disk. By default
+            it is ``{table}-{id}-{field}.{extension}``, the extension is
+            determined thanks to mimetype.
+        :param Optional[str] filename_field: like ``filename`` but use
+            one of the record's char field as filename.
+        :param Optional[str] mimetype: the data mimetype to use instead
+            of the stored one (attachment) or the one determined by
+            magic.
+        :param str default_mimetype: the mimetype to use when the
+            mimetype couldn't be determined. By default it is
+            ``image/png``.
+        :param Optional[pathlike] placeholder: in case the image is not
+            found or unaccessible, the path of an image to use instead.
+            By default the record ``_get_placeholder_filename`` on the
+            requested field or ``web/static/img/placeholder.png``.
+        :param int width: if not zero, the width of the resized image.
+        :param int height: if not zero, the height of the resized image.
+        :param bool crop: if true, crop the image instead of rezising
+            it.
+        :param int quality: if not zero, the quality of the resized
+            image.
+
+        """
+        try:
+            stream = self._get_stream_from(
+                record, field_name, filename, filename_field, mimetype,
+                default_mimetype
+            )
+        except UserError:
+            if request.params.get('download'):
+                raise
+            if not placeholder:
+                placeholder = record._get_placeholder_filename(field_name)
+            stream = self._get_placeholder_stream(placeholder)
+
+        if stream.type == 'url':
+            return stream  # Rezising an external URL is not supported
+
+        if (width, height) == (0, 0):
+            width, height = image_guess_size_from_field_name(field_name)
+        stream.etag += f'-{width}x{height}-crop={crop}-quality={quality}'
+
+        if isinstance(stream.last_modified, (int, float)):
+            stream.last_modified = datetime.utcfromtimestamp(stream.last_modified)
+        modified = werkzeug.http.is_resource_modified(
+            request.httprequest.environ,
+            etag=stream.etag,
+            last_modified=stream.last_modified
+        )
+
+        if modified and (width or height or crop):
+            if stream.type == 'path':
+                with open(stream.path, 'rb') as file:
+                    stream.type = 'data'
+                    stream.path = None
+                    stream.data = file.read()
+            stream.data = image_process(
+                stream.data,
+                size=(width, height),
+                crop=crop,
+                quality=quality,
+            )
+            stream.size = len(stream.data)
+
+        return stream
+
+    def _get_placeholder_stream(self, path=None):
+        if not path:
+            path = DEFAULT_PLACEHOLDER_PATH
+        return Stream.from_path(path, filter_ext=('.png', '.jpg'))
+
+    def _placeholder(self, path=False):
+        if not path:
+            path = DEFAULT_PLACEHOLDER_PATH
+        with file_open(path, 'rb', filter_ext=('.png', '.jpg')) as file:
+            return file.read()

odoo/addons/base/models/ir_http.py

@@ -19,10 +19,9 @@ import werkzeug.utils
 import odoo
 from odoo import api, http, models, tools, SUPERUSER_ID
 from odoo.exceptions import AccessDenied, AccessError, MissingError
-from odoo.http import request, content_disposition, Response, ROUTING_KEYS
+from odoo.http import request, Response, ROUTING_KEYS, Stream
 from odoo.service import security
 from odoo.tools import consteq, submap
-from odoo.tools.mimetypes import get_extension, guess_mimetype
 from odoo.modules.module import get_resource_path, get_module_path
 
 _logger = logging.getLogger(__name__)
@@ -147,39 +146,12 @@ class IrHttp(models.AbstractModel):
     def _handle_error(cls, exception):
         return request.dispatcher.handle_error(exception)
 
-    @classmethod
-    def _serve_attachment(cls):
-        env = request.env(user=SUPERUSER_ID)
-        attach = env['ir.attachment'].get_serve_attachment(request.httprequest.path, extra_fields=['name', 'checksum'])
-        if attach:
-            wdate = attach[0]['__last_update']
-            datas = attach[0]['datas'] or b''
-            name = attach[0]['name']
-            checksum = attach[0]['checksum'] or hashlib.sha512(datas).hexdigest()[:64]  # sha512/256
-
-            if (not datas and name != request.httprequest.path and
-                    name.startswith(('http://', 'https://', '/'))):
-                return request.redirect(name, 301, local=False)
-
-            response = werkzeug.wrappers.Response()
-            response.last_modified = wdate
-
-            response.set_etag(checksum)
-            response.make_conditional(request.httprequest)
-
-            if response.status_code == 304:
-                return response
-
-            response.mimetype = attach[0]['mimetype'] or 'application/octet-stream'
-            response.data = base64.b64decode(datas)
-            return response
-
     @classmethod
     def _serve_fallback(cls):
-        # serve attachment
-        attach = cls._serve_attachment()
+        model = request.env['ir.attachment']
+        attach = model.sudo()._get_serve_attachment(request.httprequest.path)
         if attach:
-            return attach
+            return Stream.from_attachment(attach).get_response()
 
     @classmethod
     def _redirect(cls, location, code=303):
@@ -226,210 +198,3 @@ class IrHttp(models.AbstractModel):
     @api.autovacuum
     def _gc_sessions(self):
         http.root.session_store.vacuum()
-
-    #------------------------------------------------------
-    # Binary server
-    #------------------------------------------------------
-
-    @classmethod
-    def _xmlid_to_obj(cls, env, xmlid):
-        return env.ref(xmlid, False)
-
-    def _get_record_and_check(self, xmlid=None, model=None, id=None, field='datas', access_token=None):
-        # get object and content
-        record = None
-        if xmlid:
-            record = self._xmlid_to_obj(self.env, xmlid)
-        elif id and model in self.env:
-            record = self.env[model].browse(int(id))
-
-        # obj exists
-        if not record or field not in record:
-            return None, 404
-
-        try:
-            if model == 'ir.attachment':
-                record_sudo = record.sudo()
-                if access_token and not consteq(record_sudo.access_token or '', access_token):
-                    return None, 403
-                elif (access_token and consteq(record_sudo.access_token or '', access_token)):
-                    record = record_sudo
-                elif record_sudo.public:
-                    record = record_sudo
-                elif self.env.user.has_group('base.group_portal'):
-                    # Check the read access on the record linked to the attachment
-                    # eg: Allow to download an attachment on a task from /my/tasks/task_id
-                    record.check('read')
-                    record = record_sudo
-
-            # check read access
-            try:
-                # We have prefetched some fields of record, among which the field
-                # 'write_date' used by '__last_update' below. In order to check
-                # access on record, we have to invalidate its cache first.
-                if not record.env.su:
-                    record._cache.clear()
-                record['__last_update']
-            except AccessError:
-                return None, 403
-
-            return record, 200
-        except MissingError:
-            return None, 404
-
-    @classmethod
-    def _binary_ir_attachment_redirect_content(cls, record, default_mimetype='application/octet-stream'):
-        # mainly used for theme images attachemnts
-        status = content = filename = filehash = None
-        mimetype = getattr(record, 'mimetype', False)
-        if record.type == 'url' and record.url:
-            # if url in in the form /somehint server locally
-            url_match = re.match("^/(\w+)/(.+)$", record.url)
-            if url_match:
-                module = url_match.group(1)
-                module_path = get_module_path(module)
-                module_resource_path = get_resource_path(module, url_match.group(2))
-
-                if module_path and module_resource_path:
-                    module_path = os.path.join(os.path.normpath(module_path), '')  # join ensures the path ends with '/'
-                    module_resource_path = os.path.normpath(module_resource_path)
-                    if module_resource_path.startswith(module_path):
-                        with open(module_resource_path, 'rb') as f:
-                            content = f.read()
-                        status = 200
-                        filename = os.path.basename(module_resource_path)
-                        mimetype = record.mimetype
-                        filehash = record.checksum
-
-            if not content:
-                status = 301
-                content = record.url
-
-        return status, content, filename, mimetype, filehash
-
-    def _binary_record_content(
-            self, record, field='raw', filename=None,
-            filename_field='name', default_mimetype='application/octet-stream'):
-
-        model = record._name
-        mimetype = 'mimetype' in record and record.mimetype or False
-        content = None
-        filehash = 'checksum' in record and record['checksum'] or False
-
-        field_def = record._fields[field]
-        if field_def.type == 'binary' and field_def.attachment and not field_def.related:
-            if model != 'ir.attachment':
-                field_attachment = self.env['ir.attachment'].sudo().search_read(domain=[('res_model', '=', model), ('res_id', '=', record.id), ('res_field', '=', field)], fields=['raw', 'mimetype', 'checksum'], limit=1)
-                if field_attachment:
-                    mimetype = field_attachment[0]['mimetype']
-                    content = field_attachment[0]['raw']
-                    filehash = field_attachment[0]['checksum']
-            else:
-                mimetype = record['mimetype']
-                content = record['raw']
-                filehash = record['checksum']
-
-        if not content:
-            if model == 'ir.attachment' and field in {'datas', 'raw'}:
-                content = record.raw
-            elif (
-                field_def.related_field and
-                field_def.related_field.name == 'raw' and
-                field_def.related_field.model_name == 'ir.attachment'
-            ):
-                content = record[field] or b''
-            else:
-                data = record[field] or b''
-                content = base64.b64decode(data)
-                filehash = '"%s"' % hashlib.md5(str(content).encode('utf-8')).hexdigest()
-
-        # filename
-        if not filename:
-            if filename_field in record:
-                filename = record[filename_field]
-            if not filename:
-                filename = "%s-%s-%s" % (record._name, record.id, field)
-
-        if not mimetype:
-            mimetype = guess_mimetype(content, default=default_mimetype)
-
-        # extension
-        has_extension = get_extension(filename) or mimetypes.guess_type(filename)[0]
-        if not has_extension:
-            extension = mimetypes.guess_extension(mimetype)
-            if extension:
-                filename = "%s%s" % (filename, extension)
-
-        if not filehash:
-            filehash = '"%s"' % hashlib.md5(str(base64.b64encode(content)).encode('utf-8')).hexdigest()
-
-        status = 200 if content else 404
-        return status, content, filename, mimetype, filehash
-
-    def _binary_set_headers(self, status, filename, mimetype, unique, filehash=None, download=False):
-        headers = [('Content-Type', mimetype), ('X-Content-Type-Options', 'nosniff'), ('Content-Security-Policy', "default-src 'none'")]
-        # cache
-        etag = bool(request) and request.httprequest.headers.get('If-None-Match')
-        status = status or 200
-        if filehash:
-            headers.append(('ETag', filehash))
-            if etag == filehash and status == 200:
-                status = 304
-        headers.append(('Cache-Control', 'max-age=%s' % (http.STATIC_CACHE_LONG if unique else 0)))
-        # content-disposition default name
-        if download:
-            headers.append(('Content-Disposition', content_disposition(filename)))
-
-        return (status, headers)
-
-    def binary_content(self, xmlid=None, model='ir.attachment', id=None, field='raw',
-                       unique=False, filename=None, filename_field='name', download=False,
-                       mimetype=None, default_mimetype='application/octet-stream',
-                       access_token=None):
-        """ Get file, attachment or downloadable content
-
-        If the ``xmlid`` and ``id`` parameter is omitted, fetches the default value for the
-        binary field (via ``default_get``), otherwise fetches the field for
-        that precise record.
-
-        :param str xmlid: xmlid of the record
-        :param str model: name of the model to fetch the binary from
-        :param int id: id of the record from which to fetch the binary
-        :param str field: binary field
-        :param bool unique: add a max-age for the cache control
-        :param str filename: choose a filename
-        :param str filename_field: if not create an filename with model-id-field
-        :param bool download: apply headers to download the file
-        :param str mimetype: mintype of the field (for headers)
-        :param str default_mimetype: default mintype if no mintype found
-        :param str access_token: optional token for unauthenticated access
-                                 only available  for ir.attachment
-        :returns: (status, headers, content)
-        """
-        record, status = self._get_record_and_check(xmlid=xmlid, model=model, id=id, field=field, access_token=access_token)
-
-        if not record:
-            return (status or 404, [], None)
-
-        content, headers, status = None, [], None
-
-        if record._name == 'ir.attachment':
-            status, content, default_filename, mimetype, filehash = self._binary_ir_attachment_redirect_content(record, default_mimetype=default_mimetype)
-            filename = filename or default_filename
-        if not content:
-            status, content, filename, mimetype, filehash = self._binary_record_content(
-                record, field=field, filename=filename, filename_field=filename_field,
-                default_mimetype='application/octet-stream')
-
-        status, headers = self._binary_set_headers(
-            status, filename, mimetype, unique, filehash=filehash, download=download)
-
-        return status, headers, content
-
-    def _response_by_status(self, status, headers, content):
-        if status == 304:
-            return werkzeug.wrappers.Response(status=status, headers=headers)
-        elif status == 301:
-            return request.redirect(content, code=301, local=False)
-        elif status != 200:
-            raise request.not_found()

odoo/addons/base/tests/__init__.py

@@ -1,4 +1,3 @@
-# -*- coding: utf-8 -*-
 # Part of Odoo. See LICENSE file for full copyright and licensing details.
 
 from . import common
@@ -18,7 +17,6 @@ from . import test_avatar_mixin
 from . import test_ir_actions
 from . import test_ir_attachment
 from . import test_ir_cron
-from . import test_ir_http
 from . import test_ir_filters
 from . import test_ir_mail_server
 from . import test_ir_model

odoo/addons/base/tests/test_ir_http.py

@@ -1,152 +0,0 @@
-# -*- coding: utf-8 -*-
-
-from odoo.tests import common
-import base64
-import odoo
-
-GIF = b"R0lGODdhAQABAIAAAP///////ywAAAAAAQABAAACAkQBADs="
-
-
-class test_ir_http_mimetype(common.TransactionCase):
-
-    def test_ir_http_mimetype_attachment(self):
-        """ Test mimetype for attachment """
-        attachment = self.env['ir.attachment'].create({
-            'datas': GIF,
-            'name': 'file.gif'})
-
-        status, headers, content = self.env['ir.http'].binary_content(
-            id=attachment.id,
-            mimetype=None,
-            default_mimetype='application/octet-stream',
-        )
-        mimetype = dict(headers).get('Content-Type')
-        self.assertEqual(mimetype, 'image/gif')
-
-    def test_ir_http_mimetype_attachment_name(self):
-        """ Test mimetype for attachment with bad name"""
-        attachment = self.env['ir.attachment'].create({
-            'datas': GIF,
-            'name': 'file.png'})
-
-        status, headers, content = self.env['ir.http'].binary_content(
-            id=attachment.id,
-            mimetype=None,
-            default_mimetype='application/octet-stream',
-        )
-        mimetype = dict(headers).get('Content-Type')
-        # TODO: fix and change it in master, should be image/gif
-        self.assertEqual(mimetype, 'image/png')
-
-    def test_ir_http_mimetype_basic_field(self):
-        """ Test mimetype for classic field """
-        partner = self.env['res.partner'].create({
-            'image_1920': GIF,
-            'name': 'Test mimetype basic field',
-        })
-
-        status, headers, content = self.env['ir.http'].binary_content(
-            model='res.partner',
-            id=partner.id,
-            field='image_1920',
-            default_mimetype='application/octet-stream',
-        )
-        mimetype = dict(headers).get('Content-Type')
-        self.assertEqual(mimetype, 'image/gif')
-
-    def test_ir_http_mimetype_computed_field(self):
-        """ Test mimetype for computed field wich resize picture"""
-        prop = self.env['ir.property'].create({
-            'fields_id': self.env['ir.model.fields'].search([], limit=1).id,
-            'name': "Property binary",
-            'value_binary': GIF,
-            'type': 'binary',
-        })
-
-        resized = odoo.tools.image_process(base64.b64decode(prop.value_binary), size=(64, 64))
-        # Simul computed field which resize and that is not attachement=True (E.G. on product)
-        prop.write({'value_binary': base64.b64encode(resized)})
-        status, headers, content = self.env['ir.http'].binary_content(
-            model='ir.property',
-            id=prop.id,
-            field='value_binary',
-            default_mimetype='application/octet-stream',
-        )
-        mimetype = dict(headers).get('Content-Type')
-        self.assertEqual(mimetype, 'image/gif')
-
-    def test_ir_http_attachment_access(self):
-        """ Test attachment access with and without access token """
-        public_user = self.env.ref('base.public_user')
-        attachment = self.env['ir.attachment'].create({
-            'datas': GIF,
-            'name': 'image.gif'
-        })
-
-        defaults = {
-            'id': attachment.id,
-            'default_mimetype': 'image/gif',
-        }
-
-        def test_access(**kwargs):
-            # DLE P69: `test_ir_http_attachment_access`
-            # `binary_content` relies on the `__last_update` to determine if a user has the read access to an attachment.
-            # as the attachment has just been created above as sudo, the data is in cache and if we don't remove it the below
-            # `test_access` wont have to fetch it and therefore wont raise the accesserror as its already in the cache
-            # `__last_update` must be removed from the cache when `test_access` is called, which happens and recompute the todos
-            attachment.env.flush_all()
-            attachment.env.invalidate_all()
-            status, _, _ = self.env['ir.http'].with_user(public_user).binary_content(
-                **dict(defaults, **kwargs)
-            )
-            return status
-
-        status = test_access()
-        self.assertEqual(status, 403, "no access")
-
-        status = test_access(access_token=u'Secret')
-        self.assertEqual(status, 403,
-            "no access if access token for attachment without access token")
-
-        attachment.access_token = u'Secret'
-        status = test_access(access_token=u'Secret')
-        self.assertEqual(status, 200, "access for correct access token")
-
-        status = test_access(access_token=u'Wrong')
-        self.assertEqual(status, 403, "no access for wrong access token")
-
-        attachment.public = True
-        status = test_access()
-        self.assertEqual(status, 200, "access for attachment with access")
-
-        status = test_access(access_token=u'Wrong')
-        self.assertEqual(status, 403,
-            "no access for wrong access token for attachment with access")
-
-        attachment.unlink()
-        status = test_access()
-        self.assertEqual(status, 404, "no access for deleted attachment")
-
-        status = test_access(access_token=u'Secret')
-        self.assertEqual(status, 404,
-            "no access with access token for deleted attachment")
-
-    def test_ir_http_default_filename_extension(self):
-        """ Test attachment extension when the record has a dot in its name """
-        self.env.user.name = "Mr. John"
-        self.env.user.image_128 = GIF
-        _, _, filename, _, _ = self.env['ir.http']._binary_record_content(
-            self.env.user, 'image_128',
-        )
-        self.assertEqual(filename, "Mr. John.gif")
-
-        # For attachment, the name is considered to have the extension in the name
-        # and thus the extension should not be added again.
-        attachment = self.env['ir.attachment'].create({
-            'datas': GIF,
-            'name': 'image.gif'
-        })
-        _, _, filename, _, _ = self.env['ir.http']._binary_record_content(
-            attachment,
-        )
-        self.assertEqual(filename, 'image.gif')

odoo/addons/base/tests/test_mimetypes.py

@@ -79,7 +79,7 @@ class test_guess_mimetype(BaseCase):
         self.assertEqual(get_extension('filename.Abc'), '.abc')
         self.assertEqual(get_extension('filename.scss'), '.scss')
         self.assertEqual(get_extension('filename.torrent'), '.torrent')
-        self.assertEqual(get_extension('.htaccess'), '.htaccess')
+        self.assertEqual(get_extension('.htaccess'), '')
         # enough to suppose that extension is present and don't suffix the filename
         self.assertEqual(get_extension('filename.tar.gz'), '.gz')
         self.assertEqual(get_extension('filename'), '')

odoo/addons/test_http/controllers.py

@@ -5,7 +5,7 @@ import werkzeug
 from odoo import http
 from odoo.exceptions import AccessError, UserError
 from odoo.http import request
-from odoo.tools import reraise_x_as
+from odoo.tools import replace_exceptions
 
 from odoo.addons.web.controllers.utils import ensure_db
 
@@ -151,16 +151,16 @@ class TestHttp(http.Controller):
         raise ValueError('Unknown destination')
 
     @http.route('/test_http/hide_errors/decorator', type='http', auth='none')
-    @reraise_x_as(AccessError, as_=werkzeug.exceptions.NotFound())
-    def hide_errors_deco(self, error):
+    @replace_exceptions(AccessError, by=werkzeug.exceptions.NotFound())
+    def hide_errors_decorator(self, error):
         if error == 'AccessError':
             raise AccessError("Wrong iris code")
         if error == 'UserError':
             raise UserError("Walter is AFK")
 
     @http.route('/test_http/hide_errors/context-manager', type='http', auth='none')
-    def hide_errors_cm(self, error):
-        with reraise_x_as(AccessError, as_=werkzeug.exceptions.NotFound()):
+    def hide_errors_context_manager(self, error):
+        with replace_exceptions(AccessError, by=werkzeug.exceptions.NotFound()):
             if error == 'AccessError':
                 raise AccessError("Wrong iris code")
             if error == 'UserError':

odoo/addons/test_http/data.xml

@@ -1,5 +1,28 @@
+<?xml version="1.0" encoding="utf-8"?>
 <odoo>
     <data>
+        <record id="gizeh_png" model="ir.attachment">
+            <field name="name">gizeh.png</field>
+            <field name="type">binary</field>
+            <field name="datas" type="base64" file="test_http/static/src/img/gizeh.png"/>
+            <field name="url">/test_http/gizeh.png</field>
+            <field name="public">True</field>
+        </record>
+
+        <record id="gizeh_url" model="ir.attachment">
+            <field name="name">gizeh.png</field>
+            <field name="type">url</field>
+            <field name="url">/test_http/static/src/img/gizeh.png</field>
+            <field name="public">True</field>
+        </record>
+
+        <record id="rickroll" model="ir.attachment">
+            <field name="name">rickroll</field>
+            <field name="type">url</field>
+            <field name="url">https://www.youtube.com/watch?v=dQw4w9WgXcQ</field>
+            <field name="public">True</field>
+        </record>
+
         <record id="milky_way" model="test_http.galaxy">
             <field name="name">Milky Way</field>
         </record>
@@ -8,11 +31,12 @@
             <field name="name">Pegasus</field>
         </record>
 
-
         <record id="earth" model="test_http.stargate">
             <field name="name">Earth</field>
             <field name="address">sq5Abt</field>
             <field name="galaxy_id" ref="test_http.milky_way"/>
+            <field name="glyph_attach" type="base64" file="test_http/static/src/img/gizeh.png"/>
+            <field name="glyph_inline" type="base64" file="test_http/static/src/img/gizeh.png"/>
         </record>
 
         <record id="abydos" model="test_http.stargate">

odoo/addons/test_http/models.py

@@ -16,6 +16,8 @@ class Stargate(models.Model):
     sgc_designation = fields.Char(store=True, compute='_compute_sgc_designation', help="The SGC designation name of this stargate.")
     galaxy_id = fields.Many2one('test_http.galaxy', required=True, help="The galaxy where this stargate is.")
     has_galaxy_crystal = fields.Boolean(store=True, compute='_compute_has_galaxy_crystal', readonly=False, help="Whether this stargate can dial other galaxies.")
+    glyph_attach = fields.Image(attachment=True)
+    glyph_inline = fields.Image(attachment=False)
 
     _sql_constraints = [
         ('address_length', 'CHECK(LENGTH(address) = 6)', "Local addresses have 6 glyphs"),

odoo/addons/test_http/static/src/img/gizeh.svg

@@ -1,22 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
-<!-- Creator: CorelDRAW -->
-<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" width="63.509mm" height="63.509mm" style="shape-rendering:geometricPrecision; text-rendering:geometricPrecision; image-rendering:optimizeQuality; fill-rule:evenodd; clip-rule:evenodd"
-viewBox="0 0 63.509 63.509">
- <defs>
-  <style type="text/css">
-   <![CDATA[
-    .fil1 {fill:none}
-    .fil0 {fill:#1F1A17}
-   ]]>
-  </style>
- </defs>
- <g id="Layer_x0020_1">
-  <metadata id="CorelCorpID_0Corel-Layer"/>
-  <g id="_150956456">
-   <path id="_151090152" class="fil0" d="M31.6671 7.4461c4.8933,0 8.861,3.9677 8.861,8.8598 0,4.8932 -3.9677,8.8609 -8.861,8.8609 -4.892,0 -8.8597,-3.9677 -8.8597,-8.8609 0,-4.8921 3.9677,-8.8598 8.8597,-8.8598zm0.0006 4.3597c2.4855,0 4.5007,2.0151 4.5007,4.5007 0,2.4855 -2.0152,4.5006 -4.5007,4.5006 -2.4855,0 -4.5006,-2.0151 -4.5006,-4.5006 0,-2.4856 2.0151,-4.5007 4.5006,-4.5007z"/>
-   <path id="_150956480" class="fil0" d="M44.2053 53.9278c-2.4381,4.0631 -2.4381,4.0631 -2.4381,4.0631l0 -0.0656 12.9247 0 -23.0242 -31.2605c-23.0047,31.2605 -23.0047,31.2605 -23.0047,31.2605l12.9071 0 0 0.0656c0,0 0,0 -2.4381,-4.0631l-0.0049 0.0067c3.1437,-4.2692 7.2299,-9.8181 12.5406,-17.0292l0 0c5.3108,7.2111 9.3982,12.7594 12.5425,17.0286l-0.0049 -0.0061z"/>
-  </g>
-  <rect class="fil1" x="-0.0864984" y="0.170202" width="63.509" height="63.509"/>
- </g>
-</svg>

odoo/addons/test_http/tests/__init__.py

@@ -1,2 +1,8 @@
+from . import test_common
+from . import test_echo_reply
 from . import test_error
-from . import test_http
+from . import test_greeting
+from . import test_misc
+from . import test_models
+from . import test_static
+from . import test_web_server

odoo/addons/test_http/tests/test_common.py

@@ -0,0 +1,45 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+from unittest.mock import patch
+
+import odoo
+from odoo.http import Session
+from odoo.tests.common import HttpCase
+from odoo.tools.func import lazy_property
+from odoo.addons.test_http.utils import MemoryGeoipResolver, MemorySessionStore
+
+
+class TestHttpBase(HttpCase):
+    @classmethod
+    def setUpClass(cls):
+        super().setUpClass()
+        cls.addClassCleanup(lazy_property.reset_all, odoo.http.root)
+        cls.classPatch(odoo.conf, 'server_wide_modules', ['base', 'web', 'test_http'])
+        lazy_property.reset_all(odoo.http.root)
+        cls.classPatch(odoo.http.root, 'session_store', MemorySessionStore(session_class=Session))
+        cls.classPatch(odoo.http.root, 'geoip_resolver', MemoryGeoipResolver())
+
+    def setUp(self):
+        super().setUp()
+        odoo.http.root.session_store.store.clear()
+
+    def db_url_open(self, url, *args, allow_redirects=False, **kwargs):
+        return self.url_open(url, *args, allow_redirects=allow_redirects, **kwargs)
+
+    def nodb_url_open(self, url, *args, allow_redirects=False, **kwargs):
+        with patch('odoo.http.db_list') as db_list, \
+             patch('odoo.http.db_filter') as db_filter:
+            db_list.return_value = []
+            db_filter.return_value = []
+            return self.url_open(url, *args, allow_redirects=allow_redirects, **kwargs)
+
+    def multidb_url_open(self, url, *args, allow_redirects=False, dblist=(), **kwargs):
+        dblist = dblist or self.db_list
+        assert len(dblist) >= 2, "There should be at least 2 databases"
+        with patch('odoo.http.db_list') as db_list, \
+             patch('odoo.http.db_filter') as db_filter, \
+             patch('odoo.http.Registry') as Registry:
+            db_list.return_value = dblist
+            db_filter.side_effect = lambda dbs, host=None: [db for db in dbs if db in dblist]
+            Registry.return_value = self.registry
+            return self.url_open(url, *args, allow_redirects=allow_redirects, **kwargs)

odoo/addons/test_http/tests/test_echo_reply.py

@@ -0,0 +1,173 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+import json
+from urllib.parse import urlparse
+
+from odoo.http import Request
+from odoo.tests import tagged
+from odoo.tests.common import new_test_user
+from odoo.tools import mute_logger
+from odoo.addons.test_http.controllers import CT_JSON
+
+from .test_common import TestHttpBase
+
+
+@tagged('post_install', '-at_install')
+class TestHttpEchoReplyHttpNoDB(TestHttpBase):
+    def test_echohttp0_get_qs_nodb(self):
+        res = self.nodb_url_open('/test_http/echo-http-get?race=Asgard')
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, "{'race': 'Asgard'}")
+
+    def test_echohttp1_get_form_nodb(self):
+        res = self.nodb_url_open('/test_http/echo-http-get', data={'commander': 'Thor'})
+        self.assertEqual(res.status_code, 405)
+
+    def test_echohttp2_post_qs_nodb(self):
+        res = self.nodb_url_open('/test_http/echo-http-post?race=Asgard')
+        self.assertEqual(res.status_code, 405)
+
+    def test_echohttp3_post_qs_form_nodb(self):
+        res = self.nodb_url_open('/test_http/echo-http-post?race=Asgard', data={'commander': 'Thor'})
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, "{'race': 'Asgard', 'commander': 'Thor'}")
+
+    @mute_logger('odoo.http')
+    def test_echohttp4_post_json_nodb(self):
+        payload = json.dumps({'commander': 'Thor'})
+        res = self.nodb_url_open('/test_http/echo-http-post', data=payload, headers=CT_JSON)
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, '{}')
+
+
+    def test_echohttp5_post_csrf(self):
+        res = self.nodb_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor'})
+        self.assertEqual(res.status_code, 303)
+        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/web/database/selector')
+
+    def test_echohttp6_json_over_http(self):
+        payload = json.dumps({'commander': 'Thor'})
+        res = self.nodb_url_open('/test_http/echo-json-over-http', data=payload, headers=CT_JSON)
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, payload)
+        mimetype = res.headers['Content-Type'].partition(';')[0]
+        self.assertEqual(mimetype, 'application/json')
+
+
+@tagged('post_install', '-at_install')
+class TestHttpEchoReplyJsonNoDB(TestHttpBase):
+    def test_echojson0_qs_json_nodb(self):
+        payload = json.dumps({
+            'jsonrpc': '2.0',
+            'id': 1234,
+            'params': {
+                'commander': 'Thor',
+            },
+        })
+        res = self.nodb_url_open("/test_http/echo-json?race=Asgard", data=payload, headers=CT_JSON)
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, '{"jsonrpc": "2.0", "id": 1234, "result": {"commander": "Thor"}}')
+
+    def test_echojson1_http_get_nodb(self):
+        res = self.nodb_url_open('/test_http/echo-json')  # GET
+        self.assertEqual(res.status_code, 405)
+
+    @mute_logger('odoo.http')
+    def test_echojson2_http_post_nodb(self):
+        res = self.nodb_url_open('/test_http/echo-json', data={'race': 'Asgard'})  # POST
+        self.assertIn("Bad Request", res.text)
+
+
+@tagged('post_install', '-at_install')
+class TestHttpEchoReplyHttpWithDB(TestHttpBase):
+    def setUp(self):
+        super().setUp()
+        self.jackoneill = new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
+        self.authenticate('jackoneill', 'jackoneill')
+
+    def test_echohttp0_get_qs_db(self):
+        res = self.db_url_open('/test_http/echo-http-get?race=Asgard')
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, "{'race': 'Asgard'}")
+
+    def test_echohttp1_get_form_db(self):
+        res = self.db_url_open('/test_http/echo-http-get', data={'commander': 'Thor'})
+        self.assertEqual(res.status_code, 405)
+
+    def test_echohttp2_post_qs_db(self):
+        res = self.db_url_open('/test_http/echo-http-post?race=Asgard')
+        self.assertEqual(res.status_code, 405)
+
+    def test_echohttp3_post_qs_form_db(self):
+        res = self.db_url_open('/test_http/echo-http-post?race=Asgard', data={'commander': 'Thor'})
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, "{'race': 'Asgard', 'commander': 'Thor'}")
+
+    @mute_logger('odoo.http')
+    def test_echohttp4_post_json_db(self):
+        payload = json.dumps({'commander': 'Thor'})
+        res = self.db_url_open('/test_http/echo-http-post', data=payload, headers=CT_JSON)
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, '{}')
+
+    @mute_logger('odoo.http')
+    def test_echohttp5_post_no_csrf(self):
+        res = self.db_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor'})
+        self.assertEqual(res.status_code, 400)
+        self.assertIn("Session expired (invalid CSRF token)", res.text)
+
+    @mute_logger('odoo.http')
+    def test_echohttp6_post_bad_csrf(self):
+        res = self.db_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor', 'csrf_token': 'bad token'})
+        self.assertEqual(res.status_code, 400)
+        self.assertIn("Session expired (invalid CSRF token)", res.text)
+
+    @mute_logger('odoo.http')
+    def test_echohttp7_post_good_csrf(self):
+        res = self.db_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor', 'csrf_token': Request.csrf_token(self)})
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, "{'race': 'Asgard', 'commander': 'Thor'}")
+
+
+@tagged('post_install', '-at_install')
+class TestHttpEchoReplyJsonWithDB(TestHttpBase):
+    def setUp(self):
+        super().setUp()
+        self.jackoneill = new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
+        self.authenticate('jackoneill', 'jackoneill')
+
+    def test_echojson0_qs_json_db(self):
+        payload = json.dumps({
+            'jsonrpc': '2.0',
+            'id': 1234,
+            'params': {
+                'commander': 'Thor',
+            },
+        })
+        res = self.db_url_open('/test_http/echo-json?race=Asgard', data=payload, headers=CT_JSON)
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, '{"jsonrpc": "2.0", "id": 1234, "result": {"commander": "Thor"}}')
+
+    def test_echojson1_http_get_db(self):
+        res = self.db_url_open('/test_http/echo-json')  # GET
+        self.assertEqual(res.status_code, 405)
+
+    @mute_logger('odoo.http')
+    def test_echojson2_http_post_db(self):
+        res = self.db_url_open('/test_http/echo-json', data={'race': 'Asgard'})  # POST
+        self.assertIn("Bad Request", res.text)
+
+    def test_echojson3_context_db(self):
+        payload = json.dumps({
+            "jsonrpc": "2.0",
+            "id": 0,
+            "params": {
+                "context": {
+                    "name": "Thor"
+                },
+                "race": "Asgard",
+            },
+        })
+        res = self.db_url_open("/test_http/echo-json-context", data=payload, headers=CT_JSON)
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, '{"jsonrpc": "2.0", "id": 0, "result": {"name": "Thor"}}')

odoo/addons/test_http/tests/test_error.py

@@ -1,5 +1,8 @@
-from odoo.tools import mute_logger
-from .test_http import TestHttpBase
+import json
+from unittest.mock import patch
+from odoo.tools import config, mute_logger
+from odoo.addons.test_http.controllers import CT_JSON
+from .test_common import TestHttpBase
 
 
 class TestHttpErrorHttp(TestHttpBase):
@@ -24,3 +27,62 @@ class TestHttpErrorHttp(TestHttpBase):
             res = self.nodb_url_open('/test_http/hide_errors/context-manager?error=UserError')
             self.assertEqual(res.status_code, 400, "UserError are not configured to be hidden, they should be kept as-is.")
             self.assertIn("Walter is AFK", res.text, "The real UserError message should be kept")
+
+
+class TestHttpJsonError(TestHttpBase):
+
+    jsonrpc_error_structure = {
+        'error': {
+            'code': ...,
+            'data': {
+                'arguments': ...,
+                'context': ...,
+                'debug': ...,
+                'message': ...,
+                'name': ...,
+            },
+            'message': ...,
+        },
+        'id': ...,
+        'jsonrpc': ...,
+    }
+
+    def assertIsErrorPayload(self, payload):
+        self.assertEqual(
+            set(payload),
+            set(self.jsonrpc_error_structure),
+        )
+        self.assertEqual(
+            set(payload['error']),
+            set(self.jsonrpc_error_structure['error']),
+        )
+        self.assertEqual(
+            set(payload['error']['data']),
+            set(self.jsonrpc_error_structure['error']['data']),
+        )
+
+
+    @mute_logger('odoo.http')
+    def test_errorjson0_value_error(self):
+        res = self.db_url_open('/test_http/json_value_error',
+            data=json.dumps({'jsonrpc': '2.0', 'id': 1234, 'params': {}}),
+            headers=CT_JSON
+        )
+        res.raise_for_status()
+
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.headers.get('Content-Type', ''), 'application/json; charset=utf-8')
+
+        payload = res.json()
+        self.assertIsErrorPayload(payload)
+
+        error_data = payload['error']['data']
+        self.assertEqual(error_data['name'], 'builtins.ValueError')
+        self.assertEqual(error_data['message'], 'Unknown destination')
+        self.assertEqual(error_data['arguments'], ['Unknown destination'])
+        self.assertEqual(error_data['context'], {})
+
+    @mute_logger('odoo.http')
+    def test_errorjson1_dev_mode_werkzeug(self):
+        with patch.object(config, 'options', {**config.options, 'dev_mode': 'werkzeug'}):
+            self.test_errorjson0_value_error()

odoo/addons/test_http/tests/test_greeting.py

@@ -0,0 +1,63 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+
+from odoo.tests import tagged
+from odoo.tests.common import new_test_user
+
+from .test_common import TestHttpBase
+
+
+@tagged('post_install', '-at_install')
+class TestHttpGreeting(TestHttpBase):
+    def test_greeting0_matrix(self):
+        new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
+        test_matrix = [
+            # path, database, login, expected_code, expected_re_pattern
+            ('/test_http/greeting', False, None, 200, r"Tek'ma'te"),
+            ('/test_http/greeting', True, None, 200, r"Tek'ma'te"),
+            ('/test_http/greeting', True, 'public', 200, r"Tek'ma'te"),
+            ('/test_http/greeting', True, 'jackoneill', 200, r"Tek'ma'te"),
+            ('/test_http/greeting-none', False, None, 200, r"Tek'ma'te"),
+            ('/test_http/greeting-none', True, None, 200, r"Tek'ma'te"),
+            ('/test_http/greeting-none', True, 'public', 200, r"Tek'ma'te"),
+            ('/test_http/greeting-none', True, 'jackoneill', 200, r"Tek'ma'te"),
+            ('/test_http/greeting-public', False, None, 404, r"Not Found"),
+            ('/test_http/greeting-public', True, None, 200, r"Tek'ma'te"),
+            ('/test_http/greeting-public', True, 'public', 200, r"Tek'ma'te"),
+            ('/test_http/greeting-public', True, 'jackoneill', 200, r"Tek'ma'te"),
+            ('/test_http/greeting-user', False, None, 404, r"Not Found"),
+            ('/test_http/greeting-user', True, None, 303, r".*/web/login.*"),
+            ('/test_http/greeting-user', True, 'public', 303, r".*/web/login.*"),
+            ('/test_http/greeting-user', True, 'jackoneill', 200, r"Tek'ma'te"),
+        ]
+
+        for path, withdb, login, expected_code, expected_pattern in test_matrix:
+            with self.subTest(path=path, withdb=withdb, login=login):
+                if withdb:
+                    if login == 'public':
+                        self.authenticate(None, None)
+                    elif login:
+                        self.authenticate(login, login)
+                    res = self.db_url_open(path, allow_redirects=False)
+                else:
+                    res = self.nodb_url_open(path, allow_redirects=False)
+
+                self.assertEqual(res.status_code, expected_code)
+                self.assertRegex(res.text, expected_pattern)
+
+                if withdb and login:
+                    self.logout(keep_db=False)
+
+    def test_greeting1_headers_nodb(self):
+        res = self.nodb_url_open('/test_http/greeting')
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.headers.get('Content-Type'), 'text/html; charset=utf-8')
+        self.assertEqual(res.text, "Tek'ma'te")
+
+    def test_greeting2_headers_db(self):
+        new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
+        self.authenticate('jackoneill', 'jackoneill')
+        res = self.db_url_open('/test_http/greeting')
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.headers.get('Content-Type'), 'text/html; charset=utf-8')
+        self.assertEqual(res.text, "Tek'ma'te")

odoo/addons/test_http/tests/test_http.py

@@ -1,634 +0,0 @@
-# Part of Odoo. See LICENSE file for full copyright and licensing details.
-
-import html
-import json
-from unittest.mock import patch
-from urllib.parse import urlparse
-from socket import gethostbyname
-
-import odoo
-from odoo.http import Request, Session
-from odoo.tests import tagged
-from odoo.tests.common import HOST, HttpCase, new_test_user
-from odoo.tools import config, file_open, mute_logger
-from odoo.tools.func import lazy_property
-from odoo.addons.test_http.controllers import CT_JSON
-from odoo.addons.test_http.utils import (
-    MemoryGeoipResolver, MemorySessionStore, HtmlTokenizer
-)
-
-GEOIP_ODOO_FARM_2 = {
-    'city': 'Ramillies',
-    'country_code': 'BE',
-    'country_name': 'Belgium',
-    'latitude': 50.6314,
-    'longitude': 4.8573,
-    'region': 'WAL',
-    'time_zone': 'Europe/Brussels'
-}
-
-
-class TestHttpBase(HttpCase):
-    @classmethod
-    def setUpClass(cls):
-        super().setUpClass()
-        cls.addClassCleanup(lazy_property.reset_all, odoo.http.root)
-        cls.classPatch(odoo.conf, 'server_wide_modules', ['base', 'web', 'test_http'])
-        lazy_property.reset_all(odoo.http.root)
-        cls.classPatch(odoo.http.root, 'session_store', MemorySessionStore(session_class=Session))
-        cls.classPatch(odoo.http.root, 'geoip_resolver', MemoryGeoipResolver())
-
-    def setUp(self):
-        super().setUp()
-        odoo.http.root.session_store.store.clear()
-
-    def db_url_open(self, url, *args, allow_redirects=False, **kwargs):
-        return self.url_open(url, *args, allow_redirects=allow_redirects, **kwargs)
-
-    def nodb_url_open(self, url, *args, allow_redirects=False, **kwargs):
-        with patch('odoo.http.db_list') as db_list, \
-             patch('odoo.http.db_filter') as db_filter:
-            db_list.return_value = []
-            db_filter.return_value = []
-            return self.url_open(url, *args, allow_redirects=allow_redirects, **kwargs)
-
-    def multidb_url_open(self, url, *args, allow_redirects=False, dblist=(), **kwargs):
-        dblist = dblist or self.db_list
-        assert len(dblist) >= 2, "There should be at least 2 databases"
-        with patch('odoo.http.db_list') as db_list, \
-             patch('odoo.http.db_filter') as db_filter, \
-             patch('odoo.http.Registry') as Registry:
-            db_list.return_value = dblist
-            db_filter.side_effect = lambda dbs, host=None: [db for db in dbs if db in dblist]
-            Registry.return_value = self.registry
-            return self.url_open(url, *args, allow_redirects=allow_redirects, **kwargs)
-
-
-@tagged('post_install', '-at_install')
-class TestHttpGreeting(TestHttpBase):
-    def test_greeting0_matrix(self):
-        new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
-        test_matrix = [
-            # path, database, login, expected_code, expected_re_pattern
-            ('/test_http/greeting', False, None, 200, r"Tek'ma'te"),
-            ('/test_http/greeting', True, None, 200, r"Tek'ma'te"),
-            ('/test_http/greeting', True, 'public', 200, r"Tek'ma'te"),
-            ('/test_http/greeting', True, 'jackoneill', 200, r"Tek'ma'te"),
-            ('/test_http/greeting-none', False, None, 200, r"Tek'ma'te"),
-            ('/test_http/greeting-none', True, None, 200, r"Tek'ma'te"),
-            ('/test_http/greeting-none', True, 'public', 200, r"Tek'ma'te"),
-            ('/test_http/greeting-none', True, 'jackoneill', 200, r"Tek'ma'te"),
-            ('/test_http/greeting-public', False, None, 404, r"Not Found"),
-            ('/test_http/greeting-public', True, None, 200, r"Tek'ma'te"),
-            ('/test_http/greeting-public', True, 'public', 200, r"Tek'ma'te"),
-            ('/test_http/greeting-public', True, 'jackoneill', 200, r"Tek'ma'te"),
-            ('/test_http/greeting-user', False, None, 404, r"Not Found"),
-            ('/test_http/greeting-user', True, None, 303, r".*/web/login.*"),
-            ('/test_http/greeting-user', True, 'public', 303, r".*/web/login.*"),
-            ('/test_http/greeting-user', True, 'jackoneill', 200, r"Tek'ma'te"),
-        ]
-
-        for path, withdb, login, expected_code, expected_pattern in test_matrix:
-            with self.subTest(path=path, withdb=withdb, login=login):
-                if withdb:
-                    if login == 'public':
-                        self.authenticate(None, None)
-                    elif login:
-                        self.authenticate(login, login)
-                    res = self.db_url_open(path, allow_redirects=False)
-                else:
-                    res = self.nodb_url_open(path, allow_redirects=False)
-
-                self.assertEqual(res.status_code, expected_code)
-                self.assertRegex(res.text, expected_pattern)
-
-                if withdb and login:
-                    self.logout(keep_db=False)
-
-    def test_greeting1_headers_nodb(self):
-        res = self.nodb_url_open('/test_http/greeting')
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.headers.get('Content-Type'), 'text/html; charset=utf-8')
-        self.assertEqual(res.text, "Tek'ma'te")
-
-    def test_greeting2_headers_db(self):
-        new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
-        self.authenticate('jackoneill', 'jackoneill')
-        res = self.db_url_open('/test_http/greeting')
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.headers.get('Content-Type'), 'text/html; charset=utf-8')
-        self.assertEqual(res.text, "Tek'ma'te")
-
-
-@tagged('post_install', '-at_install')
-class TestHttpStatic(TestHttpBase):
-    def test_static0_png_image(self):
-        res = self.nodb_url_open("/test_http/static/src/img/gizeh.png")
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.headers.get('Content-Length'), '814')
-        self.assertEqual(res.headers.get('Content-Type'), 'image/png')
-        cache_control = set(res.headers.get('Cache-Control', '').split(', '))
-        self.assertEqual(cache_control, {'public', 'max-age=604800'})  # one week
-        with file_open('test_http/static/src/img/gizeh.png', 'rb') as file:
-            self.assertEqual(res.content, file.read())
-
-    def test_static1_svg_image(self):
-        res = self.nodb_url_open("/test_http/static/src/img/gizeh.svg")
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.headers.get('Content-Length'), '1529')
-        self.assertEqual(res.headers.get('Content-Type'), 'image/svg+xml; charset=utf-8')
-        cache_control = set(res.headers.get('Cache-Control', '').split(', '))
-        self.assertEqual(cache_control, {'public', 'max-age=604800'})  # one week
-        with file_open('test_http/static/src/img/gizeh.svg', 'rb') as file:
-            self.assertEqual(res.content, file.read())
-
-    def test_static2_not_found(self):
-        res = self.nodb_url_open("/test_http/static/i-dont-exist")
-        self.assertEqual(res.status_code, 404)
-
-    def test_static3_attachment(self):
-        with file_open('test_http/static/src/img/gizeh.svg', 'rb') as file:
-            content = file.read()
-
-        attachment = self.env['ir.attachment'].create({
-            'name': 'point_of_origin.svg',
-            'type': 'binary',
-            'raw': content,
-            'res_model': 'test_http.stargate',
-            'res_id': self.ref('test_http.earth'),
-        })
-        attachment['url'] = f'/test_http/{attachment["checksum"]}'
-
-        res = self.db_url_open(attachment['url'])
-        self.assertEqual(res.headers.get('Content-Length'), '1529')
-        self.assertEqual(res.headers.get('Content-Type'), 'image/svg+xml; charset=utf-8')
-        self.assertEqual(res.headers.get('Content-Security-Policy'), "default-src 'none'")
-        self.assertEqual(res.content, content)
-
-
-@tagged('post_install', '-at_install')
-class TestHttpEchoReplyHttpNoDB(TestHttpBase):
-    def test_echohttp0_get_qs_nodb(self):
-        res = self.nodb_url_open('/test_http/echo-http-get?race=Asgard')
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, "{'race': 'Asgard'}")
-
-    def test_echohttp1_get_form_nodb(self):
-        res = self.nodb_url_open('/test_http/echo-http-get', data={'commander': 'Thor'})
-        self.assertEqual(res.status_code, 405)
-
-    def test_echohttp2_post_qs_nodb(self):
-        res = self.nodb_url_open('/test_http/echo-http-post?race=Asgard')
-        self.assertEqual(res.status_code, 405)
-
-    def test_echohttp3_post_qs_form_nodb(self):
-        res = self.nodb_url_open('/test_http/echo-http-post?race=Asgard', data={'commander': 'Thor'})
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, "{'race': 'Asgard', 'commander': 'Thor'}")
-
-    @mute_logger('odoo.http')
-    def test_echohttp4_post_json_nodb(self):
-        payload = json.dumps({'commander': 'Thor'})
-        res = self.nodb_url_open('/test_http/echo-http-post', data=payload, headers=CT_JSON)
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, '{}')
-
-    def test_echohttp5_post_csrf(self):
-        res = self.nodb_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor'})
-        self.assertEqual(res.status_code, 303)
-        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/web/database/selector')
-
-    def test_echohttp6_json_over_http(self):
-        payload = json.dumps({'commander': 'Thor'})
-        res = self.nodb_url_open('/test_http/echo-json-over-http', data=payload, headers=CT_JSON)
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, payload)
-        mimetype = res.headers['Content-Type'].partition(';')[0]
-        self.assertEqual(mimetype, 'application/json')
-
-
-@tagged('post_install', '-at_install')
-class TestHttpEchoReplyJsonNoDB(TestHttpBase):
-    def test_echojson0_qs_json_nodb(self):
-        payload = json.dumps({
-            'jsonrpc': '2.0',
-            'id': 1234,
-            'params': {
-                'commander': 'Thor',
-            },
-        })
-        res = self.nodb_url_open("/test_http/echo-json?race=Asgard", data=payload, headers=CT_JSON)
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, '{"jsonrpc": "2.0", "id": 1234, "result": {"commander": "Thor"}}')
-
-    def test_echojson1_http_get_nodb(self):
-        res = self.nodb_url_open('/test_http/echo-json')  # GET
-        self.assertEqual(res.status_code, 405)
-
-    @mute_logger('odoo.http')
-    def test_echojson2_http_post_nodb(self):
-        res = self.nodb_url_open('/test_http/echo-json', data={'race': 'Asgard'})  # POST
-        self.assertIn("Bad Request", res.text)
-
-
-@tagged('post_install', '-at_install')
-class TestHttpEchoReplyHttpWithDB(TestHttpBase):
-    def setUp(self):
-        super().setUp()
-        self.jackoneill = new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
-        self.authenticate('jackoneill', 'jackoneill')
-
-    def test_echohttp0_get_qs_db(self):
-        res = self.db_url_open('/test_http/echo-http-get?race=Asgard')
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, "{'race': 'Asgard'}")
-
-    def test_echohttp1_get_form_db(self):
-        res = self.db_url_open('/test_http/echo-http-get', data={'commander': 'Thor'})
-        self.assertEqual(res.status_code, 405)
-
-    def test_echohttp2_post_qs_db(self):
-        res = self.db_url_open('/test_http/echo-http-post?race=Asgard')
-        self.assertEqual(res.status_code, 405)
-
-    def test_echohttp3_post_qs_form_db(self):
-        res = self.db_url_open('/test_http/echo-http-post?race=Asgard', data={'commander': 'Thor'})
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, "{'race': 'Asgard', 'commander': 'Thor'}")
-
-    @mute_logger('odoo.http')
-    def test_echohttp4_post_json_db(self):
-        payload = json.dumps({'commander': 'Thor'})
-        res = self.db_url_open('/test_http/echo-http-post', data=payload, headers=CT_JSON)
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, '{}')
-
-    @mute_logger('odoo.http')
-    def test_echohttp5_post_no_csrf(self):
-        res = self.db_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor'})
-        self.assertEqual(res.status_code, 400)
-        self.assertIn("Session expired (invalid CSRF token)", res.text)
-
-    @mute_logger('odoo.http')
-    def test_echohttp6_post_bad_csrf(self):
-        res = self.db_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor', 'csrf_token': 'bad token'})
-        self.assertEqual(res.status_code, 400)
-        self.assertIn("Session expired (invalid CSRF token)", res.text)
-
-    @mute_logger('odoo.http')
-    def test_echohttp7_post_good_csrf(self):
-        res = self.db_url_open('/test_http/echo-http-csrf?race=Asgard', data={'commander': 'Thor', 'csrf_token': Request.csrf_token(self)})
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, "{'race': 'Asgard', 'commander': 'Thor'}")
-
-
-
-@tagged('post_install', '-at_install')
-class TestHttpEchoReplyJsonWithDB(TestHttpBase):
-    def setUp(self):
-        super().setUp()
-        self.jackoneill = new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
-        self.authenticate('jackoneill', 'jackoneill')
-
-    def test_echojson0_qs_json_db(self):
-        payload = json.dumps({
-            'jsonrpc': '2.0',
-            'id': 1234,
-            'params': {
-                'commander': 'Thor',
-            },
-        })
-        res = self.db_url_open('/test_http/echo-json?race=Asgard', data=payload, headers=CT_JSON)
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, '{"jsonrpc": "2.0", "id": 1234, "result": {"commander": "Thor"}}')
-
-    def test_echojson1_http_get_db(self):
-        res = self.db_url_open('/test_http/echo-json')  # GET
-        self.assertEqual(res.status_code, 405)
-
-    @mute_logger('odoo.http')
-    def test_echojson2_http_post_db(self):
-        res = self.db_url_open('/test_http/echo-json', data={'race': 'Asgard'})  # POST
-        self.assertIn("Bad Request", res.text)
-
-    def test_echojson3_context_db(self):
-        payload = json.dumps({
-            "jsonrpc": "2.0",
-            "id": 0,
-            "params": {
-                "context": {
-                    "name": "Thor"
-                },
-                "race": "Asgard",
-            },
-        })
-        res = self.db_url_open("/test_http/echo-json-context", data=payload, headers=CT_JSON)
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, '{"jsonrpc": "2.0", "id": 0, "result": {"name": "Thor"}}')
-
-
-@tagged('post_install', '-at_install')
-class TestHttpModels(TestHttpBase):
-    def setUp(self):
-        super().setUp()
-        self.jackoneill = new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
-        self.authenticate('jackoneill', 'jackoneill')
-
-    def test_models0_galaxy_ok(self):
-        milky_way = self.env.ref('test_http.milky_way')
-
-        res = self.url_open(f"/test_http/{milky_way.id}")
-
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(
-            HtmlTokenizer.tokenize(res.text),
-            HtmlTokenizer.tokenize('''\
-                <p>Milky Way</p>
-                <ul>
-                    <li><a href="/test_http/1/1">Earth (P4X-126)</a></li>
-                    <li><a href="/test_http/1/2">Abydos (P2X-125)</a></li>
-                    <li><a href="/test_http/1/3">Dakara (P5C-113)</a></li>
-                </ul>
-                ''')
-            )
-
-    @mute_logger('odoo.http')
-    def test_models1_galaxy_ko(self):
-        res = self.url_open("/test_http/404")  # unknown galaxy
-        self.assertEqual(res.status_code, 400)
-        self.assertIn('The Ancients did not settle there.', res.text)
-
-    def test_models2_stargate_ok(self):
-        milky_way = self.env.ref('test_http.milky_way')
-        earth = self.env.ref('test_http.earth')
-
-        res = self.url_open(f'/test_http/{milky_way.id}/{earth.id}')
-
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(
-            HtmlTokenizer.tokenize(res.text),
-            HtmlTokenizer.tokenize('''\
-                <dl>
-                    <dt>name</dt><dd>Earth</dd>
-                    <dt>address</dt><dd>sq5Abt</dd>
-                    <dt>sgc_designation</dt><dd>P4X-126</dd>
-                </dl>
-            ''')
-        )
-
-    @mute_logger('odoo.http')
-    def test_models3_stargate_ko(self):
-        milky_way = self.env.ref('test_http.milky_way')
-        res = self.url_open(f'/test_http/{milky_way.id}/9999')  # unknown gate
-        self.assertEqual(res.status_code, 400)
-        self.assertIn("The goa'uld destroyed the gate", html.unescape(res.text))
-
-
-@tagged('post_install', '-at_install')
-class TestHttpMisc(TestHttpBase):
-    def test_misc0_redirect(self):
-        res = self.nodb_url_open('/test_http//greeting')
-        self.assertEqual(res.status_code, 301)
-        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/test_http/greeting')
-
-    def test_misc1_reverse_proxy(self):
-        # client <-> reverse-proxy <-> odoo
-        client_ip = '127.0.0.16'
-        reverseproxy_ip = gethostbyname(HOST)
-        host = 'mycompany.odoo.com'
-
-        headers = {
-            'Host': '',
-            'X-Forwarded-For': client_ip,
-            'X-Forwarded-Host': host,
-            'X-Forwarded-Proto': 'https'
-        }
-
-        # Don't trust client-sent forwarded headers
-        with patch.object(config, 'options', {**config.options, 'proxy_mode': False}):
-            res = self.nodb_url_open('/test_http/wsgi_environ', headers=headers)
-            self.assertEqual(res.status_code, 200)
-            self.assertEqual(res.json()['REMOTE_ADDR'], reverseproxy_ip)
-            self.assertEqual(res.json()['HTTP_HOST'], '')
-
-        # Trust proxy-sent forwarded headers
-        with patch.object(config, 'options', {**config.options, 'proxy_mode': True}):
-            res = self.nodb_url_open('/test_http/wsgi_environ', headers=headers)
-            self.assertEqual(res.status_code, 200)
-            self.assertEqual(res.json()['REMOTE_ADDR'], client_ip)
-            self.assertEqual(res.json()['HTTP_HOST'], host)
-
-
-@tagged('post_install', '-at_install')
-class TestHttpCors(TestHttpBase):
-    def test_cors0_http_default(self):
-        res_opt = self.opener.options(f'{self.base_url()}/test_http/cors_http_default', timeout=10, allow_redirects=False)
-        self.assertIn(res_opt.status_code, (200, 204))
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Origin'), '*')
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Methods'), 'GET, POST')
-        self.assertEqual(res_opt.headers.get('Access-Control-Max-Age'), '86400')  # one day
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Headers'), 'Origin, X-Requested-With, Content-Type, Accept, Authorization')
-
-        res_get = self.url_open('/test_http/cors_http_default')
-        self.assertEqual(res_get.status_code, 200)
-        self.assertEqual(res_get.headers.get('Access-Control-Allow-Origin'), '*')
-        self.assertEqual(res_get.headers.get('Access-Control-Allow-Methods'), 'GET, POST')
-
-    def test_cors1_http_methods(self):
-        res_opt = self.opener.options(f'{self.base_url()}/test_http/cors_http_methods', timeout=10, allow_redirects=False)
-        self.assertIn(res_opt.status_code, (200, 204))
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Origin'), '*')
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Methods'), 'GET, PUT')
-        self.assertEqual(res_opt.headers.get('Access-Control-Max-Age'), '86400')  # one day
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Headers'), 'Origin, X-Requested-With, Content-Type, Accept, Authorization')
-
-        res_post = self.url_open('/test_http/cors_http_methods')
-        self.assertEqual(res_post.status_code, 200)
-        self.assertEqual(res_post.headers.get('Access-Control-Allow-Origin'), '*')
-        self.assertEqual(res_post.headers.get('Access-Control-Allow-Methods'), 'GET, PUT')
-
-    def test_cors2_json(self):
-        res_opt = self.opener.options(f'{self.base_url()}/test_http/cors_json', timeout=10, allow_redirects=False)
-        self.assertIn(res_opt.status_code, (200, 204), res_opt.text)
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Origin'), '*')
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Methods'), 'POST')
-        self.assertEqual(res_opt.headers.get('Access-Control-Max-Age'), '86400')  # one day
-        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Headers'), 'Origin, X-Requested-With, Content-Type, Accept, Authorization')
-
-        res_post = self.url_open('/test_http/cors_json', data=json.dumps({'params': {}}), headers=CT_JSON)
-        self.assertEqual(res_post.status_code, 200)
-        self.assertEqual(res_post.headers.get('Access-Control-Allow-Origin'), '*')
-        self.assertEqual(res_post.headers.get('Access-Control-Allow-Methods'), 'POST')
-
-@tagged('post_install', '-at_install')
-class TestHttpEnsureDb(TestHttpBase):
-    def setUp(self):
-        super().setUp()
-        self.db_list = ['db0', 'db1']
-
-    def test_ensure_db0_db_selector(self):
-        res = self.multidb_url_open('/test_http/ensure_db')
-        res.raise_for_status()
-        self.assertEqual(res.status_code, 303)
-        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/web/database/selector')
-
-    def test_ensure_db1_grant_db(self):
-        res = self.multidb_url_open('/test_http/ensure_db?db=db0', timeout=10000)
-        res.raise_for_status()
-        self.assertEqual(res.status_code, 302)
-        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/test_http/ensure_db')
-        self.assertEqual(odoo.http.root.session_store.get(res.cookies['session_id']).db, 'db0')
-
-        # follow the redirection
-        res = self.multidb_url_open('/test_http/ensure_db')
-        res.raise_for_status()
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, 'db0')
-
-    def test_ensure_db2_use_session_db(self):
-        session = self.authenticate(None, None)
-        session.db = 'db0'
-        odoo.http.root.session_store.save(session)
-
-        res = self.multidb_url_open('/test_http/ensure_db')
-        res.raise_for_status()
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, 'db0')
-
-    def test_ensure_db3_change_db(self):
-        session = self.authenticate(None, None)
-        session.db = 'db0'
-        odoo.http.root.session_store.save(session)
-
-        res = self.multidb_url_open('/test_http/ensure_db?db=db1')
-        res.raise_for_status()
-        self.assertEqual(res.status_code, 302)
-        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/test_http/ensure_db')
-
-        new_session = odoo.http.root.session_store.get(res.cookies['session_id'])
-        self.assertNotEqual(session.sid, new_session.sid)
-        self.assertEqual(new_session.db, 'db1')
-        self.assertEqual(new_session.uid, None)
-
-        # follow redirection
-        self.opener.cookies['session_id'] = new_session.sid
-        res = self.multidb_url_open('/test_http/ensure_db')
-        res.raise_for_status()
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.text, 'db1')
-
-
-class TestHttpSession(TestHttpBase):
-
-    @mute_logger('odoo.http')  # greeting_none called ignoring args {'debug'}
-    def test_session0_debug_mode(self):
-        session = self.authenticate(None, None)
-        self.assertEqual(session.debug, '')
-        self.db_url_open('/test_http/greeting').raise_for_status()
-        self.assertEqual(session.debug, '')
-        self.db_url_open('/test_http/greeting?debug=1').raise_for_status()
-        self.assertEqual(session.debug, '1')
-        self.db_url_open('/test_http/greeting').raise_for_status()
-        self.assertEqual(session.debug, '1')
-        self.db_url_open('/test_http/greeting?debug=').raise_for_status()
-        self.assertEqual(session.debug, '')
-
-    def test_session1_default_session(self):
-        # The default session should not be saved on the filestore.
-        with patch.object(odoo.http.root.session_store, 'save') as mock_save:
-            res = self.db_url_open('/test_http/greeting')
-            res.raise_for_status()
-            try:
-                mock_save.assert_not_called()
-            except AssertionError as exc:
-                msg = f'save() was called with args: {mock_save.call_args}'
-                raise AssertionError(msg) from exc
-
-    def test_session2_geoip(self):
-        real_save = odoo.http.root.session_store.save
-        with patch.object(odoo.http.root.geoip_resolver, 'resolve') as mock_resolve,\
-             patch.object(odoo.http.root.session_store, 'save') as mock_save:
-            mock_resolve.return_value = GEOIP_ODOO_FARM_2
-            mock_save.side_effect = real_save
-
-            # Geoip is lazy: it should be computed only when necessary.
-            self.nodb_url_open('/test_http/greeting').raise_for_status()
-            mock_resolve.assert_not_called()
-
-            # Geoip is like the defaut session: the session should not
-            # be stored only due to geoip.
-            mock_resolve.reset_mock()
-            mock_save.reset_mock()
-            res = self.nodb_url_open('/test_http/geoip')
-            res.raise_for_status()
-            self.assertEqual(res.text, str(GEOIP_ODOO_FARM_2))
-            mock_save.assert_not_called()
-
-            # Geoip is cached on the session: we shouldn't geolocate the
-            # same ip multiple times.
-            mock_resolve.reset_mock()
-            mock_save.reset_mock()
-            self.nodb_url_open('/test_http/save_session').raise_for_status()
-            self.nodb_url_open('/test_http/geoip').raise_for_status()
-            res = self.nodb_url_open('/test_http/geoip')
-            res.raise_for_status()
-            self.assertEqual(res.text, str(GEOIP_ODOO_FARM_2))
-            mock_resolve.assert_called_once()
-
-class TestHttpJsonError(TestHttpBase):
-
-    jsonrpc_error_structure = {
-        'error': {
-            'code': ...,
-            'data': {
-                'arguments': ...,
-                'context': ...,
-                'debug': ...,
-                'message': ...,
-                'name': ...,
-            },
-            'message': ...,
-        },
-        'id': ...,
-        'jsonrpc': ...,
-    }
-
-    def assertIsErrorPayload(self, payload):
-        self.assertEqual(
-            set(payload),
-            set(self.jsonrpc_error_structure),
-        )
-        self.assertEqual(
-            set(payload['error']),
-            set(self.jsonrpc_error_structure['error']),
-        )
-        self.assertEqual(
-            set(payload['error']['data']),
-            set(self.jsonrpc_error_structure['error']['data']),
-        )
-
-
-    @mute_logger('odoo.http')
-    def test_errorjson0_value_error(self):
-        res = self.db_url_open('/test_http/json_value_error',
-            data=json.dumps({'jsonrpc': '2.0', 'id': 1234, 'params': {}}),
-            headers=CT_JSON
-        )
-        res.raise_for_status()
-
-        self.assertEqual(res.status_code, 200)
-        self.assertEqual(res.headers.get('Content-Type', ''), 'application/json; charset=utf-8')
-
-        payload = res.json()
-        self.assertIsErrorPayload(payload)
-
-        error_data = payload['error']['data']
-        self.assertEqual(error_data['name'], 'builtins.ValueError')
-        self.assertEqual(error_data['message'], 'Unknown destination')
-        self.assertEqual(error_data['arguments'], ['Unknown destination'])
-        self.assertEqual(error_data['context'], {})
-
-    @mute_logger('odoo.http')
-    def test_errorjson1_dev_mode_werkzeug(self):
-        with patch.object(config, 'options', {**config.options, 'dev_mode': 'werkzeug'}):
-            self.test_errorjson0_value_error()

odoo/addons/test_http/tests/test_misc.py

@@ -0,0 +1,164 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+import json
+from socket import gethostbyname
+from unittest.mock import patch
+from urllib.parse import urlparse
+
+import odoo
+from odoo.http import root
+from odoo.tests import tagged
+from odoo.tests.common import HOST
+from odoo.tools import config, file_path
+from odoo.addons.test_http.controllers import CT_JSON
+
+from .test_common import TestHttpBase
+
+
+@tagged('post_install', '-at_install')
+class TestHttpMisc(TestHttpBase):
+    def test_misc0_redirect(self):
+        res = self.nodb_url_open('/test_http//greeting')
+        self.assertEqual(res.status_code, 301)
+        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/test_http/greeting')
+
+    def test_misc1_reverse_proxy(self):
+        # client <-> reverse-proxy <-> odoo
+        client_ip = '127.0.0.16'
+        reverseproxy_ip = gethostbyname(HOST)
+        host = 'mycompany.odoo.com'
+
+        headers = {
+            'Host': '',
+            'X-Forwarded-For': client_ip,
+            'X-Forwarded-Host': host,
+            'X-Forwarded-Proto': 'https'
+        }
+
+        # Don't trust client-sent forwarded headers
+        with patch.object(config, 'options', {**config.options, 'proxy_mode': False}):
+            res = self.nodb_url_open('/test_http/wsgi_environ', headers=headers)
+            self.assertEqual(res.status_code, 200)
+            self.assertEqual(res.json()['REMOTE_ADDR'], reverseproxy_ip)
+            self.assertEqual(res.json()['HTTP_HOST'], '')
+
+        # Trust proxy-sent forwarded headers
+        with patch.object(config, 'options', {**config.options, 'proxy_mode': True}):
+            res = self.nodb_url_open('/test_http/wsgi_environ', headers=headers)
+            self.assertEqual(res.status_code, 200)
+            self.assertEqual(res.json()['REMOTE_ADDR'], client_ip)
+            self.assertEqual(res.json()['HTTP_HOST'], host)
+
+    def test_misc3_is_static_file(self):
+        uri = 'test_http/static/src/img/gizeh.png'
+        path = file_path(uri)
+
+        # Valid URLs
+        self.assertEqual(root.get_static_file(f'/{uri}'), path, "Valid file")
+        self.assertEqual(root.get_static_file(f'odoo.com/{uri}', host='odoo.com'), path, "Valid file with valid host")
+        self.assertEqual(root.get_static_file(f'http://odoo.com/{uri}', host='odoo.com'), path, "Valid file with valid host")
+
+        # Invalid URLs
+        self.assertIsNone(root.get_static_file('/test_http/i-dont-exist'), "File doesn't exist")
+        self.assertIsNone(root.get_static_file('/test_http/__manifest__.py'), "File is not static")
+        self.assertIsNone(root.get_static_file(f'odoo.com/{uri}'), "No host allowed")
+        self.assertIsNone(root.get_static_file(f'http://odoo.com/{uri}'), "No host allowed")
+
+
+@tagged('post_install', '-at_install')
+class TestHttpCors(TestHttpBase):
+    def test_cors0_http_default(self):
+        res_opt = self.opener.options(f'{self.base_url()}/test_http/cors_http_default', timeout=10, allow_redirects=False)
+        self.assertIn(res_opt.status_code, (200, 204))
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Origin'), '*')
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Methods'), 'GET, POST')
+        self.assertEqual(res_opt.headers.get('Access-Control-Max-Age'), '86400')  # one day
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Headers'), 'Origin, X-Requested-With, Content-Type, Accept, Authorization')
+
+        res_get = self.url_open('/test_http/cors_http_default')
+        self.assertEqual(res_get.status_code, 200)
+        self.assertEqual(res_get.headers.get('Access-Control-Allow-Origin'), '*')
+        self.assertEqual(res_get.headers.get('Access-Control-Allow-Methods'), 'GET, POST')
+
+    def test_cors1_http_methods(self):
+        res_opt = self.opener.options(f'{self.base_url()}/test_http/cors_http_methods', timeout=10, allow_redirects=False)
+        self.assertIn(res_opt.status_code, (200, 204))
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Origin'), '*')
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Methods'), 'GET, PUT')
+        self.assertEqual(res_opt.headers.get('Access-Control-Max-Age'), '86400')  # one day
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Headers'), 'Origin, X-Requested-With, Content-Type, Accept, Authorization')
+
+        res_post = self.url_open('/test_http/cors_http_methods')
+        self.assertEqual(res_post.status_code, 200)
+        self.assertEqual(res_post.headers.get('Access-Control-Allow-Origin'), '*')
+        self.assertEqual(res_post.headers.get('Access-Control-Allow-Methods'), 'GET, PUT')
+
+    def test_cors2_json(self):
+        res_opt = self.opener.options(f'{self.base_url()}/test_http/cors_json', timeout=10, allow_redirects=False)
+        self.assertIn(res_opt.status_code, (200, 204), res_opt.text)
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Origin'), '*')
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Methods'), 'POST')
+        self.assertEqual(res_opt.headers.get('Access-Control-Max-Age'), '86400')  # one day
+        self.assertEqual(res_opt.headers.get('Access-Control-Allow-Headers'), 'Origin, X-Requested-With, Content-Type, Accept, Authorization')
+
+        res_post = self.url_open('/test_http/cors_json', data=json.dumps({'params': {}}), headers=CT_JSON)
+        self.assertEqual(res_post.status_code, 200)
+        self.assertEqual(res_post.headers.get('Access-Control-Allow-Origin'), '*')
+        self.assertEqual(res_post.headers.get('Access-Control-Allow-Methods'), 'POST')
+
+
+@tagged('post_install', '-at_install')
+class TestHttpEnsureDb(TestHttpBase):
+    def setUp(self):
+        super().setUp()
+        self.db_list = ['db0', 'db1']
+
+    def test_ensure_db0_db_selector(self):
+        res = self.multidb_url_open('/test_http/ensure_db')
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 303)
+        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/web/database/selector')
+
+    def test_ensure_db1_grant_db(self):
+        res = self.multidb_url_open('/test_http/ensure_db?db=db0', timeout=10000)
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 302)
+        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/test_http/ensure_db')
+        self.assertEqual(odoo.http.root.session_store.get(res.cookies['session_id']).db, 'db0')
+
+        # follow the redirection
+        res = self.multidb_url_open('/test_http/ensure_db')
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, 'db0')
+
+    def test_ensure_db2_use_session_db(self):
+        session = self.authenticate(None, None)
+        session.db = 'db0'
+        odoo.http.root.session_store.save(session)
+
+        res = self.multidb_url_open('/test_http/ensure_db')
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, 'db0')
+
+    def test_ensure_db3_change_db(self):
+        session = self.authenticate(None, None)
+        session.db = 'db0'
+        odoo.http.root.session_store.save(session)
+
+        res = self.multidb_url_open('/test_http/ensure_db?db=db1')
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 302)
+        self.assertEqual(urlparse(res.headers.get('Location', '')).path, '/test_http/ensure_db')
+
+        new_session = odoo.http.root.session_store.get(res.cookies['session_id'])
+        self.assertNotEqual(session.sid, new_session.sid)
+        self.assertEqual(new_session.db, 'db1')
+        self.assertEqual(new_session.uid, None)
+
+        # follow redirection
+        res = self.multidb_url_open('/test_http/ensure_db')
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, 'db1')

odoo/addons/test_http/tests/test_models.py

@@ -0,0 +1,66 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+
+from odoo.tests import tagged
+from odoo.tests.common import new_test_user
+from odoo.tools import mute_logger
+from odoo.addons.test_http.utils import HtmlTokenizer
+
+from .test_common import TestHttpBase
+
+
+@tagged('post_install', '-at_install')
+class TestHttpModels(TestHttpBase):
+    def setUp(self):
+        super().setUp()
+        self.jackoneill = new_test_user(self.env, 'jackoneill', context={'lang': 'en_US'})
+        self.authenticate('jackoneill', 'jackoneill')
+
+    def test_models0_galaxy_ok(self):
+        milky_way = self.env.ref('test_http.milky_way')
+
+        res = self.url_open(f"/test_http/{milky_way.id}")
+
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(
+            HtmlTokenizer.tokenize(res.text),
+            HtmlTokenizer.tokenize('''\
+                <p>Milky Way</p>
+                <ul>
+                    <li><a href="/test_http/1/1">Earth (P4X-126)</a></li>
+                    <li><a href="/test_http/1/2">Abydos (P2X-125)</a></li>
+                    <li><a href="/test_http/1/3">Dakara (P5C-113)</a></li>
+                </ul>
+                ''')
+            )
+
+    @mute_logger('odoo.http')
+    def test_models1_galaxy_ko(self):
+        res = self.url_open("/test_http/404")  # unknown galaxy
+        self.assertEqual(res.status_code, 400)
+        self.assertIn('The Ancients did not settle there.', res.text)
+
+    def test_models2_stargate_ok(self):
+        milky_way = self.env.ref('test_http.milky_way')
+        earth = self.env.ref('test_http.earth')
+
+        res = self.url_open(f'/test_http/{milky_way.id}/{earth.id}')
+
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(
+            HtmlTokenizer.tokenize(res.text),
+            HtmlTokenizer.tokenize('''\
+                <dl>
+                    <dt>name</dt><dd>Earth</dd>
+                    <dt>address</dt><dd>sq5Abt</dd>
+                    <dt>sgc_designation</dt><dd>P4X-126</dd>
+                </dl>
+            ''')
+        )
+
+    @mute_logger('odoo.http')
+    def test_models3_stargate_ko(self):
+        milky_way = self.env.ref('test_http.milky_way')
+        res = self.url_open(f'/test_http/{milky_way.id}/9999')  # unknown gate
+        self.assertEqual(res.status_code, 400)
+        self.assertIn("The goa'uld destroyed the gate", res.text)

odoo/addons/test_http/tests/test_session.py

@@ -0,0 +1,76 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+from unittest.mock import patch
+
+import odoo
+from odoo.tools import mute_logger
+from .test_common import TestHttpBase
+
+
+GEOIP_ODOO_FARM_2 = {
+    'city': 'Ramillies',
+    'country_code': 'BE',
+    'country_name': 'Belgium',
+    'latitude': 50.6314,
+    'longitude': 4.8573,
+    'region': 'WAL',
+    'time_zone': 'Europe/Brussels'
+}
+
+
+class TestHttpSession(TestHttpBase):
+
+    @mute_logger('odoo.http')  # greeting_none called ignoring args {'debug'}
+    def test_session0_debug_mode(self):
+        session = self.authenticate(None, None)
+        self.assertEqual(session.debug, '')
+        self.db_url_open('/test_http/greeting').raise_for_status()
+        self.assertEqual(session.debug, '')
+        self.db_url_open('/test_http/greeting?debug=1').raise_for_status()
+        self.assertEqual(session.debug, '1')
+        self.db_url_open('/test_http/greeting').raise_for_status()
+        self.assertEqual(session.debug, '1')
+        self.db_url_open('/test_http/greeting?debug=').raise_for_status()
+        self.assertEqual(session.debug, '')
+
+    def test_session1_default_session(self):
+        # The default session should not be saved on the filestore.
+        with patch.object(odoo.http.root.session_store, 'save') as mock_save:
+            res = self.db_url_open('/test_http/greeting')
+            res.raise_for_status()
+            try:
+                mock_save.assert_not_called()
+            except AssertionError as exc:
+                msg = f'save() was called with args: {mock_save.call_args}'
+                raise AssertionError(msg) from exc
+
+    def test_session2_geoip(self):
+        real_save = odoo.http.root.session_store.save
+        with patch.object(odoo.http.root.geoip_resolver, 'resolve') as mock_resolve,\
+             patch.object(odoo.http.root.session_store, 'save') as mock_save:
+            mock_resolve.return_value = GEOIP_ODOO_FARM_2
+            mock_save.side_effect = real_save
+
+            # Geoip is lazy: it should be computed only when necessary.
+            self.nodb_url_open('/test_http/greeting').raise_for_status()
+            mock_resolve.assert_not_called()
+
+            # Geoip is like the defaut session: the session should not
+            # be stored only due to geoip.
+            mock_resolve.reset_mock()
+            mock_save.reset_mock()
+            res = self.nodb_url_open('/test_http/geoip')
+            res.raise_for_status()
+            self.assertEqual(res.text, str(GEOIP_ODOO_FARM_2))
+            mock_save.assert_not_called()
+
+            # Geoip is cached on the session: we shouldn't geolocate the
+            # same ip multiple times.
+            mock_resolve.reset_mock()
+            mock_save.reset_mock()
+            self.nodb_url_open('/test_http/save_session').raise_for_status()
+            self.nodb_url_open('/test_http/geoip').raise_for_status()
+            res = self.nodb_url_open('/test_http/geoip')
+            res.raise_for_status()
+            self.assertEqual(res.text, str(GEOIP_ODOO_FARM_2))
+            mock_resolve.assert_called_once()

odoo/addons/test_http/tests/test_static.py

@@ -0,0 +1,263 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+from datetime import datetime, timedelta
+from os.path import basename, join as opj
+from unittest.mock import patch
+from freezegun import freeze_time
+
+from odoo.tests import tagged
+from odoo.tools import config, file_open
+
+from .test_common import TestHttpBase
+
+
+@tagged('post_install', '-at_install')
+class TestHttpStaticCommon(TestHttpBase):
+    @classmethod
+    def setUpClass(cls):
+        super().setUpClass()
+        cls.classPatch(config, 'options', {**config.options, 'x_sendfile': False})
+
+        with file_open('test_http/static/src/img/gizeh.png', 'rb') as file:
+            cls.gizeh_data = file.read()
+
+    def assertDownload(
+        self, url, assert_status_code, assert_headers, assert_content=None
+    ):
+        res = self.db_url_open(url)
+        res.raise_for_status()
+        self.assertEqual(res.status_code, assert_status_code)
+        for header_name, header_value in assert_headers.items():
+            self.assertEqual(res.headers.get(header_name), header_value)
+        if assert_content:
+            self.assertEqual(res.content, assert_content)
+        return res
+
+    def assertDownloadGizeh(self, url, x_sendfile=None, assert_filename='gizeh.png'):
+        headers = {
+            'Content-Length': '814',
+            'Content-Type': 'image/png',
+            'Content-Disposition': f'inline; filename={assert_filename}'
+        }
+
+        if x_sendfile:
+            sha = basename(x_sendfile)
+            headers['X-Sendfile'] = x_sendfile
+            headers['X-Accel-Redirect'] = f'/web/filestore/{self.cr.dbname}/{sha[:2]}/{sha}'
+            headers['Content-Length'] = '0'
+
+        return self.assertDownload(url, 200, headers, b'' if x_sendfile else self.gizeh_data)
+
+
+@tagged('post_install', '-at_install')
+class TestHttpStatic(TestHttpStaticCommon):
+    def test_static00_static(self):
+        with self.subTest(x_sendfile=False):
+            res = self.assertDownloadGizeh('/test_http/static/src/img/gizeh.png')
+            self.assertEqual(res.headers.get('Cache-Control', ''), 'public, max-age=604800')
+
+        with self.subTest(x_sendfile=True), \
+             patch.object(config, 'options', {**config.options, 'x_sendfile': True}):
+            # The file is outside of the filestore, X-Sendfile disabled
+            res = self.assertDownloadGizeh('/test_http/static/src/img/gizeh.png', x_sendfile=False)
+            self.assertEqual(res.headers.get('Cache-Control', ''), 'public, max-age=604800')
+
+    def test_static01_debug_assets(self):
+        session = self.authenticate(None, None)
+        session.debug = 'assets'
+
+        res = self.assertDownloadGizeh('/test_http/static/src/img/gizeh.png')
+        self.assertEqual(res.headers.get('Cache-Control', ''), 'no-cache, max-age=0')
+
+    def test_static02_not_found(self):
+        res = self.nodb_url_open("/test_http/static/i-dont-exist")
+        self.assertEqual(res.status_code, 404)
+
+    def test_static03_attachment_fallback(self):
+        attachment = self.env.ref('test_http.gizeh_png')
+
+        with self.subTest(x_sendfile=False):
+            self.assertDownloadGizeh(attachment.url)
+
+        with self.subTest(x_sendfile=True), \
+             patch.object(config, 'options', {**config.options, 'x_sendfile': True}):
+            self.assertDownloadGizeh(
+                attachment.url,
+                x_sendfile=opj(config.filestore(self.env.cr.dbname), attachment.store_fname),
+            )
+
+    def test_static04_web_content(self):
+        attachment = self.env.ref('test_http.gizeh_png')
+
+        with self.subTest(x_sendfile=False):
+            self.assertDownloadGizeh('/web/content/test_http.gizeh_png')
+
+        with self.subTest(x_sendfile=True), \
+             patch.object(config, 'options', {**config.options, 'x_sendfile': True}):
+            self.assertDownloadGizeh(
+                '/web/content/test_http.gizeh_png',
+                x_sendfile=opj(config.filestore(self.env.cr.dbname), attachment.store_fname),
+            )
+
+    def test_static05_web_image(self):
+        attachment = self.env.ref('test_http.gizeh_png')
+
+        with self.subTest(x_sendfile=False):
+            self.assertDownloadGizeh('/web/image/test_http.gizeh_png')
+
+        with self.subTest(x_sendfile=True), \
+             patch.object(config, 'options', {**config.options, 'x_sendfile': True}):
+            self.assertDownloadGizeh(
+                '/web/image/test_http.gizeh_png',
+                x_sendfile=opj(config.filestore(self.env.cr.dbname), attachment.store_fname),
+            )
+
+    def test_static06_attachment_internal_url(self):
+        with self.subTest(x_sendfile=False):
+            self.assertDownloadGizeh('/web/image/test_http.gizeh_url')
+
+        with self.subTest(x_sendfile=True), \
+             patch.object(config, 'options', {**config.options, 'x_sendfile': True}):
+            # The file is outside of the filestore, X-Sendfile disabled
+            self.assertDownloadGizeh('/web/image/test_http.gizeh_url', x_sendfile=False)
+
+    def test_static07_attachment_external_url(self):
+        res = self.db_url_open('/web/content/test_http.rickroll')
+        res.raise_for_status()
+        self.assertEqual(res.status_code, 301)
+        self.assertEqual(res.headers.get('Location'), 'https://www.youtube.com/watch?v=dQw4w9WgXcQ')
+
+    def test_static08_binary_field_attach(self):
+        earth = self.env.ref('test_http.earth')
+        attachment = self.env['ir.attachment'].search([
+            ('res_model', '=', 'test_http.stargate'),
+            ('res_id', '=', earth.id),
+            ('res_field', '=', 'glyph_attach')
+        ], limit=1)
+        attachment_path = opj(config.filestore(self.env.cr.dbname), attachment.store_fname)
+
+        with self.subTest(x_sendfile=False):
+            self.assertDownloadGizeh(
+                f'/web/content/test_http.stargate/{earth.id}/glyph_attach',
+                assert_filename='Earth.png'
+            )
+
+        with self.subTest(x_sendfile=True), \
+             patch.object(config, 'options', {**config.options, 'x_sendfile': True}):
+            self.assertDownloadGizeh(
+                f'/web/content/test_http.stargate/{earth.id}/glyph_attach',
+                x_sendfile=attachment_path,
+                assert_filename='Earth.png'
+            )
+
+    def test_static09_binary_field_inline(self):
+        self.assertDownloadGizeh(
+            '/web/content/test_http.earth?field=glyph_inline',
+            assert_filename='Earth.png'
+        )
+
+    def test_static10_filename(self):
+        with self.subTest("record name"):
+            self.assertDownloadGizeh(
+                '/web/content/test_http.gizeh_png',
+                assert_filename='gizeh.png',
+            )
+
+        with self.subTest("forced name"):
+            self.assertDownloadGizeh(
+                '/web/content/test_http.gizeh_png?filename=pyramid.png',
+                assert_filename='pyramid.png',
+            )
+
+        with self.subTest("filename field"):
+            self.assertDownloadGizeh(
+                '/web/content/test_http.earth?field=glyph_inline&filename_field=address',
+                assert_filename='sq5Abt.png',
+            )
+
+    def test_static11_bad_filenames(self):
+        with self.subTest("missing record name"):
+            gizeh = self.env.ref('test_http.gizeh_png')
+            realname = gizeh.name
+            gizeh.name = ''
+            try:
+                self.assertDownloadGizeh(
+                    '/web/content/test_http.gizeh_png',
+                    assert_filename=f'ir_attachment-{gizeh.id}-raw.png'
+                )
+            finally:
+                gizeh.name = realname
+
+        with self.subTest("missing file extension"):
+            self.assertDownloadGizeh(
+                '/web/content/test_http.gizeh_png?filename=pyramid',
+                assert_filename='pyramid.png',
+            )
+
+        with self.subTest("wrong file extension"):
+            res = self.assertDownloadGizeh(
+                '/web/content/test_http.gizeh_png?filename=pyramid.jpg',
+                assert_filename='pyramid.jpg',
+            )
+            self.assertEqual(res.headers['Content-Type'], 'image/png')
+
+        with self.subTest("dotted name"):
+            res = self.assertDownloadGizeh(
+                '/web/content/test_http.gizeh_png?filename=pyramid.of.gizeh',
+                assert_filename='pyramid.of.gizeh.png',
+            )
+
+
+class TestHttpStaticCache(TestHttpStaticCommon):
+    @freeze_time(datetime.utcnow())
+    def test_static_cache0_standard(self, domain=''):
+        # Wed, 21 Oct 2015 07:28:00 GMT
+        # The timezone should be %Z (instead of 'GMT' hardcoded) but
+        # somehow strftime doesn't set it.
+        http_date_format = '%a, %d %b %Y %H:%M:%S GMT'
+        one_week_away = (datetime.utcnow() + timedelta(weeks=1)).strftime(http_date_format)
+
+        res1 = self.nodb_url_open(f'{domain}/test_http/static/src/img/gizeh.png')
+        res1.raise_for_status()
+        self.assertEqual(res1.status_code, 200)
+        self.assertEqual(res1.headers.get('Cache-Control'), 'public, max-age=604800')  # one week
+        self.assertEqual(res1.headers.get('Expires'), one_week_away)
+        self.assertIn('ETag', res1.headers)
+
+        res2 = self.nodb_url_open(f'{domain}/test_http/static/src/img/gizeh.png', headers={
+            'If-None-Match': res1.headers['ETag']
+        })
+        res2.raise_for_status()
+        self.assertEqual(res2.status_code, 304, "We should not download the file again.")
+
+    @freeze_time(datetime.utcnow())
+    def test_static_cache1_unique(self, domain=''):
+        # Wed, 21 Oct 2015 07:28:00 GMT
+        # The timezone should be %Z (instead of 'GMT' hardcoded) but
+        # somehow strftime doesn't set it.
+        http_date_format = '%a, %d %b %Y %H:%M:%S GMT'
+        one_year_away = (datetime.utcnow() + timedelta(days=365)).strftime(http_date_format)
+
+        res1 = self.assertDownloadGizeh(f'{domain}/web/content/test_http.gizeh_png?unique=1')
+        self.assertEqual(res1.headers.get('Cache-Control'), 'public, max-age=31536000')  # one year
+        self.assertEqual(res1.headers.get('Expires'), one_year_away)
+        self.assertIn('ETag', res1.headers)
+
+        res2 = self.db_url_open(f'{domain}/web/content/test_http.gizeh_png?unique=1', headers={
+            'If-None-Match': res1.headers['ETag']
+        })
+        res2.raise_for_status()
+        self.assertEqual(res2.status_code, 304, "We should not download the file again.")
+
+    @freeze_time(datetime.utcnow())
+    def test_static_cache2_nocache(self, domain=''):
+        res1 = self.assertDownloadGizeh(f'{domain}/web/content/test_http.gizeh_png?nocache=1')
+        self.assertEqual(res1.headers.get('Cache-Control'), 'no-cache')
+        self.assertNotIn('Expires', res1.headers)
+        self.assertIn('ETag', res1.headers)
+
+        res2 = self.db_url_open(f'{domain}/web/content/test_http.gizeh_png?nocache=1', headers={
+            'If-None-Match': res1.headers['ETag']
+        })
+        res2.raise_for_status()
+        self.assertEqual(res2.status_code, 304, "We should not download the file again.")

odoo/addons/test_http/tests/test_web_server.py

@@ -0,0 +1,27 @@
+# Part of Odoo. See LICENSE file for full copyright and licensing details.
+
+from os import getenv
+from odoo.tests import tagged
+from .test_static import TestHttpStatic, TestHttpStaticCache
+
+
+# Small configuration to run the tests against a web server.
+# WEB_SERVER_URL=http://localhost:80 odoo-bin -i test_http --test-tags webserver
+WEB_SERVER_URL = getenv('WEB_SERVER_URL', 'http://localhost:80')
+
+
+@tagged('webserver', '-standard', '-at-install')
+class TestHttpStaticWebServer(TestHttpStatic, TestHttpStaticCache):
+    @classmethod
+    def base_url(cls):
+        return WEB_SERVER_URL
+
+    def assertDownloadGizeh(self, url, x_sendfile=None, assert_filename='gizeh.png'):
+        # X-Sendfile and X-Accel-Redirect http response headers should
+        # have been consummed by the web server. We should get the
+        # ultimate response which holds the file.
+        return super().assertDownloadGizeh(
+            url,
+            x_sendfile=False,
+            assert_filename=assert_filename
+        )

odoo/http.py

@@ -108,6 +108,7 @@ endpoint
   The @route(...) decorated controller method.
 """
 
+import base64
 import cgi
 import collections
 import collections.abc
@@ -129,7 +130,11 @@ import warnings
 import zlib
 from abc import ABC, abstractmethod
 from datetime import datetime
+from io import BytesIO
 from os.path import join as opj
+from pathlib import Path
+from urllib.parse import urlparse
+from zlib import adler32
 
 import babel.core
 import psycopg2
@@ -149,13 +154,18 @@ try:
 except ImportError:
     from werkzeug.contrib.fixers import ProxyFix
 
+try:
+    from werkzeug.utils import send_file as _send_file
+except ImportError:
+    from .tools._vendor.send_file import send_file as _send_file
+
 import odoo
 from .exceptions import UserError, AccessError, AccessDenied
 from .modules.module import get_manifest
 from .modules.registry import Registry
 from .service import security, model as service_model
-from .tools import (config, consteq, date_utils, profiler, resolve_attr,
-                    submap, unique, ustr,)
+from .tools import (config, consteq, date_utils, file_path, profiler,
+                    resolve_attr, submap, unique, ustr,)
 from .tools.geoipresolver import GeoIPResolver
 from .tools.func import filter_kwargs, lazy_property
 from .tools.mimetypes import guess_mimetype
@@ -247,9 +257,6 @@ ROUTING_KEYS = {
     'alias', 'host', 'methods',
 }
 
-# The mimetypes of safe image types
-SAFE_IMAGE_MIMETYPES = {'image/jpeg', 'image/png', 'image/gif', 'image/x-icon'}
-
 # The duration of a user session before it is considered expired,
 # three months.
 SESSION_LIFETIME = 60 * 60 * 24 * 90
@@ -261,6 +268,7 @@ STATIC_CACHE = 60 * 60 * 24 * 7
 # content (usually using a hash), one year.
 STATIC_CACHE_LONG = 60 * 60 * 24 * 365
 
+
 # =========================================================
 # Helpers
 # =========================================================
@@ -331,84 +339,6 @@ def db_filter(dbs, host=None):
 def is_cors_preflight(request, endpoint):
     return request.httprequest.method == 'OPTIONS' and endpoint.routing.get('cors', False)
 
-def send_file(filepath_or_fp, filename=None, mimetype=None, mtime=None,
-              as_attachment=False, cache_timeout=STATIC_CACHE):
-    """
-    Fle streaming utility with mime and cache handling, it takes a
-    file-object or immediately the content as bytes/str.
-
-    Sends the content of a file to the client. This will use the most
-    efficient method available and configured. By default it will try to
-    use the WSGI server's file_wrapper support.
-
-    If filename of file.name is provided it will try to guess the
-    mimetype for you, but you can also explicitly provide one.
-
-    For extra security you probably want to send certain files as
-    attachment (e.g. HTML).
-
-    :param Union[os.PathLike,io.FileIO] filepath_or_fp: the filename of
-        the file to send.  Alternatively a file object might be provided
-        in which case `X-Sendfile` might not work and fall back to the
-        traditional method. Make sure that the file pointer is position-
-        ed at the start of data to send before calling :func:`send_file`
-    :param str filename: optional if file has a 'name' attribute, used
-        for attachment name and mimetype guess.
-    :param str mimetype: the mimetype of the file if provided, otherwise
-        auto detection happens based on the name.
-    :param datetime mtime: optional if file has a 'name' attribute, last
-        modification time used for conditional response.
-    :param bool as_attachment: set to `True` if you want to send this
-        file with a ``Content-Disposition: attachment`` header.
-    :param int cache_timeout: set to `False` to disable etags and
-        conditional response handling (last modified and etags)
-    :returns: the HTTP response that streams the file.
-    """
-    if isinstance(filepath_or_fp, str):
-        if not filename:
-            filename = os.path.basename(filepath_or_fp)
-        file = open(filepath_or_fp, 'rb')
-    else:
-        file = filepath_or_fp
-        if not filename:
-            filename = getattr(file, 'name', None)
-
-    # Only used when filename or mtime argument is not provided
-    path = getattr(file, 'name', 'file.bin')
-
-    if not filename:
-        filename = os.path.basename(path)
-
-    if not mimetype:
-        mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
-
-    file.seek(0, 2)
-    size = file.tell()
-    file.seek(0)
-
-    data = werkzeug.wsgi.wrap_file(request.httprequest.environ, file)
-
-    res = werkzeug.wrappers.Response(data, mimetype=mimetype, direct_passthrough=True)
-    res.content_length = size
-
-    if as_attachment:
-        res.headers.add('Content-Disposition', 'attachment', filename=filename)
-
-    if cache_timeout:
-        if not mtime:
-            with contextlib.suppress(FileNotFoundError):
-                mtime = datetime.fromtimestamp(os.path.getmtime(path))
-        if mtime:
-            res.last_modified = mtime
-        crc = zlib.adler32(filename.encode('utf-8') if isinstance(filename, str) else filename) & 0xffffffff
-        etag = f'odoo-{mtime}-{size}-{crc}'
-        if not werkzeug.http.is_resource_modified(request.httprequest.environ, etag, last_modified=mtime):
-            res = werkzeug.wrappers.Response(status=304)
-        else:
-            res.cache_control.public = True
-            res.cache_control.max_age = cache_timeout
-            res.set_etag(etag)
-    return res
 
 def serialize_exception(exception):
     name = type(exception).__name__
@@ -422,20 +352,201 @@ def serialize_exception(exception):
         'context': getattr(exception, 'context', {}),
     }
 
-def set_safe_image_headers(headers, content):
-    """Return new headers based on `headers` but with `Content-Length` and
-    `Content-Type` set appropriately depending on the given `content` only if it
-    is safe to do, as well as `X-Content-Type-Options: nosniff` so that if the
-    file is of an unsafe type, it is not interpreted as that type if the
-    `Content-type` header was already set to a different mimetype
+
+# =========================================================
+# File Streaming
+# =========================================================
+
+def send_file(filepath_or_fp, mimetype=None, as_attachment=False, filename=None, mtime=None,
+              add_etags=True, cache_timeout=STATIC_CACHE, conditional=True):
+    warnings.warn('odoo.http.send_file is deprecated, please use odoo.http.Stream instead.', DeprecationWarning, stacklevel=2)
+    return _send_file(
+        filepath_or_fp,
+        request.httprequest.environ,
+        mimetype=mimetype,
+        as_attachment=as_attachment,
+        download_name=filename,
+        last_modified=mtime,
+        etag=add_etags,
+        max_age=cache_timeout,
+        conditional=conditional
+    )
+
+
+class Stream:
     """
-    headers = werkzeug.datastructures.Headers(headers)
-    content_type = guess_mimetype(content)
-    if content_type in SAFE_IMAGE_MIMETYPES:
-        headers['Content-Type'] = content_type
-    headers['X-Content-Type-Options'] = 'nosniff'
-    headers['Content-Length'] = len(content)
-    return list(headers)
+    Send the content of a file, an attachment or a binary field via HTTP
+
+    This utility is safe, cache-aware and uses the best available
+    streaming strategy. Works best with the --x-sendfile cli option.
+
+    Create a Stream via one of the constructors: :meth:`~from_path`:,
+    :meth:`~from_attachment`: or :meth:`~from_binary_field`:, generate
+    the corresponding HTTP response object via :meth:`~get_response`:.
+
+    Instantiating a Stream object manually without using one of the
+    dedicated constructors is discouraged.
+    """
+
+    type: str = ''  # 'data' or 'path' or 'url'
+    data = None
+    path = None
+    url = None
+
+    mimetype = None
+    as_attachment = False
+    download_name = None
+    conditional = True
+    etag = True
+    last_modified = None
+    max_age = None
+    size = None
+
+    def __init__(self, **kwargs):
+        self.__dict__.update(kwargs)
+
+    @classmethod
+    def from_path(cls, path, filter_ext=('',)):
+        """ Create a :class:`~Stream`: from an addon resource. """
+        path = file_path(path, filter_ext)
+        check = adler32(path.encode())
+        stat = os.stat(path)
+        return cls(
+            type='path',
+            path=path,
+            download_name=os.path.basename(path),
+            etag=f'{int(stat.st_mtime)}-{stat.st_size}-{check}',
+            last_modified=stat.st_mtime,
+            size=stat.st_size,
+        )
+
+    @classmethod
+    def from_attachment(cls, attachment):
+        """ Create a :class:`~Stream`: from an ir.attachment record. """
+        attachment.ensure_one()
+
+        self = cls(
+            mimetype=attachment.mimetype,
+            download_name=attachment.name,
+            conditional=True,
+            etag=attachment.checksum,
+        )
+
+        if attachment.store_fname:
+            self.type = 'path'
+            self.path = werkzeug.security.safe_join(
+                os.path.abspath(config.filestore(request.db)),
+                attachment.store_fname
+            )
+            stat = os.stat(self.path)
+            self.last_modified = stat.st_mtime
+            self.size = stat.st_size
+
+        elif attachment.db_datas:
+            self.type = 'data'
+            self.data = attachment.raw
+            self.last_modified = attachment['__last_update']
+            self.size = len(self.data)
+
+        elif attachment.url:
+            # When the URL targets a file located in an addon, assume it
+            # is a path to the resource. It saves an indirection and
+            # stream the file right away.
+            static_path = root.get_static_file(
+                attachment.url,
+                host=request.httprequest.environ.get('HTTP_HOST', '')
+            )
+            if static_path:
+                self = cls.from_path(static_path)
+            else:
+                self.type = 'url'
+                self.url = attachment.url
+
+        else:
+            self.type = 'data'
+            self.data = b''
+            self.size = 0
+
+        return self
+
+    @classmethod
+    def from_binary_field(cls, record, field_name):
+        """ Create a :class:`~Stream`: from a binary field. """
+        data_b64 = record[field_name]
+        data = base64.b64decode(data_b64) if data_b64 else b''
+        return cls(
+            type='data',
+            data=data,
+            etag=request.env['ir.attachment']._compute_checksum(data),
+            last_modified=record['__last_update'] if record._log_access else None,
+            size=len(data),
+        )
+
+    def read(self):
+        """ Get the stream content as bytes. """
+        if self.type == 'url':
+            raise ValueError("Cannot read an URL")
+
+        if self.type == 'data':
+            return self.data
+
+        with open(self.path, 'rb') as file:
+            return file.read()
+
+    def get_response(self, as_attachment=None, **send_file_kwargs):
+        """
+        Create the corresponding :class:`~Response` for the current stream.
+
+        :param bool as_attachment: Indicate to the browser that it
+            should offer to save the file instead of displaying it.
+        :param send_file_kwargs: Other keyword arguments to send to
+            :func:`odoo.tools._vendor.send_file.send_file` instead of
+            the stream sensitive values. Discouraged.
+        """
+        assert self.type in ('url', 'data', 'path'), "Invalid type: {self.type!r}, should be 'url', 'data' or 'path'."
+        assert getattr(self, self.type) is not None, "There is nothing to stream, missing {self.type!r} attribute."
+
+        if self.type == 'url':
+            return request.redirect(self.url, code=301, local=False)
+
+        if as_attachment is None:
+            as_attachment = self.as_attachment
+
+        send_file_kwargs = {
+            'mimetype': self.mimetype,
+            'as_attachment': as_attachment,
+            'download_name': self.download_name,
+            'conditional': self.conditional,
+            'etag': self.etag,
+            'last_modified': self.last_modified,
+            'max_age': self.max_age,
+            'environ': request.httprequest.environ,
+            'response_class': Response,
+            **send_file_kwargs,
+        }
+
+        if self.type == 'data':
+            return _send_file(BytesIO(self.data), **send_file_kwargs)
+
+        # self.type == 'path'
+        send_file_kwargs['use_x_sendfile'] = False
+        if config['x_sendfile']:
+            with contextlib.suppress(ValueError):  # outside of the filestore
+                fspath = Path(self.path).relative_to(opj(config['data_dir'], 'filestore'))
+                x_accel_redirect = f'/web/filestore/{fspath}'
+                send_file_kwargs['use_x_sendfile'] = True
+
+        res = _send_file(self.path, **send_file_kwargs)
+
+        if 'X-Sendfile' in res.headers:
+            res.headers['X-Accel-Redirect'] = x_accel_redirect
+
+            # In case of X-Sendfile/X-Accel-Redirect, the body is empty,
+            # yet werkzeug gives the length of the file. This makes
+            # NGINX wait for content that'll never arrive.
+            res.headers['Content-Length'] = '0'
+
+        return res
 
 
 # =========================================================
@@ -1339,7 +1450,9 @@ class Request:
         try:
             directory = root.statics[module]
             filepath = werkzeug.security.safe_join(directory, path)
-            return send_file(filepath)
+            return Stream.from_path(filepath).get_response(
+                max_age=0 if 'assets' in self.session.debug else STATIC_CACHE,
+            )
         except KeyError:
             raise NotFound(f'Module "{module}" not found.\n')
         except OSError:  # cover both missing file and invalid permissions
@@ -1680,6 +1793,36 @@ class Application:
                     mod2path[module] = static_path
         return mod2path
 
+    def get_static_file(self, url, host=''):
+        """
+        Get the full-path of the file if the url resolves to a local
+        static file, otherwise return None.
+
+        Without the second host parameters, ``url`` must be an absolute
+        path, others URLs are considered faulty.
+
+        With the second host parameters, ``url`` can also be a full URI
+        and the authority found in the URL (if any) is validated against
+        the given ``host``.
+        """
+
+        netloc, path = urlparse(url)[1:3]
+        try:
+            path_netloc, module, static, resource = path.split('/', 3)
+        except ValueError:
+            return None
+
+        if ((netloc and netloc != host) or (path_netloc and path_netloc != host)):
+            return None
+
+        if (module not in self.statics or static != 'static' or not resource):
+            return None
+
+        try:
+            return file_path(f'{module}/static/{resource}')
+        except FileNotFoundError:
+            return None
+
     @lazy_property
     def nodb_routing_map(self):
         nodb_routing_map = werkzeug.routing.Map(strict_slashes=False, converters=None)
@@ -1721,6 +1864,7 @@ class Application:
             return
 
         headers['Content-Security-Policy'] = "default-src 'none'"
+        headers['X-Content-Type-Options'] = 'nosniff'
 
     def __call__(self, environ, start_response):
         """
@@ -1764,13 +1908,9 @@ class Application:
         current_thread.url = httprequest.url
 
         try:
-            segments = httprequest.path.split('/')
-            if len(segments) >= 4 and segments[2] == 'static':
-                with contextlib.suppress(NotFound):
-                    response = request._serve_static()
-                    return response(environ, start_response)
-
-            if request.db:
+            if self.get_static_file(httprequest.path):
+                response = request._serve_static()
+            elif request.db:
                 with request._get_profiler_context_manager():
                     response = request._serve_db()
             else:

odoo/models.py

@@ -6779,9 +6779,6 @@ class BaseModel(metaclass=MetaModel):
         """ Returns the filename of the placeholder to use,
             set on web/static/img by default, or the
             complete path to access it (eg: module/path/to/image.png).
-
-            If a falsy value is returned, "ir.http"._placeholder() will use
-            the default placeholder 'web/static/img/placeholder.png'.
         """
         return False
 

odoo/tools/_vendor/send_file.py

@@ -0,0 +1,223 @@
+"""
+Vendored copy of the werkzeug.utils.send_file function defined in
+werkzeug2 which is packaged in Debian 12 "Bookworm" and Ubuntu 22.04
+"Jammy". Odoo is compatible with werkzeug2 since saas-15.4.
+
+This vendored copy is deprecated, only present to ensure backward
+compatibility with older operating systems.
+
+:copyright: 2007 Pallets
+:license: BSD-3-Clause
+"""
+
+import io
+import logging
+import mimetypes
+import os
+import typing as t
+import unicodedata
+from datetime import datetime
+from time import time
+from zlib import adler32
+
+from werkzeug.datastructures import Headers
+from werkzeug.exceptions import RequestedRangeNotSatisfiable
+from werkzeug.urls import url_quote
+from werkzeug.wrappers import Response
+from werkzeug.wsgi import wrap_file
+
+_logger = logging.getLogger(__name__)
+
+
+def send_file(
+    path_or_file: t.Union[os.PathLike, str, t.IO[bytes]],
+    environ: "WSGIEnvironment",
+    mimetype: t.Optional[str] = None,
+    as_attachment: bool = False,
+    download_name: t.Optional[str] = None,
+    conditional: bool = True,
+    etag: t.Union[bool, str] = True,
+    last_modified: t.Optional[t.Union[datetime, int, float]] = None,
+    max_age: t.Optional[
+        t.Union[int, t.Callable[[t.Optional[str]], t.Optional[int]]]
+    ] = None,
+    use_x_sendfile: bool = False,
+    response_class: t.Optional[t.Type["Response"]] = None,
+    _root_path: t.Optional[t.Union[os.PathLike, str]] = None,
+) -> "Response":
+    """Send the contents of a file to the client.
+
+    The first argument can be a file path or a file-like object. Paths
+    are preferred in most cases because Werkzeug can manage the file and
+    get extra information from the path. Passing a file-like object
+    requires that the file is opened in binary mode, and is mostly
+    useful when building a file in memory with :class:`io.BytesIO`.
+
+    Never pass file paths provided by a user. The path is assumed to be
+    trusted, so a user could craft a path to access a file you didn't
+    intend.
+
+    If the WSGI server sets a ``file_wrapper`` in ``environ``, it is
+    used, otherwise Werkzeug's built-in wrapper is used. Alternatively,
+    if the HTTP server supports ``X-Sendfile``, ``use_x_sendfile=True``
+    will tell the server to send the given path, which is much more
+    efficient than reading it in Python.
+
+    :param path_or_file: The path to the file to send, relative to the
+        current working directory if a relative path is given.
+        Alternatively, a file-like object opened in binary mode. Make
+        sure the file pointer is seeked to the start of the data.
+    :param environ: The WSGI environ for the current request.
+    :param mimetype: The MIME type to send for the file. If not
+        provided, it will try to detect it from the file name.
+    :param as_attachment: Indicate to a browser that it should offer to
+        save the file instead of displaying it.
+    :param download_name: The default name browsers will use when saving
+        the file. Defaults to the passed file name.
+    :param conditional: Enable conditional and range responses based on
+        request headers. Requires passing a file path and ``environ``.
+    :param etag: Calculate an ETag for the file, which requires passing
+        a file path. Can also be a string to use instead.
+    :param last_modified: The last modified time to send for the file,
+        in seconds. If not provided, it will try to detect it from the
+        file path.
+    :param max_age: How long the client should cache the file, in
+        seconds. If set, ``Cache-Control`` will be ``public``, otherwise
+        it will be ``no-cache`` to prefer conditional caching.
+    :param use_x_sendfile: Set the ``X-Sendfile`` header to let the
+        server to efficiently send the file. Requires support from the
+        HTTP server. Requires passing a file path.
+    :param response_class: Build the response using this class. Defaults
+        to :class:`~werkzeug.wrappers.Response`.
+    :param _root_path: Do not use. For internal use only. Use
+        :func:`send_from_directory` to safely send files under a path.
+    """
+    if response_class is None:
+        response_class = Response
+
+    path = None
+    file = None
+    size = None
+    mtime = None
+    headers = Headers()
+
+    if isinstance(path_or_file, (os.PathLike, str)) or hasattr(
+        path_or_file, "__fspath__"
+    ):
+
+        # Flask will pass app.root_path, allowing its send_file wrapper
+        # to not have to deal with paths.
+        if _root_path is not None:
+            path = os.path.join(_root_path, path_or_file)
+        else:
+            path = os.path.abspath(path_or_file)
+
+        stat = os.stat(path)
+        size = stat.st_size
+        mtime = stat.st_mtime
+    else:
+        file = path_or_file
+
+    if download_name is None and path is not None:
+        download_name = os.path.basename(path)
+
+    if mimetype is None:
+        if download_name is None:
+            raise TypeError(
+                "Unable to detect the MIME type because a file name is"
+                " not available. Either set 'download_name', pass a"
+                " path instead of a file, or set 'mimetype'."
+            )
+
+        mimetype, encoding = mimetypes.guess_type(download_name)
+
+        if mimetype is None:
+            mimetype = "application/octet-stream"
+
+        # Don't send encoding for attachments, it causes browsers to
+        # save decompress tar.gz files.
+        if encoding is not None and not as_attachment:
+            headers.set("Content-Encoding", encoding)
+            if use_x_sendfile and path is not None:
+                headers["X-Accel-Charset"] = encoding
+
+    if download_name is not None:
+        try:
+            download_name.encode("ascii")
+        except UnicodeEncodeError:
+            simple = unicodedata.normalize("NFKD", download_name)
+            simple = simple.encode("ascii", "ignore").decode("ascii")
+            quoted = url_quote(download_name, safe="")
+            names = {"filename": simple, "filename*": f"UTF-8''{quoted}"}
+        else:
+            names = {"filename": download_name}
+
+        value = "attachment" if as_attachment else "inline"
+        headers.set("Content-Disposition", value, **names)
+    elif as_attachment:
+        raise TypeError(
+            "No name provided for attachment. Either set"
+            " 'download_name' or pass a path instead of a file."
+        )
+
+    if use_x_sendfile and path is not None:
+        headers["X-Sendfile"] = path
+        data = None
+    else:
+        if file is None:
+            file = open(path, "rb")  # type: ignore
+        elif isinstance(file, io.BytesIO):
+            size = file.getbuffer().nbytes
+        elif isinstance(file, io.TextIOBase):
+            raise ValueError("Files must be opened in binary mode or use BytesIO.")
+
+        data = wrap_file(environ, file)
+
+    rv = response_class(
+        data, mimetype=mimetype, headers=headers, direct_passthrough=True
+    )
+
+    if size is not None:
+        rv.content_length = size
+
+    if last_modified is not None:
+        rv.last_modified = last_modified  # type: ignore
+    elif mtime is not None:
+        rv.last_modified = mtime  # type: ignore
+
+    rv.cache_control.no_cache = True
+
+    # Flask will pass app.get_send_file_max_age, allowing its send_file
+    # wrapper to not have to deal with paths.
+    if callable(max_age):
+        max_age = max_age(path)
+
+    if max_age is not None:
+        if max_age > 0:
+            rv.cache_control.no_cache = None
+            rv.cache_control.public = True
+
+        rv.cache_control.max_age = max_age
+        rv.expires = int(time() + max_age)  # type: ignore
+
+    if isinstance(etag, str):
+        rv.set_etag(etag)
+    elif etag and path is not None:
+        check = adler32(path.encode("utf-8")) & 0xFFFFFFFF
+        rv.set_etag(f"{mtime}-{size}-{check}")
+
+    if conditional:
+        try:
+            rv = rv.make_conditional(environ, accept_ranges=True, complete_length=size)
+        except RequestedRangeNotSatisfiable:
+            if file is not None:
+                file.close()
+
+            raise
+
+        # Some x-sendfile implementations incorrectly ignore the 304
+        # status code and send the file anyway.
+        if rv.status_code == 304:
+            rv.headers.pop("x-sendfile", None)
+
+    return rv

odoo/tools/config.py

@@ -139,6 +139,10 @@ class configmanager(object):
         group.add_option("--proxy-mode", dest="proxy_mode", action="store_true", my_default=False,
                          help="Activate reverse proxy WSGI wrappers (headers rewriting) "
                               "Only enable this when running behind a trusted web proxy!")
+        group.add_option("--x-sendfile", dest="x_sendfile", action="store_true", my_default=False,
+                         help="Activate X-Sendfile (apache) and X-Accel-Redirect (nginx) "
+                              "HTTP response header to delegate the delivery of large "
+                              "files (assets/attachments) to the web server.")
         # HTTP: hidden backwards-compatibility for "*xmlrpc*" options
         hidden = optparse.SUPPRESS_HELP
         group.add_option("--xmlrpc-interface", dest="http_interface", help=hidden)
@@ -438,7 +442,7 @@ class configmanager(object):
             self.options['server_wide_modules'] = 'base,web'
 
         # if defined do not take the configfile value even if the defined value is None
-        keys = ['http_interface', 'http_port', 'longpolling_port', 'http_enable',
+        keys = ['http_interface', 'http_port', 'longpolling_port', 'http_enable', 'x_sendfile',
                 'db_name', 'db_user', 'db_password', 'db_host', 'db_sslmode',
                 'db_port', 'db_template', 'logfile', 'pidfile', 'smtp_port',
                 'email_from', 'smtp_server', 'smtp_user', 'smtp_password', 'from_filter',

odoo/tools/mimetypes.py

@@ -8,6 +8,7 @@ import collections
 import functools
 import io
 import logging
+import mimetypes
 import re
 import zipfile
 
@@ -197,11 +198,25 @@ def neuter_mimetype(mimetype, user):
     return mimetype
 
 def get_extension(filename):
-    """ Return the extension the current filename based on the heuristic that
-    ext is less than or equal to 10 chars and is alphanumeric.
+    # A file has no extension if it has no dot (ignoring the leading one
+    # of hidden files) or that what follow the last dot is not a single
+    # word, e.g. "Mr. Doe"
+    _stem, dot, ext = filename.lstrip('.').rpartition('.')
+    if not dot or not ext.isalnum():
+        return ''
 
-    :param str filename: filename to try and guess a extension for
-    :returns: detected extension or ``
-    """
-    ext = '.' in filename and filename.split('.')[-1]
-    return ext and len(ext) <= 10 and ext.isalnum() and '.' + ext.lower() or ''
+    # Assume all 4-chars extensions to be valid extensions even if it is
+    # not known from the mimetypes database. In /etc/mime.types, only 7%
+    # known extensions are longer.
+    if len(ext) <= 4:
+        return f'.{ext}'.lower()
+
+    # Use the mimetype database to determine the extension of the file.
+    guessed_mimetype, guessed_ext = mimetypes.guess_type(filename)
+    if guessed_ext:
+        return guessed_ext
+    if guessed_mimetype:
+        return f'.{ext}'.lower()
+
+    # Unknown extension.
+    return ''