Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closes odoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
302 lines
13 KiB
Python
302 lines
13 KiB
Python
# -*- coding: utf-8 -*-
|
|
# Part of Odoo. See LICENSE file for full copyright and licensing details.
|
|
|
|
import base64
|
|
import werkzeug
|
|
import werkzeug.exceptions
|
|
import werkzeug.urls
|
|
import werkzeug.wrappers
|
|
import math
|
|
|
|
from dateutil.relativedelta import relativedelta
|
|
from operator import itemgetter
|
|
|
|
from odoo import fields, http, modules, tools
|
|
from odoo.http import request
|
|
from odoo.osv import expression
|
|
|
|
|
|
class WebsiteProfile(http.Controller):
|
|
_users_per_page = 30
|
|
_pager_max_pages = 5
|
|
|
|
# Profile
|
|
# ---------------------------------------------------
|
|
|
|
def _check_avatar_access(self, user_id, **post):
|
|
""" Base condition to see user avatar independently form access rights
|
|
is to see published users having karma, meaning they participated to
|
|
frontend applications like forum or elearning. """
|
|
try:
|
|
user = request.env['res.users'].sudo().browse(user_id).exists()
|
|
except:
|
|
return False
|
|
if user:
|
|
return user.website_published and user.karma > 0
|
|
return False
|
|
|
|
def _check_user_profile_access(self, user_id):
|
|
user_sudo = request.env['res.users'].sudo().browse(user_id)
|
|
# User can access - no matter what - his own profile
|
|
if user_sudo.id == request.env.user.id:
|
|
return user_sudo
|
|
if user_sudo.karma == 0 or not user_sudo.website_published or \
|
|
(user_sudo.id != request.session.uid and request.env.user.karma < request.website.karma_profile_min):
|
|
return False
|
|
return user_sudo
|
|
|
|
def _prepare_user_values(self, **kwargs):
|
|
kwargs.pop('edit_translations', None) # avoid nuking edit_translations
|
|
values = {
|
|
'user': request.env.user,
|
|
'is_public_user': request.website.is_public_user(),
|
|
'validation_email_sent': request.session.get('validation_email_sent', False),
|
|
'validation_email_done': request.session.get('validation_email_done', False),
|
|
}
|
|
values.update(kwargs)
|
|
return values
|
|
|
|
def _prepare_user_profile_parameters(self, **post):
|
|
return post
|
|
|
|
def _prepare_user_profile_values(self, user, **post):
|
|
return {
|
|
'uid': request.env.user.id,
|
|
'user': user,
|
|
'main_object': user,
|
|
'is_profile_page': True,
|
|
'edit_button_url_param': '',
|
|
}
|
|
|
|
@http.route([
|
|
'/profile/avatar/<int:user_id>',
|
|
], type='http', auth="public", website=True, sitemap=False)
|
|
def get_user_profile_avatar(self, user_id, field='avatar_256', width=0, height=0, crop=False, **post):
|
|
if field not in ('image_128', 'image_256', 'avatar_128', 'avatar_256'):
|
|
return werkzeug.exceptions.Forbidden()
|
|
|
|
if (int(width), int(height)) == (0, 0):
|
|
width, height = tools.image_guess_size_from_field_name(field)
|
|
|
|
can_sudo = self._check_avatar_access(int(user_id), **post)
|
|
return request.env['ir.binary']._get_image_stream_from(
|
|
request.env['res.users'].sudo(can_sudo).browse(int(user_id)),
|
|
field_name=field, width=int(width), height=int(height), crop=crop
|
|
).get_response()
|
|
|
|
@http.route(['/profile/user/<int:user_id>'], type='http', auth="public", website=True)
|
|
def view_user_profile(self, user_id, **post):
|
|
user = self._check_user_profile_access(user_id)
|
|
if not user:
|
|
return request.render("website_profile.private_profile")
|
|
values = self._prepare_user_values(**post)
|
|
params = self._prepare_user_profile_parameters(**post)
|
|
values.update(self._prepare_user_profile_values(user, **params))
|
|
return request.render("website_profile.user_profile_main", values)
|
|
|
|
# Edit Profile
|
|
# ---------------------------------------------------
|
|
@http.route('/profile/edit', type='http', auth="user", website=True)
|
|
def view_user_profile_edition(self, **kwargs):
|
|
user_id = int(kwargs.get('user_id', 0))
|
|
countries = request.env['res.country'].search([])
|
|
if user_id and request.env.user.id != user_id and request.env.user._is_admin():
|
|
user = request.env['res.users'].browse(user_id)
|
|
values = self._prepare_user_values(searches=kwargs, user=user, is_public_user=False)
|
|
else:
|
|
values = self._prepare_user_values(searches=kwargs)
|
|
values.update({
|
|
'email_required': kwargs.get('email_required'),
|
|
'countries': countries,
|
|
'url_param': kwargs.get('url_param'),
|
|
})
|
|
return request.render("website_profile.user_profile_edit_main", values)
|
|
|
|
def _profile_edition_preprocess_values(self, user, **kwargs):
|
|
values = {
|
|
'name': kwargs.get('name'),
|
|
'website': kwargs.get('website'),
|
|
'email': kwargs.get('email'),
|
|
'city': kwargs.get('city'),
|
|
'country_id': int(kwargs.get('country')) if kwargs.get('country') else False,
|
|
'website_description': kwargs.get('description'),
|
|
}
|
|
|
|
if 'clear_image' in kwargs:
|
|
values['image_1920'] = False
|
|
elif kwargs.get('ufile'):
|
|
image = kwargs.get('ufile').read()
|
|
values['image_1920'] = base64.b64encode(image)
|
|
|
|
if request.uid == user.id: # the controller allows to edit only its own privacy settings; use partner management for other cases
|
|
values['website_published'] = kwargs.get('website_published') == 'True'
|
|
return values
|
|
|
|
@http.route('/profile/user/save', type='http', auth="user", methods=['POST'], website=True)
|
|
def save_edited_profile(self, **kwargs):
|
|
user_id = int(kwargs.get('user_id', 0))
|
|
if user_id and request.env.user.id != user_id and request.env.user._is_admin():
|
|
user = request.env['res.users'].browse(user_id)
|
|
else:
|
|
user = request.env.user
|
|
values = self._profile_edition_preprocess_values(user, **kwargs)
|
|
whitelisted_values = {key: values[key] for key in user.SELF_WRITEABLE_FIELDS if key in values}
|
|
user.write(whitelisted_values)
|
|
if kwargs.get('url_param'):
|
|
return request.redirect("/profile/user/%d?%s" % (user.id, kwargs['url_param']))
|
|
else:
|
|
return request.redirect("/profile/user/%d" % user.id)
|
|
|
|
# Ranks and Badges
|
|
# ---------------------------------------------------
|
|
def _prepare_badges_domain(self, **kwargs):
|
|
"""
|
|
Hook for other modules to restrict the badges showed on profile page, depending of the context
|
|
"""
|
|
domain = [('website_published', '=', True)]
|
|
if 'badge_category' in kwargs:
|
|
domain = expression.AND([[('challenge_ids.challenge_category', '=', kwargs.get('badge_category'))], domain])
|
|
return domain
|
|
|
|
def _prepare_ranks_badges_values(self, **kwargs):
|
|
ranks = []
|
|
if 'badge_category' not in kwargs:
|
|
Rank = request.env['gamification.karma.rank']
|
|
ranks = Rank.sudo().search([], order='karma_min DESC')
|
|
|
|
Badge = request.env['gamification.badge']
|
|
badges = Badge.sudo().search(self._prepare_badges_domain(**kwargs))
|
|
badges = badges.sorted("granted_users_count", reverse=True)
|
|
values = self._prepare_user_values(searches={'badges': True})
|
|
|
|
values.update({
|
|
'ranks': ranks,
|
|
'badges': badges,
|
|
'user': request.env.user,
|
|
})
|
|
return values
|
|
|
|
@http.route('/profile/ranks_badges', type='http', auth="public", website=True, sitemap=True)
|
|
def view_ranks_badges(self, **kwargs):
|
|
values = self._prepare_ranks_badges_values(**kwargs)
|
|
return request.render("website_profile.rank_badge_main", values)
|
|
|
|
# All Users Page
|
|
# ---------------------------------------------------
|
|
def _prepare_all_users_values(self, users):
|
|
user_values = []
|
|
for user in users:
|
|
user_values.append({
|
|
'id': user.id,
|
|
'name': user.name,
|
|
'company_name': user.company_id.name,
|
|
'rank': user.rank_id.name,
|
|
'karma': user.karma,
|
|
'badge_count': len(user.badge_ids),
|
|
'website_published': user.website_published
|
|
})
|
|
return user_values
|
|
|
|
@http.route(['/profile/users',
|
|
'/profile/users/page/<int:page>'], type='http', auth="public", website=True, sitemap=True)
|
|
def view_all_users_page(self, page=1, **kwargs):
|
|
User = request.env['res.users']
|
|
dom = [('karma', '>', 1), ('website_published', '=', True)]
|
|
|
|
# Searches
|
|
search_term = kwargs.get('search')
|
|
group_by = kwargs.get('group_by', False)
|
|
render_values = {
|
|
'search': search_term,
|
|
'group_by': group_by or 'all',
|
|
}
|
|
if search_term:
|
|
dom = expression.AND([['|', ('name', 'ilike', search_term), ('partner_id.commercial_company_name', 'ilike', search_term)], dom])
|
|
|
|
user_count = User.sudo().search_count(dom)
|
|
my_user = request.env.user
|
|
current_user_values = False
|
|
if user_count:
|
|
page_count = math.ceil(user_count / self._users_per_page)
|
|
pager = request.website.pager(url="/profile/users", total=user_count, page=page, step=self._users_per_page,
|
|
scope=page_count if page_count < self._pager_max_pages else self._pager_max_pages)
|
|
|
|
users = User.sudo().search(dom, limit=self._users_per_page, offset=pager['offset'], order='karma DESC')
|
|
user_values = self._prepare_all_users_values(users)
|
|
|
|
# Get karma position for users (only website_published)
|
|
position_domain = [('karma', '>', 1), ('website_published', '=', True)]
|
|
position_map = self._get_position_map(position_domain, users, group_by)
|
|
|
|
max_position = max([user_data['karma_position'] for user_data in position_map.values()], default=1)
|
|
for user in user_values:
|
|
user_data = position_map.get(user['id'], dict())
|
|
user['position'] = user_data.get('karma_position', max_position + 1)
|
|
user['karma_gain'] = user_data.get('karma_gain_total', 0)
|
|
user_values.sort(key=itemgetter('position'))
|
|
|
|
if my_user.website_published and my_user.karma and my_user.id not in users.ids:
|
|
# Need to keep the dom to search only for users that appear in the ranking page
|
|
current_user = User.sudo().search(expression.AND([[('id', '=', my_user.id)], dom]))
|
|
if current_user:
|
|
current_user_values = self._prepare_all_users_values(current_user)[0]
|
|
|
|
user_data = self._get_position_map(position_domain, current_user, group_by).get(current_user.id, {})
|
|
current_user_values['position'] = user_data.get('karma_position', 0)
|
|
current_user_values['karma_gain'] = user_data.get('karma_gain_total', 0)
|
|
|
|
else:
|
|
user_values = []
|
|
pager = {'page_count': 0}
|
|
render_values.update({
|
|
'top3_users': user_values[:3] if not search_term and page == 1 else [],
|
|
'users': user_values,
|
|
'my_user': current_user_values,
|
|
'pager': pager,
|
|
})
|
|
return request.render("website_profile.users_page_main", render_values)
|
|
|
|
def _get_position_map(self, position_domain, users, group_by):
|
|
if group_by:
|
|
position_map = self._get_user_tracking_karma_gain_position(position_domain, users.ids, group_by)
|
|
else:
|
|
position_results = users._get_karma_position(position_domain)
|
|
position_map = dict((user_data['user_id'], dict(user_data)) for user_data in position_results)
|
|
return position_map
|
|
|
|
def _get_user_tracking_karma_gain_position(self, domain, user_ids, group_by):
|
|
""" Helper method computing boundaries to give to _get_tracking_karma_gain_position.
|
|
See that method for more details. """
|
|
to_date = fields.Date.today()
|
|
if group_by == 'week':
|
|
from_date = to_date - relativedelta(weeks=1)
|
|
elif group_by == 'month':
|
|
from_date = to_date - relativedelta(months=1)
|
|
else:
|
|
from_date = None
|
|
results = request.env['res.users'].browse(user_ids)._get_tracking_karma_gain_position(domain, from_date=from_date, to_date=to_date)
|
|
return dict((item['user_id'], dict(item)) for item in results)
|
|
|
|
# User and validation
|
|
# --------------------------------------------------
|
|
|
|
@http.route('/profile/send_validation_email', type='json', auth='user', website=True)
|
|
def send_validation_email(self, **kwargs):
|
|
if request.env.uid != request.website.user_id.id:
|
|
request.env.user._send_profile_validation_email(**kwargs)
|
|
request.session['validation_email_sent'] = True
|
|
return True
|
|
|
|
@http.route('/profile/validate_email', type='http', auth='public', website=True, sitemap=False)
|
|
def validate_email(self, token, user_id, email, **kwargs):
|
|
done = request.env['res.users'].sudo().browse(int(user_id))._process_profile_validation_token(token, email)
|
|
if done:
|
|
request.session['validation_email_done'] = True
|
|
url = kwargs.get('redirect_url', '/')
|
|
return request.redirect(url)
|
|
|
|
@http.route('/profile/validate_email/close', type='json', auth='public', website=True)
|
|
def validate_email_done(self, **kwargs):
|
|
request.session['validation_email_done'] = False
|
|
return True
|