Purpose
=======
Allow non administrators to use relational properties.
Standard internal users can not read ir.model. Because of that we
created a component that simulate the behavior of a many2one, but
that call custom public method that check which model the user can
access.
Task-2852259
Part-of: odoo/odoo#95184
Purpose
=======
For security reason access on ir.model is not granted to internal suers. Due
to this constraint a component exists in spreadsheet to be able to select
the models for which we have a read access on the records of this model.
This is required for the properties fields feature, hence moving its code
to web.
Some renaming is performed to make it generic. This generates some changes
in other addons, notably some class renaming.el.
Task-2852259
Part-of: odoo/odoo#95184
Pillow 9.1 deprecates most if not all toplevel Image constants:
https://pillow.readthedocs.io/en/stable/releasenotes/9.1.0.html#constants
These constants have been moved to thematic enum classes (e.g. all the
resampling constants in `PIL.Image.Resampling`).
This triggers warnings in Odoo, and the removal delay is quite short
(slated for Pillow 10, release planned mid 2023). Pillow 9.1 is also
already in Debian Bookworm (current testing).
Fix by shimming at the import level: if the enums are available import
them into the local namespace, otherwise alias `PIL.Image` itself as
to the enum.
closesodoo/odoo#96799
X-original-commit: 7be04d31bad078681ef0a2919234c11840a2e8e2
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Doing a `search_count` on millions of records could be very slow. On
large databases, the pager counter "1-80 / 8000000" could take several
seconds just to get the total number of records.
This commit limits the counter in list and kanban views to 10k records,
and shows "1-80 / 10000+" if the limit is reached. If you click on
10000+, it updates with the real count (or if you go up to 10000 with
pager or input value).
The search_count on res.partner of odoo.com takes 4s. With this patch,
it will take ~20ms.
Task 2761165
closesodoo/odoo#95642
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Damien Bouvy <dbo@odoo.com>
Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closesodoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.
e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`
The goal of this revision is to change that so it sends the list of all fields
only once.
In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
- `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
- `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`
The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
As it no longer contains the fields,
the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
which is a dict with as key the model name and as values
the model fields description. It contains the fields description
for all models implied in the view:
the model of the main view and the model of all one2many and many2many fields.
With this change, the fields description will only be sent once by model
implied in the view.
In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.
- one2many and many2many fields views are passed directly in the main view
architecture rather than being put in the `views` key
of the field description.
This is actually easier to treat by the web client,
and this will allow in a future work to cache an entire view in one block
of text rather than having to combine multiple cached blocks of text
to return one view.
- one2many and many2many fields which do not have directly embedded views
have their views directly injected in the architecture,
so the web client doesn't have to do RPC calls to `load_views`
for each one2many and many2many fields not having embedded views.
For instance, this allow to reduce the number of RPC calls to `load_views`
from 8 to 1 when loading the form of `product.product`.
Currently, this behavior is limited to 1 level deep but we consider making it
go all the way down in future works. We did not do it for the moment because
in certain cases it rises the processing time and the size (bytes) too much.
e.g. the sale.order view can be 5 levels deep,
meaning you can reach 4 dialogs on top the main view.
```
sale.order form > order_line > sale.order.line form > invoice_lines >
account.move.line form > asset_ids > account.asset form >
depreciation_move_ids > account.move form.
```
This will also benefit in future works to cache an entire view in one block
of text rather to having to combine multiple cached block of text
to get one view.
- `fields_view_get` becomes `get_view`.
As it no longer returns the fields description,
keeping the `fields` in the name `fields_view_get` no longer makes sense.
Hence removing `fields` from the method name, it becomes `view_get`.
As it gets renamed anyway, we take the opportunity to rename it `get_view`,
which is more in line with the general getter/setter guidelines
in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
This is not mandatory, there is no technical reason to rename `load_views` as
it practically sends the same info as before,
the view architectures and their fields description. Just in another way.
We just take the opportunity of this pull request to suggest a cleaner API:
`_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
`_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
in `_get_view` and `get_view`.
The rationale is that submenu was already no longer used (deprecated)
and the mobile options is introduced.
The mobile options is necessary to tell the server to send the mobile views
for x2many fields (kanban instead of tree).
Instead of adding a new argument each time we add a new option to
`fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
view information. Now, `get_view` returns a tuple with the view architecture
as an `etree` node, and the view as a browse record. The rationale is that all
overrides of `_fields_view_get` were about modifying the arch only
(e.g. changing the address format/re-organizing the address related field
nodes of the partner according to the company country).
To do so, all these overrides were doing `etree.fromstring` to parse the arch
which was sent in text to convert it to an `etree`,
then operations were done on the `etree`,
and then `etree.tostring` was called to convert back the arch to string.
With this change of signature to send the arch as an `etree`,
all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
has been performed in `get_view`:
- `fields` is removed, as explained above,
- `view_id` is renamed `id`,
- `name` is removed, it was unused by the web client,
- `type` is removed, it was unused by the web client,
- `field_parent` is removed, it was unused by the web client,
- `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
(now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
`fields_view_get`, `_fields_view_get` and `load_views` are provided,
with deprecation warnings in them.
- The web client could cache the model fields description
(as it already caches the views),
so it doesn't need to fetch them again if it asks for another view of a model
for which he already has the fields description.
If we do so, `get_views` could return only the list of models used by
the views, without the fields description as of now,
and the web client would then call `fields_get` independently only for
the models for which it doesn't have yet the fields description.
This would avoid the server to return the fields description
and to call `fields_get`, which is costly, for each `get_views`,
therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
This is already done for qweb views, it's not done for back-end views.
Therefore the postprocessing of the views is performed for each `get_views`,
which is costly, while the view architecture doesn't change for users
belonging to the same groups, according to the groups implied by the view.
This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.
Part-of: odoo/odoo#87522
Field name on sponsor are not required. So when we display the image of a
sponsor without field-name it will use the rec_name 'name' that is not set
and so False.
It will crash during rendering of the widget image that use it.
Now we force to use a field required 'partner_name' to avoid the crash.
By default, when you confirm the sale_order from the backend, we copy
the partner name in field name.
Because we use a class avatar to have a max-width of 96, we don't need
to use a image_512, image_128 is enough in most of case.
The max-width option on widget image has no effect on rendering.
In other case where the rec_name is Falsy and we don't provide other name to
the widget, we now use a default 'name' value to avoid a crash:
closesodoo/odoo#88296
Attributeerror: 'bool' object has no attribute 'replace'
X-original-commit: b054dfc715afb225eb032a722ec58e6888b1b905
Signed-off-by: Jérémy Kersten <jke@odoo.com>
The odoo.addons.web.controllers.main python module have been splitted
over multiple files on the basis 1 controller = 1 file. In this work we
adapt all modules to use the new imports.
A non-exhaustive list of where stuff have been moved:
* main.Home --> home.Home
* main.Session --> session.Session
* main.WebClient --> webclient.WebClient
* main.clean_action --> action.clean_action
* main.ensure_db --> home.ensure_db
The complete list is accessible in odoo.addons.web.controllers.main.
closesodoo/odoo#87571
Related: odoo/enterprise#25746
Signed-off-by: Raphael Collet <rco@odoo.com>
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.
The report rendering and call `ir.qweb` instead of `ir.ui.view`.
Part-of: odoo/odoo#85110
Install website, create a custom web page, we'll call it page_1. Ensure
you are not in debug mode (go to /web/health?debug=0 to disable it).
Open the web page enabling the debug-mode /page_1?debug=1, the page
opens but the debug mode is disabled.
Because web pages are served using another routing mechanism than
controlers we have to ensure we load the debug query-string in those
mechanisms too.
closesodoo/odoo#85340
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
When working on qweb fields it is difficult to find some custom override as
they are never in the right file. Finding them is always a bit of random pick.
Task-2607416
Part-of: odoo/odoo#74171
This commit is the 12th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
The web module is twofold, on one side there are many controllers: /,
/web, /web/login, /web/database/selector, /web/dataset/call_kw, etc, on
the other side there is `session_info`: the method responsible to create
the web client's environ.
This module is kinda an exception as it is (with base) a server wide
module. In the case of the HTTP framework, it means that the controllers
of web are always accessible, i.e. going to / or /web/login will never
return a 404 Not Found even if the user is not connected to a database.
This is both a blessing and a curse. It is a blessing because the
controllers are always accessible it means that a new users can freely
access those routes. It is a curse because *any* user can access them,
even user who don't have a session yet thus who are not connected to a
database yet. From a developer standpoint, we have to put extra care to
correct serve users with and without a database. An example is the
/web/login route, the login/password pair is stored in a database,
without database it is impossible to validate a user login but users can
still access this route without db.
To solve this problem, there is the `ensure_db` function. This function
attempts to find a database using various sources (?db= query-string,
session db, mono db) and to save it on the user session. In case no db
is found, the user is redirected to the database selector. In a way,
this function grants a database to the user in a seamingly experience.
In a way, this function brings a welcome differentiation between
`auth='none'` with a database and `auth='none'` without a database. Such
differentiation only matters for the server wide modules as "regular"
module controllers are only accessible via the ir.http routing map, i.e.
it is not possible to declare a nodb controller outside of server wide
modules.
An important changement is the `session.authenticate` method, before it
was possible to call the method when the cursor was not yet initialized,
authenticate would open a cursor against the given database, setup a
registry and an environment and ultimately save everything on the
current request. Because the cursor is now greedily created, it is no
more possible to update the request environment when authenticating on
another database.
PR: odoo#78857
Task: 2571224
When changing the background image with studio the image does not update.
It works in debug=assets mode.
This is due to the fact that the caching system is not aware of the presence of a possible background image
opw-2696786
closesodoo/odoo#83374
X-original-commit: 184029e3e1e8907e25dd712dd87afd65885695bb
Related: odoo/enterprise#23744
Signed-off-by: Achraf <abz@odoo.com>
Avoid to base64 encode, then decode to process assets and images for a ~25% speed improvement.
Change image processing tool to work on images, rather than base64 encoded strings.
Performance is ~25% faster on assets & images:
/web/assets/...frontend.min.css: 13ms to 7ms, base64 enc/dec: 2 -> 0
/web/image/XML_ID: 10ms to 8ms, base64 enc/dec: 3 -> 0
/web/image/res.users/2/avatar_128: 40ms to 20ms, base64 enc/dec: 6 -> 2
closesodoo/odoo#82851
Related: odoo/enterprise#23537
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
Current behavior :
PDF Preview in document layout settings doesn't match the downloaded one (colors of div with the total and the columns displayed)
Steps to reproduce :
- Go to Settings
- 'Configure Document Layout' under Companies
- Set layout to Boxed
- Change the colors
- Download the PDF Preview
Reason :
- The columns for the unit price and taxes had the 'd-none' class so they weren't visible when printing the pdf preview. To keep coherence, if we see them in the preview we must see them in the pdf.
- Colors, especially the total div with a 'Boxed' layout, weren't taken into account for some elements in the preview. This is due to a character sanitized in the CSS, specifically '>' which becomes '>' for the '.row > div > table' selector.
OPW-2716089
closesodoo/odoo#81932
X-original-commit: 158c0ae425b3be446d81c3a6f4387b2d9426d10e
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Claude Thibault (thcl) <thcl@odoo.com>
As JS is not taking the properties in the order they are written,
the order in the rendering was always following the property name
sorting order (=id).
Previous to this commit:
- The companies were ordered by their id in the company switcher as
JS is not tacking the object properties order into account.
After this commit:
- The companies will be sorted by their sequence prior to be used in the
rendering.
task-2722235
closesodoo/odoo#81893
X-original-commit: 39c678a1ccb50d3a1871a4049a8df26427b27a3c
Signed-off-by: Laurent Stukkens (ltu) <ltu@odoo.com>
Purpose
=======
Avoid counting requests from social bots (twitter, facebook, linkedin...)
when tracking a link.
Specifications
=============
Social media platforms have a specific user agent in the HTTP headers that
can be used to detect them and to not increment the click count in that case.
Task-2578902
closesodoo/odoo#78806
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Following-up to previous commit, this one makes the
`get_translations_for_webclient()` more defensive with regards to the
absence of a `lang` key in the context, rather than relying on callers
to protect it. The rest of the logic was already fine with this.
This allows simplification of the `session_info` logic and removal of
the conditional for `translation_hash`.
Further cleanups in `session_info()`:
- removed a duplicate calls to `session.get_context()`
- removed duplicated resolutions of `request.session.uid`
closesodoo/odoo#79182
X-original-commit: 3eca1a2e58b8c644c0701d33d0c9d0330bdc234f
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Signed-off-by: Romeo Fragomeli (rfr) <rfr@odoo.com>
Since [1] and [2], the mobile app gets this error when trying to login
on v15, while it was working fine in v14 with TOTP enabled.
The 'authenticate' JSON-RPC route tries to authenticate the user and
then call `session_info()`. As no UID is defined, some methods in
`session_info()` raise an exception and an unexpected error is sent:
* `_is_public()` -> "Expected singleton: res.users()"
* `get_web_translations_hash()` -> "lang"
In this fix, this exception is avoided and the proper result is sent,
allowing the authentication process to continue.
Steps to reproduce:
* Try to connect to an account with TOTP on the mobile app (v15+) => BUG
Refs:
[1] odoo/odoo@80d74e7ee0
[2] odoo/odoo@401fc7efe9
X-original-commit: 65dca67ecdcc2228d90781a9f5ccd99f290ada6c
Part-of: odoo/odoo#79182
The show_effect was either "True" or false which is inconsistent.
This was caused by the fact that the string value came from the backend
and the boolean came from the default value of get_param.
By wrapping the result of get_param in a bool constructor, we make sure
the type is always consistent.
closesodoo/odoo#78742
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
In https://github.com/odoo/odoo/pull/78652 a fix was made in the layout
designer guaranteeing that the company_details field followed the correct
address format set by the company. However, this fix didn't take into account
all company data, making some `format_address`es raise a `KeyError`. This PR
fixes that by using the company_date defined in `_display_address`.
Fixes https://github.com/odoo/odoo/issues/78942closesodoo/odoo#79097
X-original-commit: 3cdc770c4966cef88362f8119a96de5501d21b18
Signed-off-by: Leonardo Pavan Rocha <lpr@odoo.com>
In task-2355704 changes were made to the pdf layout designer to allow more
flexibility when setting company data. However, for the default values of both
company_details and report_footer, the data wasn't being escaped, therefore
offering security risks. Also, they didn't take into account the address_format
when computing the default value. This PR implements Markup usage in the html
fields and fixes _default_company_details to use the set address_format.
closesodoo/odoo#78725
X-original-commit: 83aeb8fbc3b2a4b39123f756232e34c1ce603299
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
With the introduction of the guest feature, a lot of fields are now
accessed through a controller so that they can be available to guests,
with additional checking in the controllers.
Avatar is one such field, however the extra checks were too restrictive:
we only allowed people to see the avatar if the user whose avatar was
requested was also a member of the channel. This breaks the avatar on
previous messages if a user leaves a channel.
This commit relaxes this restriction for internal users (they can now
see all avatars through this route), and adds a fallback to the avatar
placeholder for people who do not have acces (ie: guests) so that the
UI doesn't look broken.
closesodoo/odoo#76341
X-original-commit: b849fd426806332c1ecc1bbba7b568384a021c65
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
* = crm_livechat, hr, hr_holidays, im_livechat, mail_bot, purchase, sms,
snailmail, survey, test_discuss_full, test_mail, web_editor, website,
website_livechat
- Create new model `mail.guest` for guests.
- Rewrite some RPCs to target routes rather than model methods so that
guests are able to use them.
- Patch JS and python models to support guests.
- Create a stand-alone page and boot the channel in it.
task-2494829
closesodoo/odoo#75496
Related: odoo/enterprise#20417
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
The value returned by the search_read in read_progress_bar when passing
a m2m field is a list of ids, which then read_progress_bar tries to use
in a dictionary, list is not a hashable type thus it crashes.
With this commit we convert the group_by_value from list to tuple,
which is hashable, if we're dealing with a many2many field.
task-2508608
Part-of: odoo/odoo#74985
The algorithm extracting primary and secondary colors from
an image automatically mitigates too bright colors. A color
component is considered as too bright if its value is above
an arbitrary value. We re-use this algorithm in the website
configurator to extract colors from logo and to generate a
palette. In this particular case we don't want the mitigation
to be applied. We therefore added the mitigation threshold as
a parameter of 'extract_image_primary_secondary_colors' and
avoided any mitigation by setting it to 255 in our case.
task-2602521
closesodoo/odoo#75255
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes
closesodoo/odoo#68299
Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
* Remove AST in favor of pure Pyhon. This should make it easier for
developers to understand and create new directives because they do not
need to know AST.
* Remove `t-call-options` as it has been merged into `t-options` for more
consistency. Support for t-call-options is retained.
* Use generators for lists. This increases performances as the rendering
can be sent directly without having to wait for the creation of the
entire list.
* Optimize expressions runtime computation by pre-computing the static
parts.
Example:
'<' + 'div' + '>' + '<' + dynamic_value + '>'
Now compiles as:
'<div><' + dynamic_value + '>'
The frontend code was still using the legacy RainbowMan, with the wowl
webclient, we rewrote this RainbowMan and started using that one
instead, but because the frontend code had no access to the wowl
environment, the legacy version was still used in the frontend. Since
the frontend now has access to the wowl environment, the old code can be
removed and the calls can go through the new effect service.
closesodoo/odoo#74148
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Fetching all the groups from the DB causes MemoryError on some DBs
Example Accounting > Accounting > Journals > Miscellaneous menu:
```
Traceback (most recent call last):
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 176, in crawl_menu
self.mock_action(action_vals)
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 267, in mock_action
mock_method(model, view, fields_list, domain, group_by)
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 380, in mock_view_tree
self.mock_web_read_group(model, view, domain, group_by, fields_list, limit_group=5)
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 425, in mock_web_read_group
data = model.web_read_group(domain, fields_list, group_by, limit=limit)["groups"]
File "/home/odoo/src/odoo/14.0/addons/web/models/models.py", line 96, in web_read_group
all_groups = self.read_group(domain, ['display_name'], groupby, lazy=True)
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2248, in read_group
result = self._read_group_raw(domain, fields, groupby, offset=offset, limit=limit, orderby=orderby, lazy=lazy)
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in _read_group_raw
result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in <listcomp>
result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
MemoryError
```
This issue was observed during the upgrade requests upg-18830, target
14.0; and upg-61934 (legacy), target 13.0
The issue can be reproduced on a clean DB with just account_accountant
installed and ~2 millions account moves on a Misc journal.
closesodoo/odoo#73855
X-original-commit: a9b3d9cf9bfd2bf4c9b0904059f1606ad6e04ee4
Signed-off-by: Christophe Simonis <chs@odoo.com>
The method is used to get progress per column in kanban view
(green-yellow-red-red lines in Project, CRM etc). There are two main
usages:
1. get statistics for ``kanban_state`` (red/green circles)
2. get statistics for ``activity_state`` (colored clock icon for overdue/today/planned)
Before this commit all cases were handled by calling search_read and then
counting records per group in a python script. This is very inefficient,
especially for ``activity_state``.
This new implementation relies on ``read_group`` when possible, i.e.,
when both grouping fields (kanban column and progressbar field) are
stored (case 1). It then falls back on a naive implementation inside
``_read_progress_bar``. Cases like 2 above can be addressed by
overriding ``_read_progress_bar``.
We also added some minimal test to ensure that we don't break anything.
1. Performance test on 60 K project.task records (kanban_state):
With a filter for 6 records:
```
| measurement | before | after |
|--------------------+--------+-------|
| number of queries | 8 | 5 |
| query time, ms | 11 | 7 |
| remaining time, ms | 21 | 9 |
```
All records:
```
| measurement | before | after |
|--------------------+--------+-------|
| number of queries | 67 | 5 |
| query time, ms | 300 | 55 |
| remaining time, ms | 1780 | 12 |
```
---
opw-2346901
task-1915411
X-original-commit: 153621bdbab94a2a94a5bbfcabb4111cbc5970d8
Co-authored-by: Raphael Collet <rco@odoo.com>
This commit implements improvements in the layout designer, such as addition of a new custom background, adds custom report footer and company details.
task-2355704
closesodoo/odoo#66860
Related: odoo/upgrade#2275
Signed-off-by: Arnaud Joset <arj-odoo@users.noreply.github.com>
This commit is the first phase of the conversion of the web/ JS
codebase to the owl framework. The impact of this commit is two-fold.
First, it rewrites the framework part of web with a new system of
services and registries. Services allow to execute code (e.g. do rpcs,
setup things) before launching the application. They can also expose
an API to be used by other parts of the application (e.g. a notification
service would expose a function to display notifications). Services are
often a good extension point for external modules that want to execute
code at webclient startup. Registries offer another way to extend the
application. They provide well designed extension points to add
elements/behaviors from the outside (for instance, to add a systray item,
an error handler...).
Second, this commit initiates the conversion of the webclient to owl
with a top-down approach, around those notions of services and registries.
The root of the web application is now an owl application. Among others,
the WebClient, ActionManager, Navbar, UserMenu, DebugManager, Dialogs,
services (e.g. notification, ajax...) have been converted to the new
framework/architecture.
Legacy views and client actions are still supported (and used). They
will be converted in the next months, and at some point, the support
will be dropped.
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Géry Debongnie <ged@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Simon Genin (ges) <ges@odoo.com>
Co-authored-by: Francois (fge) <fge@odoo.com>
Co-authored-by: Michael Mattiello (mcm) <mcm@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais (lpe) <lpe@odoo.com>
Co-authored-by: Jorge Pinna Puissant <jpp@odoo.com>
Also make this a configurable value, from the server.
This is the limit of the SAAS servers.
closesodoo/odoo#72143
X-original-commit: bcd1c8aade6123dd20d7aebaae8a0f204f258604
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Signed-off-by: William André (wan) <wan@odoo.com>
Replace text fields to html fields as we have our own 'OdooEditor'.
Indeed, it gives more options to users in the way they format their
content without weighting too much on the UI
(tools appear on demand and not by default).
In this commit we replace report_footer and report_header field of
"base.document.layout, res.config.settings" model as
they are related fields of res.company models field
report_footer, report_header.
Task Id: 2499504
X-original-commit: 4c474d0d2055efe81fdc21502aa5957fe80af7ef
The profiling tools can be useful to profile a test of some execution
point but this is not convenient to identify a problem on a running
instance.
With this commit, an option available in the debug menu allows to add a
flag on the user sessions to enable profiling of all requests. Each
request will be saved in a different 'ir.profile' entry, but will be
grouped under the same session.
The profiling can be activated on all sessions, even for a public user,
but only if profiling is enabled on the database globally.
This commits also adds a speedscope view to visualize saved results in
the web client.
closesodoo/odoo#66590
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
* IMPACTED VERSIONS
12.0+
* HOW TO REPRODUCE
locale : Locale is en_US (or other SUNDAY based)
view: CRM - My Pipeline - Kanban view
groupBy: date_deadline:week (Expected closing)
records: one record with a planned activity, on date_deadline = 2021-05-02 (SUNDAY)
one record with no planned activity, on date_deadline = 2021-05-09 (SUNDAY)
remark: don't keep any other record in MAY for better visibility
* PROBLEM
The progressbar of the week containing 2021-05-09 displays information about the record
from the week containing 2021-05-02
* CAUSE
1. PostgreSQL `date_trunc` function follows ISO8601 which essentially means that
the start of a WEEK is always MONDAY. There is no argument to change this.
2. _read_group_format_result
https://github.com/odoo/odoo/blob/27da86a138089c1838e4b94f8a6976995b9c1fff/odoo/models.py#L2210-L2219
- Computes a label for a group of records.
- Follows the locale for the label of the week, based on a date which is
always a MONDAY because of `date_trunc`.
3. read_progress_bar
https://github.com/odoo/odoo/blob/88957afca09662af7eaa19df1e40b3699e45e79e/addons/web/models/models.py#L167-L175
- Associates a group label to a record.
- Follows the locale for the label of the week, based on the date of a record
which can be any day of the week. If the record is related to a SUNDAY and
SUNDAY is the first day of the week, it would have been in a group with a
different label in (2.) than in (3.) prior to this change.
* FIX
In 3., before associating a label to a record, we truncate the date to the
ISO start of the period, so that the label is determined for a record in the
same conditions than in 2. The locale is still used to get language-dependent
outputs with babel, but the grouping will always follows ISO8601 (date_trunc).
* TEST
Added a test for this problem case
TASK-ID : 2517848
closesodoo/odoo#70498
X-original-commit: 4560925b26fa79740b9618fd9241d3517b64f43f
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Previously, lazy-loading xml templates was only possible by fetching the
xml file directly, this meant that it was impossible to lazy-load an
entire bundle's templates with a single request. Additionally,
requesting the xml files directly meant that no inheritance was applied,
causing the need for a separate inheritance system using t-jquery on the
client-side.
This commit alters the /web/webclient/qweb route so that it now takes a
bundle id, meaning that it is now possible to lazy-load the xml from
arbitrary bundles.
task-2497943
closesodoo/odoo#70084
Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
Add a big fat warning when the qweb compiler finds a `t-raw`.
`t-esc` should now be used everywhere, the use-case for `t-raw` should
be handled by converting the corresponding values to `Markup`
objects. Even though it's convenient, this constructor *should never
be made available in the qweb rendering context* (maybe that should be
checked for explicitely?).
Replace `werkzeug.escape` by `markupsafe.escape` in
`odoo.tools.html_escape`, this means the output of `html_escape` is
markup-safe.
Updated qweb to work correctly with escaping and `Markup`, amongst
other things QWeb bodies should be markup-safe internally (so that a
`t-set` value can be fed into a `t-esc`). See at the bottom for the
attributes handling as it's a bit complicated.
`to_text` needed updating: `markupsafe.Markup` is a subclass of `str`,
but `str` is not a passthrough for strings. So `Markup` instances
going through would be converted to normal `str`, losing their safety
flag. Since qweb internally uses `to_text` on pretty much
everything (in order to handle None / False), this would then cause
almost every `Markup` to get mistakenly double-escaped.
Also mark a bunch of APIs as markup-safe by default
* html_sanitize output.
* HTML fields content, sanitization is applied on intake (so stripped
by the trip through the database) and if the field is unsanitised
the injection is very much intentional, probably. Note: this
includes automatically decoding bytes as a number of default values
& computes yield bytes, which Markup will happily accept... by
repr-ing them which is useless. This is hard to notice without `-b`.
* Script-safe json, it's rather the point (though it uses a
non-standard escaping scheme).
* Note that `nl2br`, kinda: it should work correctly whether or not
the input is markup-safe, this means we should not need to escape
values fed to `nl2br`, but it doesn't hurt either.
Update some qweb field serialisations to mark their output as
markup-safe when necessary (e.g. monetary, barcode,
contact). Otherwise either using proper escaping internally or doing
nothing should do the trick.
Also update qweb to return markup-safe bytes: we want qweb to return
markup-safe contents as a common use-case is to render something with
one template, and inject its content in an other one (with Python code
inbetween, as `t-call` works a bit differently and does not go through
the external rendering interface).
However qweb returns `bytes` while `Markup` extends `str`. After a
quick experiment with changing qweb rendering to return `str` (rather
unmitigated failure I fear), it looks like the safest tack is to add a
somewhat similar bytes-based type, which decodes to a `Markup` but
keeps to bytes semantics.
For debugging and convenience reasons, MarkupSafeBytes does *not*
stringify and raises an error instead (`__repr__` works fine). This is
to avoid implicit stringifications which do the wrong thing (namely
create a string `"b'foo'"`).
Also add some configuration around BytesWarning (which still has to be
enabled at the interpreter level via `-b`, there's no way to enable it
programmatically smh), and monkeypatch `showwarning` to show warning
tracebacks, as it's common for warnings to be triggered in the bowels
of the application, and hard to relate to business logic without the
complete traceback.
`t-out`
=======
`t-esc` is a bit confusing for the new behaviour of "maybe escape
maybe not", so add a `t-out` alias with the same behaviour.
Unlike `t-raw`, `t-esc` is only soft-deprecated for now: there are
thousands of instances, so editing all the templates is not
great. Eventually we'll add a `ci/style` to prevent addition of new
ones, and eventually we might do a bulk-replace and hard-deprecate.
Attributes handling
===================
There are a few issues with respect to attributes. The first issue is
that markup-safe content is not necessarily attributes-safe
e.g. markup-safe content can contain unescaped `<` or double-quotes
while attributes can not. So we must forcefully escape the input, even
if it's supposedly markup-safe already.
This causes a problem for script-safe JSON: it's markup-safe but
really does its own thing. So instead of escaping it up-front and
wrapping it in Markup, make script-safe JSON its own type which
applies JSON-escaping *during the `__html__` call.
This way if a script-safe JSON object goes through `markupsafe.escape`
we'll apply script-safe escaping, otherwise it'll be treated as a
regular strings and eventually escaped the normal way.
A second issue was the processing of format-valued
attributes (`t-attf`): literal segments should always be markup-safe,
while non-literal may or may not be. This turns out to be an issue if
the non-literal segment *is* markup-safe: in that case when the
literal and non-literal segments get concatenated the literal segments
will get escaped, then attributes serialization will escape
them *again* leading to doubly-escaped content in attributes.
The most visible instance of this was the `snippet_options` template,
specifically:
<t t-set="so_content_addition_selector" t-translation="off">blockquote, ...</t>
<div id="so_content_addition"
t-att-data-selector="so_content_addition_selector"
t-attf-data-drop-near="p, h1, h2, h3, .row > div > img, #{so_content_addition_selector}"
data-drop-in=".content, nav"/>
Here `so_content_addition_selector` is a qweb body therefore
markup-safe, When concatenated with the literal part of
`t-atff-data-drop-near` it would cause the HTML-escaping of that
yielding a new Markup object. Normal attributes processing would then
strip the markup flag (using `str()`) and escape it again, leading to
doubly-escaped literals.
The original hack around was to unescape() `Markup` content before
stringifying it and escaping it again, in the attribute serialization
method (`_append_attributes`).
That's pretty disgusting, after some more consideration & testing it
looks like a much better and safer fix is to ensure the
expression (non-literal) segments of format strings always result in
`str`, never `Markup`, which is easy enough: just all `str()` on the
output of strexpr. We could also have concatenated all the bits using
`''.join` instead of repeated concatenation (`+`).
Also add a check on the type of the format string for safety, I think
it should always be a proper str and the bytes thing is only when
running in py2 (where lxml uses bytestrings as a space optimization
for ascii-only values) but it should not hurt too much to perform a
single typecheck assertion on the value... instead of performing one
per literal segment.
Note: we may need to implement unescape anyway, because it's still
possible to get double-escaping with the current scheme: given an
explicitly escape-ed `foo` and `t-att-foo="foo"`, `foo` will be
re-escaped.
fixup! [CHG] core, web: deprecate t-raw
On website app installation and on new website creation a configurator is launched.
The purpose of this configurator is to generate a website that meet the user's needs.
The configurator is composed of 4 steps:
1) Business description: the user is asked to describe its need with its website purpose (dropdown), its industry (autocomplete search) and its objective (dropdown).
2) Logo and palette selection: the user must select a color palette for its website. He can also upload its logo. In this case color palettes recommendations are generated based on the logo's colors.
3) Features selection: the user select the pages and applications he needs.
4) Theme selection: three themes are recommended to the user based on its industry. This screen display a preview of these three themes.
task-id: 2451965
ENT PR: odoo/enterprise#16949
UPG PR: odoo/upgrade#2316closesodoo/odoo#67537
Signed-off-by: Sébastien Mottet <smottet@users.noreply.github.com>
This commit changes the way assets are declared in Odoo modules.
Before: assets were declared in template files. Template bundles were
generated from primary templates, so technically any qweb template could
have been called as an asset bundle, with the 't-call-assets' directive.
Being standard qweb templates, they had access to standard HTML tags
(script, link, with or without raw scripts or style definition), qweb
directives (t-call, t-raw, etc.) and could be inherited by other
templates.
Now: assets are defined in the module's manifest and generated by the
't-call-assets' directive.
More information on the new system can be found on the updated user
documentation (see the "JavaScript Reference" section).
Task: 2352566
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
This commit's primary objective is to ensure that the timesheet uom is working in accordance
with the company settings in a multi company environment.
Prior to this commit:
- The timesheet preferences sent to the front end were always those of the default
company of the user.
- The timesheet related widgets initialisation process (adding the correct ones in
the fieldRegistry) was performed before the front end treatment of cids and coockies
which prevented applying the front end selected company settings.
After this commit:
- A dictionnary is used in the session in order to structure the companies info.
- The company timesheet preferences are sent to the front end through the company dict.
- The uom info is sent to the front end through the session.
- The timesheet uom is now managed from the frontend and is now in sync with the settings.
- Timesheet widgets are initialised during the AbstractWebClient init and are in sync with
the multicompany front end settings
task-2168337
Closes: #66551
Done for res.partner.bank, res.company and board models. Some override may
have been ignored because heavily linked to business code (like company
in stock).
See merge commit for more details.
Task ID-2330149
COM PR odoo/odoo#61246
ENT PR odoo/enterprise#14561
Pillow 7.0 changed the default resampling filter from NEAREST to
BICUBIC. Because the logo parser resizes the image before sampling it
and doesn't specify a filter, its behavior changes depending on the
version of Pillow, even if the input image is the same. And there's a
test depending on its output with a fixed image.
Explicitly specify the old default as that's what's in the test.
closesodoo/odoo#60121
X-original-commit: 3dbc0711b65ae47a230bcd8b3348962127bcaf14
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Behavior before the fix:
Uploading a grayscale image with transparency in the company logo field
of the document layout causes a crash:
```
File "/data/build/odoo/addons/web/models/base_document_layout.py", line 89, in _compute_logo_colors
wizard.logo_primary_color, wizard.logo_secondary_color = wizard_for_image._parse_logo_colors()
File "/data/build/odoo/addons/web/models/base_document_layout.py", line 191, in _parse_logo_colors
color[1][2] > white_threshold) and color[1][3] > 0:
Exception
...
IndexError: tuple index out of range
```
To reproduce, use the `logo_ci.png` image included in the unit test
data and load it in the document layout (under General Settings).
The system does not detect that the image does not have color
information and attempts to read the R,G,B values (but the image only
has 2 values per pixel: the grayscale value and the alpha)
Versions affected: 13, 14
Behavior after the fix:
The image is loaded correctly (arguably, there is not a lot of interest
in detecting the primary color of a grayscale image, but the script
still returns a (gray) value which is preferrable to a traceback)
opw-2352394
closesodoo/odoo#59498
X-original-commit: 6d6e48f34f850fa533ce00532ada0e3464c8144b
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Signed-off-by: Nicolas Galler <nicocrm@users.noreply.github.com>
Forgotten TODO
Instead of checking in _for_xml_id (where 99% of calls are static),
verify on the few calls that accept an arbitrary argument
closesodoo/odoo#57117
X-original-commit: efc0263b2574fa355ea794514eb3285d8c449260
Related: odoo/enterprise#12975
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
The document layout preview is a complete defferent simplified template
with its own css that replicates at best the different styles.
It does not have the external layout features and lack of fidelity.
The new preview actually use the real documents templates and put the
result in an iframe. It now has a high fidelity, though not perfect.
The goal is for a better onboarding, where clients see easely how
documents will look if they had an app to generate them. Of course, the
data on the document is a false invoice.
Refactor all this from base to web.
Task ID 2304177
closesodoo/odoo#56995
X-original-commit: c121a246f16899735306266a3a12b526e08e7620
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
The new "image_url" widget logic provides the ability to provide static image
links in Char fields instead of duplicating the image file in database (+ filestore)
through an Image field (& ir_attachment).
This commits implements the frontend qweb logic for this widget, following the logic
of the existing "image" qweb widget.
PURPOSE
Before this commit, when you used the 'image' widget when the record name was
``..`` (double dots) it couldn't retrieve the image since the double dots was
decoded as the parent directory shortcut. After this commit, the double dots
will be replaced by -- with the downside to loose the true record name.
SPECIFICATIONS
Replace the .. (double dots) with -- in the filename.
Using .. may seem like a strange use case but if you actually try to find
a partner called ".." in a production database, you might be surprised ...
LINKS
Task ID-2241513 (eLearning onboarding and testing)
PR #55567
X-original-commit: dcb7a924c36cd3f3aeef891ddd8830bd66322025
std-odoo