In order to manage different branches, the usability of the company
selector is being improved
* display in a hierarchic manner
* when selecting a company, select all the available children with it
task-3371677
Part-of: odoo/odoo#125642
When a domain field value is edited via the debug textarea, no
search_count is done for performance reasons. A single exception is done
when saving the record. Then we check the validity of the domain created
in the debug textarea in order to avoid to save an invalid domain in db
(and get tracebacks,..). The problem is that a search_count can take a
very long time to be executed if the domain is valid. Here we introduce
a route /web/domain/validate in order to quickly check the validity of a
domain and use it in domain field in order to fix the above mentionned
performance issue. Note that the search_count is still done if it useful
but does not have to be waited anymore.
X-original-commit: 40288221c39ff8fba41cd6ac231ab33600dc0cd4
Part-of: odoo/odoo#128913
Co-authored-by: Oliver Dony <odo@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.
Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.
The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.
Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.
The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.
We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.
Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.
Note that this poc is inspired from the long term cache but not all
use case where applie yet.
Part-of: odoo/odoo#119813
1. move cache to _get_asset_paths
The `_get_asset_content` cache has many cache key that are related to a
posprocessing of the `_get_asset_paths` result, the heavy part of this
method. Moving the cache to _get_asset_content will have the benefit
to create less duplicates entries in the ormcache as well as less cache
miss.
To simplify even further, the css and js parameters are removed since
they only filter the output of get_paths, the heavy part of globing the
file will be done before that. Anyway, they are both true when called
from _get_asset_content, and the only other call, in
`_get_related_bundle` don't really need to filter them since it is not
a critical part regarding performance, and the funtional result will
stay the same.
The initial orm cache key was using `_get_template_cache_keys`, a little
overkill and possibly creating duplicates entries again. The only
context key needed is website_id for `_get_related_assets`.
Note that it is not really enough, the orm cache key should actually
contain `request.session.get('force_website_id')` as well has
`request.httprequest.host`. This will be addressed latter since a nicer
solution would be to have website_id as a unique parameter computed
earlier.
2. better _get_asset_paths cache key
The orm cache key was simplified in previous point but there is still
one concern, the website_id depends on more parameters than that:
- request.session.get('force_website_id')
- request.httprequest.host
- existing websites
The idea here is to call `get_current_website` instead of using all
parameters that could define the webiste.
In the same spirit of `_get_template_cache_keys` `_assets_path_params`
can be overriden to give extra params that are usefull to list assets
path. Those params are computed before entering the method
`_get_asset_paths`. This may latter put at a higher level latter, in
get_asset_node, to simplify the _generate_asset_nodes_cache key.
3. better assets_node caches key
The main purpose of this part is to improve ormcache containing assets
nodes. The ormcache key contains
- to much context key
- missing session/host/env info
- unwanted boolean options.
- keys leading to the same cache value
The main goal being to reduce the size of the cache keys, decrease the
number of cache entries and improve the cache hit.
This will also make the behaviour more coherent and hopefully less bug
prone because of mismatch in parameters.
The main reason of the orm cache is the slowness of the validation of
the assets. This includes:
- listing files (dedicated orm cache)
- computing version
The cache key was depending on
- `debug`
The only relevant value for debug is "contains assets"
We dont need to differ between debug='', debug='1', debug='test',
and 'debug=assets', 'debug=tests,assets', ...
- `defer_load`, `lazy_load`, `media`
Those values are only useful to generate html node, a leightweight
operations that does not really needs to be in cache. `media` was also
used in the generation but it looks useless if we have the media on the
node. THIS NEEDS TO BE VALIDATED but in any case, since media is not
used to generate the url, it doesn't make sence to use it in the
generation.
The main idea to remove them from the ormcache key is simply to generate
the nodes outide the ormcached values.
-`async_load`
This one is similar to `defer_load` and `lazy_load` but it looks like
it wasn't used anymore. This was simply removed
- context.get('lang')
The only information needed is the direction, rtl or ltr. This means
en and fr languages, despite sharing the same css assets, will duplicate
the ormcache entries.
-`_get_template_cache_keys`
Only the lang and webiste where really relevant in this flow. Other
keys are actually useless in this flow.
Some information used in the generation where not in the orm cache key
- `self.env.user.lang` if there is no lang in the context
- `request.session.get('force_website_id')`
- `request.httprequest.host`
- ...
The proposed solution is to:
- extract any informùation needed from thecontext, request, environment
before entering the ormcache, reduce it to the minimal possible set of
values needed
```
rtl = self.env['res.lang']._lang_get_direction(self.env.context.get('lang') or self.env.user.lang) == 'rtl'
assets_params = self.env['ir.asset']._get_assets_params() # website_id
debug_assets = debug and 'assets' in debug
```
and remove a leightweight part of the logic
```
def _get_asset_nodes(self, bundle, css=True, js=True, debug=False, defer_load=False, lazy_load=False, media=None):
links = self._get_asset_links(bundle, css=css, js=js, debug=debug)
return self._links_to_nodes(links, defer_load=defer_load, lazy_load=lazy_load, media=media)
```
Where _get_asset_links is the cached part, and _links_to_nodes is the
lightweight part generating the nodes based on the `defer_load`, ....
Additionnal notes:
- data-asset-version and data-asset-bundle are removed from the node
since they don't seem to be used anymore since 65d70acdbf
- async_load is removed since there is no occurence of this in the code.
- a small hack is still needed to pass javascript content instead of
links, this is only to manage css compile error and will hopefully be
removed in the future.
- a context key is still in use to generate the bundle, the
`commit_assetsbundle` but it has no impact on content and will hopefully
be removed in the future.
4. Add test for ormcache hit/miss
In this context, hit/miss is about having the same cache key for the
same result. This test demonstrates the current state, were entries are
create in the ormcache only if the key is really different and will lead
to a different result.
5. remove cache invalidation
This cache invalidation is quite agressive since everytime an
assetbundle is updated, all workers will clear their cache.
The concerned cache by this clear_cache is `_generate_asset_nodes_cache`
throug `_get_asset_nodes`.
The cache is ignored, both in dev=xml and debug=assets.
This clear cache was made conditionnal in 553ea82f81 but this does
not solve an issue we can have in production.
Lets imagine a clean solution
- all sources are updated
- all workers are restarted.
The orm caches are all empty, but since the sources
changed, all bundles will be recomputed. This means that every bundle
updated in database with save_attachement will invalidate the cache of
all workers. Rendering a pdf report of any kind using a specific bundle
will invalidate all cache. Starting a debug=assets for the first time
will invalidate all cache, even if the cache is not used in this case.
But for a regenerated bundle we would expect the ormcache to be:
- empty (did not generate the same bundle yet)
- have the same value (concurrent generation of the same bundle)
Having a different value would mean that the bundle was generated with
another version of the sources. In this case it is maybe even better not
to invalidate the cache since it could lead to an invalidation war
between two workers.
The only case where invalidating this cache is useful is when a bundle
changes, Usually if an ir_asset is created, modified, ...
There is still another rare but possible possibility to have a 404 if
the transaction is rollbacked after populating the assets node cache.
In this case, we only need to clear the cache locally in case of
rollback.
Part-of: odoo/odoo#121376
Since changes made in https://github.com/odoo/enterprise/pull/41117 we
cannot provide icons for menus in an other format than png. As it was
allowed before to use SVG, we don't want to restrict to only one format.
Before, it was trying to gess the mimetype based on the image content
which is not also the best case.
So as the icon is a binary field store=True, we can just take the
mimetype from the attachment.
closesodoo/odoo#122580
X-original-commit: 044e6b680bc988708a9bbcc3c93dabf787c4a190
Related: odoo/enterprise#41529
Signed-off-by: Masereel Pierre <pim@odoo.com>
As the ChromeBrowser API was simplified in 2b0d9fa6a9, the
terminate_browser method was removed but the shiny author forgot to
remove the call in the click_everywhere test.
closesodoo/odoo#122038
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Initialize a DB. Install website when it is initialized.
Launch tests of the class TestBaseDocumentLayout
Before this commit, there was a crash because those tests render the report_layout and its
assets.
The full explanation is that, the module website adds an ir.asset `website.s_badge_000_variables_scss`
that the report wants to fetch. But, the class of tests is executed at_install. Given the topological order
and the order of the modules installation, website is not in the registry at that point, but the ir.asset is retrieved
from the database.
The ir.asset algorithm determines at that point that `/website/` is not an admissible path and raise an exception.
This commit solves the problem by tagging this class of test "post_install" and not "at_install".
runbot-error-21203
runbot-error-21204
runbot-error-21205
runbot-error-21206
runbot-error-21207
runbot-error-21208
runbot-error-21352
runbot-error-21353
runbot-error-21354
runbot-error-21355
runbot-error-21356
runbot-error-21357
closesodoo/odoo#121998
X-original-commit: 48b833513ac8f76febbc4ca7b935260d7ea85522
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
One of the most costly part of a page loading when the ormcache is cold
is computing the assets node, the unique identifier of an attachment to
validate whether the existing attachment is still valid with the current
version of the static files.
This operation needs to glob assets path in the filesystem,
get the modification date, check attachments, ...
Right now this task is not really optimized and can take some time
because of an excessive number of glob on the filesystem, unnecessary
exists to define absolute path, double computation of file list and
modified times when getting js and css bundle separately, ...
A list of modifications mainly discussed in the pr message are made
with this commit to speedup things.
- split css and js unique
- prepare api for an in memory glob
- change api to propagate absolute path and meta information through
`ir.asset._get_paths`-> _get_asset_paths -> `_get_asset_content` ->
`AssetsBundle`
closesodoo/odoo#121159
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Before this commit, when the user wants to self assign to a task for
instance, he has to write his name to be able to select himself.
This commit displays the current user first in the result of
`name_search` (the method called by relational widgets in JS).
If the current user does not satisfy the search condition then
he will not be in the result of the `name_search`.
task-3291745
closesodoo/odoo#121146
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Add a test_logs_assets_check_time test to make stats on the time needed
to validate all asset bundle (will be added to runbot stats)
Add a test_logs_pregenerate_time, mainly to profile this part during
devlopment. This test is -standard and will only run if requested.
closesodoo/odoo#120767
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Introduce an optimised way of reading a graph of data from the webclient.
Before this commit:
When reading data from the webclient it could at most read multiple ids of the same model in one RPC.
This mean that when reading x2many or specific information on many2one, that could only be done after the initial read (when the client knows the ids of the comodels) and model by model.
After this commit:
Introduce methods web_read and web_search_read_unity. Both method receive a specificiation for the fields instead of a list of fields. The specification can request fields from the model, as well as follow relations and request fields for each relations, recursively.
Example of web_read specification for account_move
```python
{'name': {}},
{'date': {}},
{'journal_id': {'fields': {'display_name':{}}}
},
...
{'invoice_line_ids' :
{
'fields': {
'journal_id' : {'fields': {'display_name:{}}},
'move_name' : {},
...
'tax_ids' : {
fields: {
'display_name':{},
...
}
}
}
}
}
```
Result for this example with 2 invoice lines
```python
{
'id': 1234,
'name' : 'invoice name ABC',
'journal_id: {
'id': 999,
'display_name': 'Customer Invoices'
},
...
'invoice_line_ids': [
{
'id': 666,
'journal_id': {
'id': 999,
'display_name': 'Customer Invoices'
},
'move_name': 'a move name',
'tax_ids': [
{
'id': 333,
'display_name: "15% tax",
...
}
]
},
{
'id': 667,
'journal_id': {
'id': 999,
'display_name': 'Customer Invoices'
},
'move_name': 'another move name',
'tax_ids': [
{
'id': 334,
'display_name: "21% customer tax",
...
}
]
}
]
}
```
closesodoo/odoo#119034
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Steps to reproduce:
- Make sure language preference is 'en_US'
- In accounting, in the dashboard click on bills
- Filter 'due_date' by week
Issue: The start day is Monday and should be, for 'en_US', Sunday as it
is the case in the dashboard view in accounting (see appendix).
Cause: The query uses the `date_trunc('week', date)` which in Postgres
retrieves the first day of the week as Monday (ISO week).
Solution: Create an offset in the query depending on the first day of
the locale variable.
Note: the `web/tests/test_read_progress_bar.py` has been modified: since
the default language is 'en_US' there will be an offset of one day. To
make it less confusing, I used only two anglo-saxons countries so the
day offset is not the variable tested. (for this matter, pleaser refer
to `test_read_group/tests/test_read_group_process_groupby.py`)
Appendix:
Language (english-US)
VIEW (per week) | DASHBOARD
___________________________________________________________________
W23 -> 06/05 | 05/29 -> 06/04
W24 06/06 -> 06/12 | 06/05 -> 06/11
W25 06/13 -> | 06/12 -> 06/18
(Monday - Sunday) (Sunday - Saturday)
Language (french-BE)
VIEW (per week) | DASHBOARD
___________________________________________________________________
W22 -> 06/05 | 05/30 -> 06/05
W23 06/06 -> 06/12 | 06/06 -> 06/12
W24 06/13 -> | 06/13 -> 06/19
(Monday - Sunday) (Monday - Sunday)
opw-2747066
closesodoo/odoo#93053
Related: odoo/enterprise#29539
Signed-off-by: Raphael Collet <rco@odoo.com>
In werkzeug 2.2.2, the following characters "$!'()*+,;" are now
considered as safe by url_quote. This makes the filename_secure test
fail with the hard coded expected string containing a single quote as
'%27'.
This commit adapt the filename_secure test in order to work with all
versions of werkzeug.
closesodoo/odoo#112298
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
This commit fixes a change in behavior between 15.2 and 15.3.
Previously, if an unidentified user tried to reach a route that had auth='user', it would simply redirect to the login page.
Currently, it redirects and invalidates the session_id.
This is an issue in the latest version of master after this PR https://github.com/odoo/enterprise/pull/36521
This commit changes the route of service-worker.js to auth='user'.
This route is called on the login page, which rotates the sid and therefore invalidates the csrf token. Making it impossible for a user to log in.
This is a race condition, meaning it would only appear if the user stayed on the login page for a few seconds, hence why the automated testing did not block the commit.
closesodoo/odoo#112239
X-original-commit: d5d80d172616afe02bd41934930ea18dc273c739
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Co-authored-by: Julien Castiaux <juc@odoo.com>
When count_limit is set and lower than the current number of fetched records, making an extra search_count is useless.
closesodoo/odoo#111895
X-original-commit: 6f90d6924e24e700694111732ee85465f802b22f
Signed-off-by: Rémy Voet <ryv@odoo.com>
Before this commit, if you have some special char like a return line \n,
or \r the get_stream_from method will crash with exception:
```
File "/home/odoo/src/odoo/odoo/addons/web/controllers/binary.py", line 163, in content_image
return stream.get_response(**send_file_kwargs)
File "/home/odoo/src/odoo/odoo/odoo/http.py", line 578, in get_response
res = _send_file(self.path, **send_file_kwargs)
File "/home/odoo/src/odoo/odoo/odoo/tools/_vendor/send_file.py", line 156, in send_file
headers.set("Content-Disposition", value, **names)
File "/usr/local/lib/python3.9/dist-packages/werkzeug/datastructures.py", line 1218, in set
self._validate_value(_value)
File "/usr/local/lib/python3.9/dist-packages/werkzeug/datastructures.py", line 1182, in _validate_value
raise ValueError(
ValueError: Detected newline in header value. This is a potential security problem
```
Now we replace `\n` `\r` by `_` before to serve the stream to avoid this
security exception from a safe way.
We decided to not use secure_filename from werzkeug because we want to
continue the support of non ascii char.
closesodoo/odoo#111851
X-original-commit: 95584e71a898017a92112f89e8a94315dd9235ac
Signed-off-by: Jérémy Kersten <jke@odoo.com>
Using patcher.start() can easily lead to incorrect cleanup.
-> after a copy paste, patcher is working, but stop is forgotten
-> stop is present, but won't be called if something fails during the
test
This commit add an utility `start(patcher)` to always have the add
cleanup.
Using a standard way to start the patcher with an automated addCleanup
should prevent this kind of mistake. This is why this commit also
replaces all valid patch.start() (followed immediately by a addCleanup)
closesodoo/odoo#102873
X-original-commit: 7d5a193d86316965a0908c65cfacfb607dc3f3ad
Related: odoo/enterprise#32618
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This commit avoid to have a dict by reference that will be global.
Now get_default_session return a new dict each time for the context key.
From this way the session.context['lang'] is not shared between several
users on the same worker.
To reproduce the bug, restart the server with 2 workers, make request in
lang A on these 2 workers. DEFAULT_SESSION['context']['lang'] now is set
to this lang A.
Now, make request to an url without lang in path and without cookies and
withtout session, you should be redirected to lang B (preferred lang
from the request header) but you will be redirect to lang A due to the
dict session.context that is shared for the worker...
When we initialize the new Session, we get the wrong lang A as value for
context.lang, so we don't recompute the expected lang for the end user.
X-original-commit: 62179de74862210fe2a055d15b367b1850c24263
fwd-port of #100102closesodoo/odoo#100910
X-original-commit: 42e46b2d89dde276f796b980f29e33cc216e7cb2
Signed-off-by: Jérémy Kersten <jke@odoo.com>
The purpose of the task is twofold:
1. Remove empty lines in the company address.
Until now, the address format was fixed, which could
lead to empty lines if one or more field(s) were missing.
We are now removing empty fields to avoid that.
2. Make sure the external report layout is configured
before generating the PDF.
This will ensure that the company data will appear
in the file. If no layout is defined,
it would not be shown.
task-2834517
closesodoo/odoo#100936
X-original-commit: f36bb6acdacaaba26afdd8f62c48fd2c8784d1e1
Signed-off-by: Olivier Colson (oco) <oco@odoo.com>
Signed-off-by: John Laterre (jol) <jol@odoo.com>
Since #99912 logging an error message doesn't always end qunit tests.
This was mainly to allow to failfast logging qunit errors earlier
without stopping the tests in order to test all qunit anyway.
The logic was to have an end message that stops the test.
Unfortunately some errors will prevent the qunit suite to start
and the test will wait a 1800 long timer. An example was because of
a Missing dependencies. https://runbot.odoo.com/runbot/build/19306352
This new approach will avoid to stop only if the message looks like a
qunit failure and the final message is not there (to be sure).
closesodoo/odoo#100238
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
XML files are now declared in python module manifests. During the qweb
't-call-asset' directive, assetbundle will fetch the declared xml files,
apply the inheritance (t-inherit) and create a javascript service (for
eg: 'web.assets_backend.bundle.xml') which is added at the end of the
*.js mimifier file.
When the debug mode is activated, comments are added in the template
indicating which file the template comes from as well as the
inheritances applied to it.
****
JavaScript:
assets.js (module @web/core/assets) takes care of loading libraries,
javascripts and styles.
`loadJS(url)` (loads the javascript and returns a resolved promise when
the templates are also loaded via the '*.bundle.xml' service)
`loadCSS(url)` (loads the style a resolved promise when the file is
loaded)
`loadXML(xml, app=assets.defaultApp)` (load template into
application/owl, used by the `*.bundle.xml` services)
`getBundle(bundleName)` (get the bundle descriptor)
`loadBundle(desc)` (load the files and bundle from a descriptor)
templates (XML element content all owl templates)
A new `ready(serviceName)` method on boot.js lets you know when a
service is loaded are the require.
The xmlDependencies attribute no longer exists.
Python:
The xmls taken into account by assetbundle.py, applying `t-inherit`
inheritances and adding an `name_of_the_bundle.bundle.xml` service in
the generated JavaScript file.
****
Every manifest changes is into the next commit, except 'web_tour' in
this current commit as example.
Part-of: odoo/odoo#95500
Since bundle generations should be generated before post install tests,
a loading of a /web shouldn't try to save a new version
This is currently breaking for / because of the website_id added in the
extra part of the attachement. Even if the content is the same.
Part-of: odoo/odoo#99176
Right now the qunit will log all results at the end.
This means that the runbot may wait for all qunit before detecting the
failure.
This also mean that all failure are in one ir.logging on runbot, making
the automated parsing difficult if multiple modules fails during the
same build.
We could also log all failure immediately, but grouping them my qunit
module will avoid duplicating logs for linked causes (one failure
leading to a `Expected %s assertions, but %s were run` message)
closesodoo/odoo#99912
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Time comparison can always be slightly random (this is why this test
is a nightly one). The 20% margin left by the 12 ratio is not enough
in all cases. This test was sometime breaking with a
12.944994188420822 not less than or equal to 12
This is one of the max value found by quickly checking the builds.
A ratio of 14 should be hopefully enough.
closesodoo/odoo#99156
X-original-commit: cc86b80342d38913f7af79474c41f8764c8b2dd2
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Purpose
=======
For security reason access on ir.model is not granted to internal suers. Due
to this constraint a component exists in spreadsheet to be able to select
the models for which we have a read access on the records of this model.
This is required for the properties fields feature, hence moving its code
to web.
Some renaming is performed to make it generic. This generates some changes
in other addons, notably some class renaming.el.
Task-2852259
Part-of: odoo/odoo#95184
Skip test on database manager rendered page when option
--no-database-list is used
closesodoo/odoo#98771
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The clickbot click default timeout was increased in #98495, but missed
the fact that the default was overridden for the "Settings" menu to a
lower value.
With this commit, the "Settings" exception is completely removed as the
default timeout is higher. Also, checking that the text contains
"Settings" was a bit weak.
While at it, the timeout for the global testing of an app is also
increased to 10 minutes instead of 5 as this limit is reached by the
Field Service app.
closesodoo/odoo#98617
X-original-commit: 0247e6f4abef4121b451c1373ae3c1ce184451c0
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
We can't really know where the clickall tour will end, and if it ends
on the settings action things are rather difficult as saving or
discarding the settings form returns to the edition mode.
Part-of: odoo/odoo#96517
Auto retry can be usefull to avoid breaking a build because of a
small tour or query count, but for long tests like qunit, this can be
painfull when a real error is triggered.
This commit proposes to disable autoretry on demand for some tests
to solve this issue.
This may be applied on all tests longer than a few seconds.
closesodoo/odoo#95440
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this commit Chrome's "touch mode" was enabled in both desktop and
mobile-like tests suite (when run headless).
To better match real usecases, this commit adds an option to
enable "touch mode" only in mobile tests suites; keeping it disabled in
desktop ones.
Part-of: odoo/odoo#95924
Each cookie binds to a domain name, multiple cookies can be set for the
same name if they are for different domain name. In this case, two
`session_id` cookies were set: (1) the first set right on the opener at
`opener.cookies[...] = ...`, (2) the second set upon inside of
`http.Request._save_session` because the session was rotated upon login.
The problem is that the former cookie (the one set on the opener, the
one *not* rotated) was used instead of the second cookie (the one
holding the registered user) in the subsequent queries. There is a long
comment explaining the same problem inside of
`odoo.tests.common.HttpCase.authenticate`, we used the same solution as
they did inside of `authenticate`: we diched the previous opener.
closesodoo/odoo#94773
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When portal is not installed and `auth_signup.invitation_scope` is "b2c",
visitors can create an account, leading to a blank page. Still, accounts can be
required for several use cases in apps that do not require portal (such as
survey).
We here add a landing page for users that created an account but have no
requested redirections and cannot be redirected to a customer portal either.
auth_signup_uninvited is also updated in model to be consistent with config
data.
Tests are added to check this behavior.
Task-2762102
Part-of: odoo/odoo#85703
Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closesodoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
When a filename is given to the function binary_content it was shadowed
by _binary_ir_attachment_redirect_content if it is an ir.attachment.
To reproduce:
1. Start a brand new database in V14, install any app on which you can add an attachment (like Project or CRM)
2. Add a file as an attachment
3. Try to call the route "/web/content/<string:model>/<int:id>/<string:field>/<string:filename>"
closesodoo/odoo#87879
Solution: Use another variable to store the return value of _binary_ir_attachment_redirect_content and use it if the filename is not provided.
X-original-commit: 55e6097502da6169b85056b4769f4bf7fd230968
Signed-off-by: Wanderscheid Mathieu (mawa) <wama@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
The odoo.addons.web.controllers.main python module have been splitted
over multiple files on the basis 1 controller = 1 file. In this work we
adapt all modules to use the new imports.
A non-exhaustive list of where stuff have been moved:
* main.Home --> home.Home
* main.Session --> session.Session
* main.WebClient --> webclient.WebClient
* main.clean_action --> action.clean_action
* main.ensure_db --> home.ensure_db
The complete list is accessible in odoo.addons.web.controllers.main.
closesodoo/odoo#87571
Related: odoo/enterprise#25746
Signed-off-by: Raphael Collet <rco@odoo.com>
Install auth_signup, go to /web/login, 500 Internal Server Error.
auth_signup extends the /web/login template and in this extension calls
`keep_query()` which has been wrongly moved from base to http_routing in
commit 880954ebfc. Here, we restored `keep_query()` in the base module
but moved in ir_qweb.
closesodoo/odoo#87491
Related: odoo/enterprise#25754
Signed-off-by: Julien Castiaux <juc@odoo.com>
This commit is the 12th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
The web module is twofold, on one side there are many controllers: /,
/web, /web/login, /web/database/selector, /web/dataset/call_kw, etc, on
the other side there is `session_info`: the method responsible to create
the web client's environ.
This module is kinda an exception as it is (with base) a server wide
module. In the case of the HTTP framework, it means that the controllers
of web are always accessible, i.e. going to / or /web/login will never
return a 404 Not Found even if the user is not connected to a database.
This is both a blessing and a curse. It is a blessing because the
controllers are always accessible it means that a new users can freely
access those routes. It is a curse because *any* user can access them,
even user who don't have a session yet thus who are not connected to a
database yet. From a developer standpoint, we have to put extra care to
correct serve users with and without a database. An example is the
/web/login route, the login/password pair is stored in a database,
without database it is impossible to validate a user login but users can
still access this route without db.
To solve this problem, there is the `ensure_db` function. This function
attempts to find a database using various sources (?db= query-string,
session db, mono db) and to save it on the user session. In case no db
is found, the user is redirected to the database selector. In a way,
this function grants a database to the user in a seamingly experience.
In a way, this function brings a welcome differentiation between
`auth='none'` with a database and `auth='none'` without a database. Such
differentiation only matters for the server wide modules as "regular"
module controllers are only accessible via the ir.http routing map, i.e.
it is not possible to declare a nodb controller outside of server wide
modules.
An important changement is the `session.authenticate` method, before it
was possible to call the method when the cursor was not yet initialized,
authenticate would open a cursor against the given database, setup a
registry and an environment and ultimately save everything on the
current request. Because the cursor is now greedily created, it is no
more possible to update the request environment when authenticating on
another database.
PR: odoo#78857
Task: 2571224
Before this commit: static XML templates could only be defined on the
file system, and called by manifest assets or ir.asset records.
Attachments were not taken into account when evaluating static
templates.
Now, if a given path does not match a file on the system, an
additional check is run on ir.attachment records instead of failing
directly.
Task 2715333
closesodoo/odoo#83438
X-original-commit: e022c4bfafa77f1a3433c3b980631510af696ade
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
Signed-off-by: Julien Mougenot (jum) <jum@odoo.com>
On runbot database can be create concurently by other builds, meaning
that the teardown may fail randomly if a database is create between
setup and teardown. Filtering on the dbfilter in all case may miss some
errors but will be enough in this case.
closesodoo/odoo#83414
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When changing the background image with studio the image does not update.
It works in debug=assets mode.
This is due to the fact that the caching system is not aware of the presence of a possible background image
opw-2696786
closesodoo/odoo#83374
X-original-commit: 184029e3e1e8907e25dd712dd87afd65885695bb
Related: odoo/enterprise#23744
Signed-off-by: Achraf <abz@odoo.com>
Database manager may easily be broken since it wasn't tested and is a
special case (can be rendered without databases). This commit adds a
basic test to check that the database manager is rendered as expected.
Testing database rendering is not enough, in some cases the database
operations may be broken. Another test will be executed on runbot to
test basic create/duplicate/delete operations. The test is tagged as
"-standard" since it can be a risk to execute such operation
automatically with other tests.
closesodoo/odoo#82874
Signed-off-by: Raphael Collet <rco@odoo.com>
As this test is always red, notably because of the qunit asset bundle, the
threshold is raised for some bundles based on runbot builds
observations.
closesodoo/odoo#82972
X-original-commit: ab8350b5432a6a6d671bcc76413815274f7daaca
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
As JS is not taking the properties in the order they are written,
the order in the rendering was always following the property name
sorting order (=id).
Previous to this commit:
- The companies were ordered by their id in the company switcher as
JS is not tacking the object properties order into account.
After this commit:
- The companies will be sorted by their sequence prior to be used in the
rendering.
task-2722235
closesodoo/odoo#81893
X-original-commit: 39c678a1ccb50d3a1871a4049a8df26427b27a3c
Signed-off-by: Laurent Stukkens (ltu) <ltu@odoo.com>
std-odoo