This commit reworks a little bit the backend assets to remove a
bundle and thus save a call at webclient startup. The bundle
"assets_backend_prod_only" existed only to allow to add files in
production, but not in the tests (typically, the file that spawns
the webclient).
This commit introduces a new bundle "web.assets_web" that contains
"assets_backend" and the few files that we only want in production.
In the /web page, we now load "assets_web" instead of
"assets_backend" and "assets_backend_prod_only". In the /web/tests
page, we keep loading "assets_backend", which is now directly
included into "web.tests_assets".
For the sake of consistency, this commit also renames the dark
mode bundle "dark_mode_assets_backend" into "assets_web_dark".
closesodoo/odoo#135204
Related: odoo/enterprise#47316
Signed-off-by: Samuel Degueldre (sad) <sad@odoo.com>
*: auth_password_policy, bus, event, im_livechat, mail, mass_mailing,
mrp_subcontracting, point_of_sale, pos_self_order, project, stock,
survey, web, web_editor, web_tour, website, website_event,
website_forum, website_sale, website_slides, base
Historically, the web.assets_common bundle was used to contain assets
that were needed by both the frontend and the backend. In practice, this
caused a bunch of issues where people would add things in assets common
that were not needed by both, and it was also abused as a way to get
bootstrap working in unrelated places by only using that bundle's css.
Because of this, as a first step, the assets_common stop being used in
the frontend, but was left everywhere else.
This commit removes the bundle completely, and moves the files that used
to be in that bundle in the other bundles that need them, this will
allow those bundles to evolve independently going forward.
in im_livechat and mail, some of the unneeded legacy code was removed, this
allows us to avoind including all of the legacy code from web in the
livechat embed bundle and in the dicuss public bundle respectively.
closesodoo/odoo#132190
Related: odoo/enterprise#45884
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
*: bus, crm_livechat, crm_mail_plugin, im_livechat, payment, payment_demo,
pos_online_payment, test_discuss_full, test_mail_full, tests, website_livechat,
website_sale.
Several python tests have their own way to make jsonrpc requests. This PR adds a
generic `make_jsonrpc_request` method to the `HttpCase` in order to provide a
generic way to do so.
At the same time, calls to `_open_livechat_channel` are removed in favor of
jsonrpc request to `get_session`. This makes the tests more realistics (some
incoherences were present like passing `country_id` to the open channel method
while the user country id is not set...)
Finally, this will ease the diff in the PR introducing livechat visitors as mail
guests.
part of task-3332628
closesodoo/odoo#130036
Related: odoo/enterprise#44840
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.
Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.
The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.
Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.
The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.
We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.
Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.
Note that this poc is inspired from the long term cache but not all
use case where applie yet.
Part-of: odoo/odoo#119813
Purpose of this commit is to regroup some tests, or rename some too specific
files in order to avoid explosion of files, making tests hard to find when
looking for feature-specific tests.
Task-3346388
Part-of: odoo/odoo#123678
* = bus, calendar, crm_livechat, hr, hr_holidays, im_livechat, mail_bot,
mass_mailing, privacy_lookup, test_discuss_full, test_mail,
test_mail_full, website_crm_livechat, website_livechat, base
In preparation of splitting discuss and mail modules.
Part of task-3265211
closesodoo/odoo#118354
Related: odoo/upgrade#4553
Related: odoo/enterprise#39661
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
This PR fixes the `15350`runbot issue that was due to the
websocket subscribe not being properly awaited before logging
out.When the log out occurred before the subscribe was received,
the `odoo_ws` variable was `None`.
closesodoo/odoo#118097
X-original-commit: 59af91840c3ed7253c1c3c9dd80a4919ebdee593
Signed-off-by: Alexandre Kühn (aku) <aku@odoo.com>
Before this commit, the keep alive timeout could occur for many websockets at
the same time resulting in a burst of transaction (cursor is open when connecting,
disconnecting a websocket).
This issue was even worse because the cursor was opened even when it was
not needed: we don't need to open a cursor if there is no callback
registered for the lifecycle event (OPEN/CLOSE).
This commit fixes those issues by:
- adding a random delay to the keep alive timeout of every websocket.
- not triggering lifecycle events if no callbacks are registered.
closesodoo/odoo#110127
X-original-commit: b4fb1359021b6c882ece3037c82aaaf65cfc2e1e
Signed-off-by: Julien Castiaux <juc@odoo.com>
Before this PR, the last notification id known by the client
was not reset after restoring the database. This was an issue
since there can be a gap between the last notification and the
one that has been restored with the database. In this scenario,
messages are not received after restoring the database since the
client subscribes to higher notification ids that the ones that are
created.
This PR fixes this issue by defaulting to 0 if the one the client
passed is higher.
closesodoo/odoo#103311
X-original-commit: 5118da53a6e9adb94577a4f16999522a7e1e8c10
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Signed-off-by: Stockbauer Matthieu (tsm) <tsm@odoo.com>
Using patcher.start() can easily lead to incorrect cleanup.
-> after a copy paste, patcher is working, but stop is forgotten
-> stop is present, but won't be called if something fails during the
test
This commit add an utility `start(patcher)` to always have the add
cleanup.
Using a standard way to start the patcher with an automated addCleanup
should prevent this kind of mistake. This is why this commit also
replaces all valid patch.start() (followed immediately by a addCleanup)
closesodoo/odoo#102873
X-original-commit: 7d5a193d86316965a0908c65cfacfb607dc3f3ad
Related: odoo/enterprise#32618
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
In order to know whether or not the websocket session is up to date,
the session is saved when opening a websocket connection. Then, for
each incoming/outgoing message, we check if the session still exists
on the file system. When it does not exist anymore, the websocket
connection is refreshed. This ensures the session is always up to date.
Odoo sh proxies the websocket connection and needs a way to tell
when a websocket session is expired. This commit adds the same
check that is done for each incoming websocket message in the
websocket peek route in order to raise a `SessionExpiredException`
when the session is outdated.
This will allow odoo sh to catch this error and to refresh their
websocket connection accordingly.
closesodoo/odoo#100416
Signed-off-by: Julien Castiaux <juc@odoo.com>
Before [1], only string channels were allowed for polling. This
ensured no one could send a server-side channel from the frontend,
this PR restores this behavior.
[1]: odoo/odoo@a5623d2closesodoo/odoo#100309
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
When executing python tests with Odoo opened in the browser, the
websocket connection coming from the browser can lead to issues
with savepoints/rollbacks. Indeed, during test set up, a savepoint
is created. This savepoint is released at the end of each test.
The issue occurs when the websocket connection opens a cursor (and
thus creates a savepoint) after the one created by the test but releases
it after the one created by the test:
- SAVEPOINT TEST
- SAVEPOINT WS
- ROLLBACK TO SAVEPOINT TEST
- SAVEPOINT WS DOES NOT EXIST
In order to solve this issue, let's prevent browsers from opening a
websocket connection during python tests. This does not apply to chrome
headless since remaining threads are awaited before the end of every tour
(which means the websocket connection will be close before releasing the
test cursor).
closesodoo/odoo#99538
Signed-off-by: Julien Castiaux <juc@odoo.com>
Before the websockets were introduced, longpolling coroutines were
sleeping until postgres notify. Each coroutine was then wake up and
notifications were fetched.
Since the websocket introduction, the main loop, responsible for listening
to postgres sends the notifications itself. This means, the postgres loop
is blocked during message fetch/dispatching and notifications are dispatched
in a sequential fashion resulting in a slow message dispatching.
In order to solve this issue, websocket coroutines are now responsible to
fetch/dispatch notifications, letting the main loop free to relay notifications
as they come and allowing notifications to be sent simultaneously.
When instructed to dispatch available notifications, the websocket coroutines
will try to acquire a cursor. Each coroutine will try up to `MAX_TRY_ON_POOL_ERROR`
times, sleeping between each try. If no cursor can be acquired, the connection is
closed with the TRY_LATER` close code.
closesodoo/odoo#98880
Signed-off-by: Antony Lesuisse <al@odoo.com>
The test_user_logout_outgoing_message is undeterministic because
it speculate on the fact that the channel subscription was made
when we try to dispatch notifications. Sometimes it is not and
the websocket to channel map is empty resulting in a StopIteration
exception being raised. Let's wait for the subscribe to occur to
ensure the websocket will be registered before trying to dispatch
the message.
closesodoo/odoo#99204
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
When a websocket is disconnected, its subscription is removed.
Each channel leads to a set of subscribed websockets. When no
more sockets are listening to a channel, this channel should be
pop from this mapping or the map will keep growing.
closesodoo/odoo#98766
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
The websocket client library is used during tests but is not in
requirements.txt. Therefore, imports are usually wrap into a try/except.
An import present in `websocket_rate_limiting` is not wrapped leading to
errors for those who have not the library installed. This PR fixes this
issue by wrapping this import in a try/except block.
closesodoo/odoo#98763
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
*: hr_presence, web_editor.
This commit is part of the websocket integration in Odoo.
It focuses on adapting the bus to support websockets:
- last notification id is now kept on the server
- channel list is built by overriding the `_build_bus_channel_list`
method of the `ir_websocket` model instead of overriding the `_poll`
method of the bus controller.
- The bus presence was updated during polls, since there is no more poll,
bus presence update will be the responsability of the client.
- The `/websocket/peek_notifications`, `/websocket/update_bus_presence`
routes will be available so that odoo sh can access notifications/update presence
from http requests.
- /longpolling routes are now prefixed with /bus thus won't be redirected to the
gevent worker anymore except for `/longpolling/health` which is the
health check route of the gevent server.
Since websocket now handle incoming messages, a way to manage authentication
have been introduced :
- The session is retrieved from the HTTP handshake.
- When a websocket message comes/leaves the session is retrieved
on the file system so that we're sure it still exists and that
it is up to date.
- The session is checked
- If no session is found on the file system or `check_session`
fails, the websocket connection is closed with the `SESSION_EXPIRED`
close code (which is a custom close code: 4001).
- Note that websocket connections are closed every `KEEP_ALIVE_TIMEOUT`
seconds to ensure no websocket connection will stay open if the user
clears its cookies.
- Note that a wsrequest object is available when processing incoming
messages. It is similar to the http request and contains various
useful informations (session, env, ...).
Part-of: odoo/odoo#75510
This commit is part of the websocket integration in Odoo.
It focuses on implementing websocket lifecycle events.
Two lifecycle hooks are currently available:
- onopen: called after the websocket opening
- onclose: called after the websocket closure.
Those callbacks will be passed an environment and the websocket
related to the lifecycle event.
In order to subscribe to websocket lifecycle events, one must decorate
a free functions with either `@Websocket.onopen` or `@Websocket.onclose`.
Part-of: odoo/odoo#75510
This commit is part of the websocket integration in Odoo.
It focuses on implementing rate limiting to the incoming
websocket messages. When opening a connection, a burst of messages
is allowed. When requests are received too fast, an exception is
raised and the websocket is closed.
Two config parameters are added to customize the rate limtier
behavior:
- websocket_rate_limit_delay: Integer specifying the seconds that
should space out two requests.
- websocket_rate_limit_burst: Integer specifying how many websocket
frames can be accepted in excess of the specified rate.
Part-of: odoo/odoo#75510
This commit is the first commit of the websocket integration in Odoo.
It focuses on the implementation of the websocket protocol as per RFC6455.
The implementation is tested thanks to the autobahn test suite.
A config parameter is available to customize the websocket connection:
- websocket_keep_alive_timeout (default 600): Integer specifying how
many seconds a websocket connection should be kept alive
Part-of: odoo/odoo#75510
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
=> `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
=> `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.
Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.
Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.
PR: odoo#78857
Task: 2571224
Before this commit: a notification asking to reload the current window
appeared as soon as the server detected a change in one of the assets
bundles, even on first load.
To fix this problem and make the feature more meaningful, it has been
decided to only notify the client when the server version (not the
bundle version) is outdated (i.e. on database upgrades, when the changes
in the code are actually relevant).
closesodoo/odoo#82032
X-original-commit: a3b5a9d715be6a93c7f2859074b916f3249f97c3
Signed-off-by: Antony Lesuisse <al@odoo.com>
Signed-off-by: Julien Mougenot (jum) <jum@odoo.com>
* = auth_signup, calendar, im_livechat, snailmail_account, survey, test_mail,
web_editor, website_crm_iap_reveal, website_livechat
The aim of this PR is to improve/fix various flaws and limitation of the current
API, to make it easier to use and more efficient.
Notification are now defined with 3 distinct parts:
- the channel determines which client(s) should receive it
- the type determines how it should be handled
- the payload determines any extra information helpful for handling it
Channel
=======
Business code
-------------
- Record channel is introduced for ease of subscribing to and sending
notifications to specific partners, channels, documents, ...
- String channel is still supported (but it is converted internally to the tuple
channel).
- Tuple channel is still supported without any change (but should be avoided
whenever possible due to its complex syntax).
The channel is no longer sent to the client. When the channel was used for
business purpose, the information it contained has been moved into either the
new type, or the payload itself.
Technical note
--------------
All channels are now internally converted to the tuple (db, ...) channel, which
is necessary for the platform code (saas/sh).
Internally, the bus.bus table is not changed, type and payload are grouped
together into what was (and still is) called message.
Type
====
Type is introduced to uniformize the way notifications are sent and handled.
All existing notifications already had some kind of manually-built type in them.
This is now officially supported at the bus API.
In client code this will allow (to be done in future commits) to register one
handler per specific type, instead of having to iterate and to filter all
received notifications on every handler.
Payload
=======
Payload (ex message) did not change, it can still be anything depending on
business needs.
Few adaptations:
- When the type was included on the payload, the type has been moved to the new
type parameter.
- When the channel was used in business code, its data has been copied into the
payload.
task-1891151
closesodoo/odoo#79201
X-original-commit: 543af27c7d6836ffac9e80ff8490b6ddbd849221
Related: odoo/enterprise#21998
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
This commit adapts the community codebase to the rewriting of the
/web application in owl.
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Géry Debongnie <ged@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Simon Genin (ges) <ges@odoo.com>
Co-authored-by: Francois (fge) <fge@odoo.com>
Co-authored-by: Michael Mattiello (mcm) <mcm@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais (lpe) <lpe@odoo.com>
Co-authored-by: Jorge Pinna Puissant <jpp@odoo.com>
Bus holds tests about bus notifications sent when changing bundles. Currently
it checks that exactly 4 notifications are sent during the test, being of
type ``bundle_changed``. However if other notifications are sent during
``bundle_changed_notification`` tour test fails even if notifications are
not linked to bundle events at all.
In this commit we fix this test by checking that we have 4 ``bundle_changed``
notifications sent during the tour. Other notifications are ignored for that
specific test.
LINKS
Task ID-2070632 (main task)
Task ID-2419762 (followup task)
COM PR odoo/odoo#62859
ENT PR odoo/enterprise#15172
UPG PR odoo/upgrade#2005
Use case:
When the server is restarted, the python is updated,
but some users may have an ongoing session in a browser tab
This may lead to code being unsynchronized and ultimately to some
odd bugs.
Purpose:
When we are in such a case, that is, the assets were recomputed
after a update of the code and a restart of the server by the request of another user,
notify connected users that assets have changed.
Then propose them to reload the page.
Known caveats:
- This is not a developer's feature.
Since assets computing is ORM cached, they have limited
opportunities to rebuild. Namely, the feature won't trigger
each time the JS has changed, rather, it will
when JS has changed AND the cache has been reset somehow (e.g. when the server is restarted).
- This not a portal/website feature either, but only in backend.
Business clients won't be notified that the JS has changed.
- While requests debug=assets do trigger a recomputing
of the *components* of bundles, they do not save a bundle
This means that the requests that sends the notification
cannot be debug=assets.
Task 2034462
closesodoo/odoo#39875
Signed-off-by: Mathieu Duckerts-Antoine <Polymorphe57@users.noreply.github.com>
Aaron Bohy