/longpolling/poll requests are most often requests to Odoo. Moreover,
such requests may be sent at the same moment from all users. For
example, when all users are subscribed to a common channel and someone
sent a message, all current polls are closed to deliver the notification
and after that, all clients start the request again.
If some users have several clients open (e.g. on desktop and mobile),
they may send many parallel requests and hence make concurrent queries
to update presence. We don't need be sure that every such query is
processed. So, just fail fast and carry on polling.
To test perfomance impact of this commit, copy curl command for poll request
from browser network tool and repeatly execute it, e.g.,
```
for i in {1..1000}
do
sleep 0.1
curl ... &
done
```
Without this commit you may notice such warnings in the logs:
```
... odoo.service.model: SERIALIZATION_FAILURE, retry 1/5 in 0.2071 sec...
```
At that moment, try to make normal odoo operations (e.g. create a sale
order): it would work slower than usual.
---
opw-2451865
close#57067closesodoo/odoo#67390closesodoo/odoo#67440
X-original-commit: aad9cbe80dae28c0869fe6f543fbe0118357ce57
Signed-off-by: Ivan Yelizariev // IEL <yelizariev@users.noreply.github.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Start odoo in threading mode with bus installed. Login in the browser
using any internal user. Make sure the browser call the
/longpolling/poll uri. While the browser is waiting for a response, stop
the server. The server takes up to 50 seconds to stop.
When started in threading mode, a request to /longpolling/poll is served
by a casual http thread. It searches for messages enqueued in the bus
and returns them. If there are no message for the user in the queue yet,
it creates a `threading.Event`, attach it to the user in a shared
dictionnary and `wait()` on it with a timeout of 50 seconds (hardcoded
value). When the bus thread (the one responsible to listen on the
database) receives new messages, it `set()` the events which resume any
http thread that was waiting.
Because when we stop the server, there is no way to server new requests,
there are no way new messages arrive in the bus. All the threads that
were waiting for a new message will just wait until the event timeouts
which slow down the shutdown of the server.
Now we actively `set()` all events in order to resume all those workers
when we stop the server.
The `ImDispatch.poll` signature has been changed too so it is possible
to change (via code) the hardcoded default. The function was using the
object referenced by `TIMEOUT` at the time the function was defined,
using `timeout None` then `if None: timeout=TIMEOUT` ensures we lookup
the variable.
closesodoo/odoo#64530
Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
The ir.autovacuum model purpose is to run several garbage collecting
operations like removing files from the filestore when no attachment
references them anymore.
The precedent strategy to register new garbage collection tasks was to
override the `power_on` method and to imperatively execute a vacuum
cleaning method on a given model. All calls were executed in a single
SQL transaction without any error handling, meaning a single fail during
any call resulted in a complete failure of the entire vacuum cleaning
chain.
We introduce a new `@autovacuum` api decorator, its purpose it to
register garbage collecting methods that will be safely executed in
their own transaction by the vacuum cleaner. In order to ensure this
new strategy is used, we deprecate `power_on` extensions.
By the way, garbage-collecting methods can be quite heavy and we don't
want users to directly call them. We now ensure they are private.
closesodoo/odoo#47842
Task: 2154079
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Olivier Dony <odo@odoo.com>
Multi is the default api for methods, it is not necessary to explicitly
decorate methods with it, adds clutter and most people use it because
they see that the rest of the code uses it.
Done with `find . -type f -name '*.py' | xargs sed -i '/@api.multi/d'`
When making a poll, an event is registered on each channel. Once
a notification is sent on a channel, only the events of the
notified channel are removed. A pointer to the event is kept
in all other channel the user subscribes to until those channel
are notified.
Since a user usually has many mail.channels but only a few active ones,
a new event and a bunch of pointers are created at each poll and
never removed.
This fix simply remove the pointers to the current event at the
end of a poll.
closesodoo/odoo#31215
Signed-off-by: Toufik Benjaa (tbe) <tbe@odoo.com>
This commit is a refactoring of im_status management.
This commit also add im_status in two places, near the author in a mail thread and on each suggestion when using mentions.
A service to manage im_status will have multiple benefits here:
-centralise information, avoid to call the server multiple time for the same im_status
-update all im_status at once and keep consistency in display.
With this commit, the im_status updates are now done by rpc call.
Updates where previously made with the bus but this has some drawback,
since the bus will only give the information 50 seconds after the beginning of
the request, in the worst case, we van wait 2*50 seconds to get an update.
More than that, the im_status where only updates for pinned dm_chat. Dm chat
are synchronized cross tabs, making the use of the bus possible for this purpose.
Since we will need to display im_status not linked to dm_chat, the list to update
will be different from tab to tab making the use of bus difficult for this purpose.
Technical notes:
-im_search has been moved from bus to mail addons since it concerns mail.channel
-Update of an im_status should be reflected everywhere in the page.
Since im_status is a rendered template used in multiple widget, we should add
the correct logic to all concerened widget. The current solution is simple:
use a jquery selector to find every place where im_status is rendered.
-we add a new im_status: im_partner. This will indicate that that
the partner has no user linked to him, making it possible to avoid to ask
for status updates for this partner.
-The update will only be done when the tab is focused. (and will be done
asap once the tab get focus back)
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
Running the bus garbage collector synchronously during the handling of a
request can stall the request for a very long time.
Instead, we add this step to the existing auto-vacuum scheduled job that
handles this kind of housecleaning. It brings another interesting bonus:
it can be scheduled outside of peak hours, which will avoid blocking other
bus-related transactions (the GC deletes a lot of rows and takes a lot
of exclusive locks in the database)
closesodoo/odoo#28326
Purpose of this commit is to give description more "business oriented"
because those descriptions appears in Odoo Studio which is supposed to be used by end users, not only by developers.
Related Task ID : 37311
From this commit onwards, Date fields will return datetime.date objects and Datetime fields will return datetime.datetime objects, this implies a number of things that are clearly explained both in the ORM API for master.
This commit also introduces a number of helper functions for dates and datetimes that are exposed in tools.date_utils and fields.Date[time], explained in the documentation as well.
Task-ID: 47189
P3 got rid of all __private attributes in the `threading` module,
via python/cpython@d06489945f.
Our old code for forcing the `daemon` attribute on an already started
thread used the mangled private name and does not work anymore on P3.
We need to use the new private attribute name (actually both,
to keep backwards-compatibility w/ P2)
This might have deserved a pycompat counterpart, but setting both
variants of the attribute works with no hassle. It should not be a very
frequent use case either.
Add a `peek` option in the bus event dispatcher polling in order to let
an external event dispatcher fetch the notifications and their
corresponding channels.
Cherry-pick of rev. f2f99dc091
Only start the bus events dispatcher when needed (lazy starting).
On Odoo PAAS it's never started because the platform has its own event
dispatcher.
Cherry-pick of rev. a2ed3d3d5b
Add a `peek` option in the bus event dispatcher polling in order to let
an external event dispatcher fetch the notifications and their
corresponding channels.
When you are connected as a portal user, you can access to the list of
company users when you try to create a channel of type "Direct Message".
This is caused by a SQL request that doesn't check the access rights of
the user before returning the list of users.
To fix this, we check that the user can create a mail channel, before
returning partners information.
opw-704127
Commit 5d1b323. re-enabled the user presence updates and notifications.
Unfortunately, it showed poor performance due to the high frequency of bus
notifications triggered on our instance (~600k users).
In this rev., we don't trigger notifications on presence changes anymore, but
we rather send the presence of users we have a DM open with, at the end of each
poll period. For performance reasons, we ensure not to do that more than once
every 30 seconds.
We also refined the detection of the 'away' status client side. 'Last presence'
timestamps are stored in the local storage, and the current inactivity period
is sent at each poll. Those presence timestamps now rely on browser activity
detection (click, keypress... events) rather than on the focus on Odoo tabs.
Finally, we removed the no more necessary cron introduced in 5d1b323.
Before this rev., no notification was sent on the bus when the user presence
changed. Thus, the bullets displayed in Discuss were never updated and stayed
as they were on the initialilization of the chat_manager (on webclient launch).
This rev. makes the bus.presence notifications work, and handles them client
side.
Moreover, the disconnections detection is now performed at each poll (with a
maximum of 1 per minute), instead of randomly (1/100 chance) at each poll, as
it scales better than the former solution. We also added a cron that performs
this check every 5 minutes. It is needed to detect that the last connected user
just disconnected (useful for visitors in the website, trying to talk with a
livechat operator).
Also, the 'away' status is now handled client side, as it makes more sense
that way (being away at each poll, e.g. every 50seconds, during 10 minutes
doesn't mean that we didn't come back between two polls).
Finaly, in bus.js, CrossTabBus, we moved the code writing in/reading the local
storage after the tab registration as this code depends on the fact that the
tab is the master tab or not (and this is known only once the tab is
registered).
This rev. was necessary in stable because the livechat uses the user status to
detect if there is an operator available, and this was often inaccurate.
Moreover, it improves the user experience of the chat in the backend.
If calling more than once `sendmany` (or `sendone`) on the same http request, the `NOTIFY imbus` will tell pertinent polling thread to
fetch their notifications in the bus table, which will not contain the new notifications, since they are not yet committed. For some reason, this
doesn't happen when calling only once `sendmany`. Committing the notifications breaks the tests since no rollback is possible.
So, `self._cr.commit()` should not be called in a test environement.
In this case, we cannot use `isinstance(self._cr, TestCursor)` since TestCursor is only avaiblable for http request in test mode. Here the cursor is a normal cursor, and to determine if we are in test mode, there is no other option than checking the config of the odoo server.
The stdlib version of the json library is more recent than the 3.5.3
version we are pinning in `requirements.txt`
There is no reason to use it.
Closes#6940
This commit add instant messaging features in mail module to make it a real team collaboration tool. Lots of these new features will replace Timeleine view (which is not removed in this commit). Many files have been moved, split or created for a better structure.
- My Channels : A user can be member of channel (mail.channel). These are discussion group, but also an aggregate of notification. Setting a channel as document follower, it will receive all the message (of the chatter) in real time, as information stream.
- A new Inbox : this aims to replace the timeline view. The user will see its channels grouped by 'slot', and receive the message in real time. Jump to the form view, invite people, create private discussion group, talk directly to another employee, ... are the main features. Your message can still be starred. The 'Inbox' will only contain your needaction.
- Needaction concept : a needaction is a message calling you to do something. If you follow a document as a person, the posted message will be 'needaction' for you. It can also be a message where you are mention with a '@my_name' (this feature is not added in this commit).
- The chatter doesn't change so much. It still allow you to see the discussion about the document, to star some message, and treat your needaction.
- Chat : the only way to have a minified conversation is from the Inbox Client Action. It allow you to keep an eye on a channel when working on document.
- Compose Message : it is now common to the Inbox, and the chatter. It offers shortcode substitution, attachments management, and optional subtype.
- UI : the chat window has been clean, they have a new look now.
To do so,
- Chatter code have been completely rewrite, independently of timeline view
- Add 'bus' as mail dependency, to insure real time
- A mail.message is broadcasted on the bus channel (db, mail.channel, channel_id), and a channel header on (db, res.partner, partner_id)
- Extracting mail_thread mixin used for Chatter and ChatMailThread (Inbox client action). This uniformize the programming structure with the server side
- Clean CSS style and use LESS instead of CSS
- Define a new messafe format (message_format method in mail_message.py), compatible with fetching and broadcasting
- Apply guideline coding conventions
This commit also break the module im_livechat, since im_chat models don't exists anymore and will be fixed in the next commit.
Thanks to Thibault Delavallee (tde) for the long debates and precious advices, Richard Mathot (rim) for the support, Simon Lejeune (sle) for the client action layout, and the Usability Team (apr, lle, bst yti) for the long testing.
Conflicts:
addons/mail/models/mail_thread.py
- im_chat.session will be replaced by mail.channel
- im_chat.shortode is renamed into mail.shortcode
- im_chat.presence is moved to bus module
- js and controller code is moved from im_chat to mail module
This commit only move files, and modify manifests, bundles, ... The code will be adapt in the next commits.
Migrate the models and controller to new API. Renaming xml id according to convention, renaming openerp tag into odoo tag. Add comment strings and documentations.
Ivan Yelizariev