Files
odoo_source/addons/google_calendar/utils/google_calendar.py
T
Thibault Francois 30f4be4889 [FIX] (google|microsoft)_calendar: make sync more resilient
Issue
-----
- Currently the cron can take too much time and timeout.
  In case of timeout process data are rollback.
  The sync per user also crash and stop the process for all user and make
  the transaction rollback

  In both case we cannot ensure that all user will end up synchronized

- The real time sync can end up in deadlock:
  If you create a holiday, it trigger a write on the user and the
  creation of an event that will be synchronize with google

  The synchronization may ask for a new token if this request fail
  a new cursor is create to empty on the user the refresh token.
  2 cursor that are trying to write on the same record => Deadlock

- Bad refresh token management
  When the refresh token is not anymore valid it's erased but
  the calendar_token and validity are kept. Is odoo consider
  the token valid it will try to sync an event with google|microsoft
  despite the refresh token is not valid anymore. Of course this lead to
  an error and the impossibility to create an event anymore without
  manually wiping the calendar_token and calendar_token_validity

- Error 401 are handled in context manager
(google|microsoft)_calendar_token, but get_*_calendar_token never raise
http error only user error.

Solution
---------

- Improve cron resilience
	- Commit between each user and rollback in case of issue for a user
	- Solve the issue when stop < start
        - Solve issue with rrule not being the first element in the list
          of recurrence
	- Improve perf when searching for organizer
	- In order to limit the volume during the first synchronization, we only
	synchronized event from x days in the past to x days in the future with
	x = 365 by default
	- Limit the number of event to synchronize to google each time to 200,
	since this operation is time consuming (1 request is sent per event)

- Avoid deadlock
  A new cursor is not needed, we only need to ensure that write on the
  user is done with a clean cursor and will be commited despite the error
  raise. So rollabck before the write and commit explicitly afterwards

- Erase properly all the token and the token validity if the refresh token is not valid anymore
	- For google calendar, erase also the token in case of 401
          error: Wrong client id, this mean the database parameter have been
          change and all the user should reconfigure the synchronization

- Directly handle error 401 with error 400 in refresh_token method

closes odoo/odoo#66250

X-original-commit: 486f69abad6226e5ce016ad89a0eeb103d709a3f
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2021-03-25 09:55:10 +00:00

114 lines
4.8 KiB
Python

# -*- coding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
from uuid import uuid4
import requests
import json
import logging
from odoo import fields
from odoo.tools import exception_to_unicode
from odoo.addons.google_calendar.utils.google_event import GoogleEvent
from odoo.addons.google_account.models.google_service import TIMEOUT
_logger = logging.getLogger(__name__)
def requires_auth_token(func):
def wrapped(self, *args, **kwargs):
if not kwargs.get('token'):
raise AttributeError("An authentication token is required")
return func(self, *args, **kwargs)
return wrapped
class InvalidSyncToken(Exception):
pass
class GoogleCalendarService():
def __init__(self, google_service):
self.google_service = google_service
@requires_auth_token
def get_events(self, sync_token=None, token=None, timeout=TIMEOUT):
url = "/calendar/v3/calendars/primary/events"
headers = {'Content-type': 'application/json'}
params = {'access_token': token}
if sync_token:
params['syncToken'] = sync_token
else:
# full sync, limit to a range of 1y in past to 1y in the futur by default
ICP = self.google_service.env['ir.config_parameter'].sudo()
day_range = int(ICP.get_param('google_calendar.sync.range_days', default=365))
_logger.info("Full cal sync, restricting to %s days range", day_range)
lower_bound = fields.Datetime.subtract(fields.Datetime.now(), days=day_range)
upper_bound = fields.Datetime.add(fields.Datetime.now(), days=day_range)
params['timeMin'] = lower_bound.isoformat() + 'Z' # Z = UTC (RFC3339)
params['timeMax'] = upper_bound.isoformat() + 'Z' # Z = UTC (RFC3339)
try:
status, data, time = self.google_service._do_request(url, params, headers, method='GET', timeout=timeout)
except requests.HTTPError as e:
if e.response.status_code == 410 and 'fullSyncRequired' in str(e.response.content):
raise InvalidSyncToken("Invalid sync token. Full sync required")
raise e
events = data.get('items', [])
next_page_token = data.get('nextPageToken')
while next_page_token:
params = {'access_token': token, 'pageToken': next_page_token}
status, data, time = self.google_service._do_request(url, params, headers, method='GET', timeout=timeout)
next_page_token = data.get('nextPageToken')
events += data.get('items', [])
next_sync_token = data.get('nextSyncToken')
default_reminders = data.get('defaultReminders')
return GoogleEvent(events), next_sync_token, default_reminders
@requires_auth_token
def insert(self, values, token=None, timeout=TIMEOUT):
url = "/calendar/v3/calendars/primary/events"
headers = {'Content-type': 'application/json', 'Authorization': 'Bearer %s' % token}
if not values.get('id'):
values['id'] = uuid4().hex
self.google_service._do_request(url, json.dumps(values), headers, method='POST', timeout=timeout)
return values['id']
@requires_auth_token
def patch(self, event_id, values, token=None, timeout=TIMEOUT):
url = "/calendar/v3/calendars/primary/events/%s" % event_id
headers = {'Content-type': 'application/json', 'Authorization': 'Bearer %s' % token}
self.google_service._do_request(url, json.dumps(values), headers, method='PUT', timeout=timeout)
@requires_auth_token
def delete(self, event_id, token=None, timeout=TIMEOUT):
url = "/calendar/v3/calendars/primary/events/%s" % event_id
headers = {'Content-type': 'application/json'}
params = {'access_token': token}
try:
self.google_service._do_request(url, params, headers=headers, method='DELETE', timeout=timeout)
except requests.HTTPError as e:
# For some unknown reason Google can also return a 403 response when the event is already cancelled.
if e.response.status_code not in (410, 403):
raise e
_logger.info("Google event %s was already deleted" % event_id)
#################################
## MANAGE CONNEXION TO GMAIL ##
#################################
def is_authorized(self, user):
return bool(user.sudo().google_calendar_rtoken)
def _get_calendar_scope(self, RO=False):
readonly = '.readonly' if RO else ''
return 'https://www.googleapis.com/auth/calendar%s' % (readonly)
def _google_authentication_url(self, from_url='http://www.odoo.com'):
return self.google_service._get_authorize_uri(from_url, service='calendar', scope=self._get_calendar_scope())
def _can_authorize_google(self, user):
return user.has_group('base.group_erp_manager')