Translated fields no longer use the model ir.translation. Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table. The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages). Empty text is allowed in translation values, but not NULL.
Here are examples for a field with translate=True:
NULL
{"en_US": "Foo"}
{"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
{"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}
Like before, writing False to the field makes it NULL, i.e., False in all
languages. However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.
Here are examples for a field with translate=xml_translate:
NULL
{"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}
Change for callable(translate) fields: one can now write any value in any
language on such a field. The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values. Basically the
structure of the value must remain the same in all languages, like before.
Reading a translated field is now both simpler and faster than the former
implementation. We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:
SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...
The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages). For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.
Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language. Before this change, only the source value of the
field ('en_US') could be indexed.
Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages. We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.
Code translations are no longer stored into the database. They become static,
and are extracted from the PO files when needed. The worker simply uses a cache
with extracted code translations for performance. This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker. Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).
Performance summary:
(+) reading 'model' translated fields is faster
(+) reading 'model_terms' translated fields is much faster (no need to inject
translations into the source value)
(+) searching translated fields with operator 'ilike' is much faster when the
field is indexed with 'trigram'
(+) updating translated fields requires less ORM flushing
(-) importing translations from PO files is 2x slower
Some extra fixes:
- make field 'name' of ir.actions.actions translated; because of the PG
inheritance, this is necessary to make the column definition consistent in
all models that inherit from ir.actions.actions.
- add some backend API for the web/website client for editing translations
- move methods get_field_string() to model ir.model.fields
- move _load_module_terms to model ir.module.module
- adapt tests in test_impex, test_new_api
- because env.lang is injected into SQL queries, its returned value is
now guaranteed to correspond to a valid active language or None
- remove wizard to insert missing translations (no longer makes sense)
task-id: 2081307
Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
446 lines
18 KiB
Python
446 lines
18 KiB
Python
# -*- coding: utf-8 -*-
|
|
# Part of Odoo. See LICENSE file for full copyright and licensing details.
|
|
|
|
import base64
|
|
import json
|
|
import logging
|
|
import requests
|
|
from werkzeug.exceptions import Forbidden
|
|
|
|
from odoo import http, tools, _
|
|
from odoo.addons.iap.tools import iap_tools
|
|
from odoo.exceptions import AccessError
|
|
from odoo.http import request
|
|
|
|
_logger = logging.getLogger(__name__)
|
|
|
|
|
|
class MailPluginController(http.Controller):
|
|
|
|
@http.route('/mail_client_extension/modules/get', type="json", auth="outlook", csrf=False, cors="*")
|
|
def modules_get(self, **kwargs):
|
|
"""
|
|
deprecated as of saas-14.3, not needed for newer versions of the mail plugin but necessary
|
|
for supporting older versions
|
|
"""
|
|
return {'modules': ['contacts', 'crm']}
|
|
|
|
@http.route('/mail_plugin/partner/enrich_and_create_company',
|
|
type="json", auth="outlook", cors="*")
|
|
def res_partner_enrich_and_create_company(self, partner_id):
|
|
"""
|
|
Route used when the user clicks on the create and enrich partner button
|
|
it will try to find a company using IAP, if a company is found
|
|
the enriched company will then be created in the database
|
|
"""
|
|
|
|
partner = request.env['res.partner'].browse(partner_id).exists()
|
|
|
|
if not partner:
|
|
return {'error': _("This partner does not exist")}
|
|
|
|
if partner.parent_id:
|
|
return {'error': _("The partner already has a company related to him")}
|
|
|
|
normalized_email = partner.email_normalized
|
|
if not normalized_email:
|
|
return {'error': _('The email of this contact is not valid and we can not enrich it')}
|
|
|
|
company, enrichment_info = self._create_company_from_iap(normalized_email)
|
|
|
|
if company:
|
|
partner.write({'parent_id': company})
|
|
|
|
return {
|
|
'enrichment_info': enrichment_info,
|
|
'company': self._get_company_data(company),
|
|
}
|
|
|
|
@http.route('/mail_plugin/partner/enrich_and_update_company', type='json', auth='outlook', cors='*')
|
|
def res_partner_enrich_and_update_company(self, partner_id):
|
|
"""
|
|
Enriches an existing company using IAP
|
|
"""
|
|
partner = request.env['res.partner'].browse(partner_id).exists()
|
|
|
|
if not partner:
|
|
return {'error': _("This partner does not exist")}
|
|
|
|
if not partner.is_company:
|
|
return {'error': 'Contact must be a company'}
|
|
|
|
normalized_email = partner.email_normalized
|
|
if not normalized_email:
|
|
return {'error': 'The email of this contact is not valid and we can not enrich it'}
|
|
|
|
domain = tools.email_domain_extract(normalized_email)
|
|
iap_data = self._iap_enrich(domain)
|
|
|
|
if 'enrichment_info' in iap_data: # means that an issue happened with the enrichment request
|
|
return {
|
|
'enrichment_info': iap_data['enrichment_info'],
|
|
'company': self._get_company_data(partner),
|
|
}
|
|
|
|
phone_numbers = iap_data.get('phone_numbers')
|
|
|
|
partner_values = {}
|
|
|
|
if not partner.phone and phone_numbers:
|
|
partner_values.update({'phone': phone_numbers[0]})
|
|
|
|
if not partner.iap_enrich_info:
|
|
partner_values.update({'iap_enrich_info': json.dumps(iap_data)})
|
|
|
|
if not partner.image_128:
|
|
logo_url = iap_data.get('logo')
|
|
if logo_url:
|
|
try:
|
|
response = requests.get(logo_url, timeout=2)
|
|
if response.ok:
|
|
partner_values.update({'image_1920': base64.b64encode(response.content)})
|
|
except Exception:
|
|
pass
|
|
|
|
model_fields_to_iap_mapping = {
|
|
'street': 'street_name',
|
|
'city': 'city',
|
|
'zip': 'postal_code',
|
|
'website': 'domain',
|
|
}
|
|
|
|
# only update keys for which we dont have values yet
|
|
partner_values.update({
|
|
model_field: iap_data.get(iap_key)
|
|
for model_field, iap_key in model_fields_to_iap_mapping.items() if not partner[model_field]
|
|
})
|
|
|
|
partner.write(partner_values)
|
|
|
|
partner.message_post_with_view(
|
|
'iap_mail.enrich_company',
|
|
values=iap_data,
|
|
subtype_id=request.env.ref('mail.mt_note').id,
|
|
)
|
|
|
|
return {
|
|
'enrichment_info': {'type': 'company_updated'},
|
|
'company': self._get_company_data(partner),
|
|
}
|
|
|
|
@http.route(['/mail_client_extension/partner/get', '/mail_plugin/partner/get']
|
|
, type="json", auth="outlook", cors="*")
|
|
def res_partner_get(self, email=None, name=None, partner_id=None, **kwargs):
|
|
"""
|
|
returns a partner given it's id or an email and a name.
|
|
In case the partner does not exist, we return partner having an id -1, we also look if an existing company
|
|
matching the contact exists in the database, if none is found a new company is enriched and created automatically
|
|
|
|
old route name "/mail_client_extension/partner/get is deprecated as of saas-14.3, it is not needed for newer
|
|
versions of the mail plugin but necessary for supporting older versions, only the route name is deprecated not
|
|
the entire method.
|
|
"""
|
|
|
|
if not (partner_id or (name and email)):
|
|
return {'error': _('You need to specify at least the partner_id or the name and the email')}
|
|
|
|
if partner_id:
|
|
partner = request.env['res.partner'].browse(partner_id).exists()
|
|
return self._get_contact_data(partner)
|
|
|
|
normalized_email = tools.email_normalize(email)
|
|
if not normalized_email:
|
|
return {'error': _('Bad Email.')}
|
|
|
|
# Search for the partner based on the email.
|
|
# If multiple are found, take the first one.
|
|
partner = request.env['res.partner'].search(['|', ('email', 'in', [normalized_email, email]),
|
|
('email_normalized', '=', normalized_email)], limit=1)
|
|
|
|
response = self._get_contact_data(partner)
|
|
|
|
# if no partner is found in the database, we should also return an empty one having id = -1, otherwise older versions of
|
|
# plugin won't work
|
|
if not response['partner']:
|
|
response['partner'] = {
|
|
'id': -1,
|
|
'email': email,
|
|
'name': name,
|
|
'enrichment_info': None,
|
|
}
|
|
company = self._find_existing_company(normalized_email)
|
|
|
|
can_create_partner = request.env['res.partner'].check_access_rights('create', raise_exception=False)
|
|
|
|
if not company and can_create_partner: # create and enrich company
|
|
company, enrichment_info = self._create_company_from_iap(normalized_email)
|
|
response['partner']['enrichment_info'] = enrichment_info
|
|
response['partner']['company'] = self._get_company_data(company)
|
|
|
|
return response
|
|
|
|
@http.route('/mail_plugin/partner/search', type="json", auth="outlook", cors="*")
|
|
def res_partners_search(self, search_term, limit=30, **kwargs):
|
|
"""
|
|
Used for the plugin search contact functionality where the user types a string query in order to search for
|
|
matching contacts, the string query can either be the name of the contact, it's reference or it's email.
|
|
We choose these fields because these are probably the most interesting fields that the user can perform a
|
|
search on.
|
|
The method returns an array containing the dicts of the matched contacts.
|
|
"""
|
|
normalized_email = tools.email_normalize(search_term)
|
|
|
|
if normalized_email:
|
|
filter_domain = [('email_normalized', 'ilike', search_term)]
|
|
else:
|
|
filter_domain = ['|', '|', ('display_name', 'ilike', search_term), ('ref', '=', search_term),
|
|
('email', 'ilike', search_term)]
|
|
|
|
# Search for the partner based on the email.
|
|
# If multiple are found, take the first one.
|
|
partners = request.env['res.partner'].search(filter_domain, limit=limit)
|
|
|
|
partners = [
|
|
self._get_partner_data(partner)
|
|
for partner in partners
|
|
]
|
|
return {"partners": partners}
|
|
|
|
@http.route(['/mail_client_extension/partner/create', '/mail_plugin/partner/create'],
|
|
type="json", auth="outlook", cors="*")
|
|
def res_partner_create(self, email, name, company):
|
|
"""
|
|
params email: email of the new partner
|
|
params name: name of the new partner
|
|
params company: parent company id of the new partner
|
|
"""
|
|
# old route name "/mail_client_extension/partner/create is deprecated as of saas-14.3,it is not needed for newer
|
|
# versions of the mail plugin but necessary for supporting older versions
|
|
# TODO search the company again instead of relying on the one provided here?
|
|
# Create the partner if needed.
|
|
partner_info = {
|
|
'name': name,
|
|
'email': email,
|
|
}
|
|
|
|
#see if the partner has a parent company
|
|
if company and company > -1:
|
|
partner_info['parent_id'] = company
|
|
partner = request.env['res.partner'].create(partner_info)
|
|
|
|
response = {'id': partner.id}
|
|
return response
|
|
|
|
@http.route('/mail_plugin/log_mail_content', type="json", auth="outlook", cors="*")
|
|
def log_mail_content(self, model, res_id, message, attachments=None):
|
|
"""Log the email on the given record.
|
|
|
|
:param model: Model of the record on which we want to log the email
|
|
:param res_id: ID of the record
|
|
:param message: Body of the email
|
|
:param attachments: List of attachments of the email.
|
|
List of tuple: (filename, base 64 encoded content)
|
|
"""
|
|
if model not in self._mail_content_logging_models_whitelist():
|
|
raise Forbidden()
|
|
|
|
if attachments:
|
|
attachments = [
|
|
(name, base64.b64decode(content))
|
|
for name, content in attachments
|
|
]
|
|
|
|
request.env[model].browse(res_id).message_post(body=message, attachments=attachments)
|
|
return True
|
|
|
|
@http.route('/mail_plugin/get_translations', type="json", auth="outlook", cors="*")
|
|
def get_translations(self):
|
|
return self._prepare_translations()
|
|
|
|
def _iap_enrich(self, domain):
|
|
"""
|
|
Returns enrichment data for a given domain, in case an error happens the response will
|
|
contain an enrichment_info key explaining what went wrong
|
|
"""
|
|
enriched_data = {}
|
|
try:
|
|
response = request.env['iap.enrich.api']._request_enrich({domain: domain}) # The key doesn't matter
|
|
except iap_tools.InsufficientCreditError:
|
|
enriched_data['enrichment_info'] = {'type': 'insufficient_credit', 'info': request.env['iap.account'].get_credits_url('reveal')}
|
|
except Exception:
|
|
enriched_data["enrichment_info"] = {'type': 'other', 'info': 'Unknown reason'}
|
|
else:
|
|
enriched_data = response.get(domain)
|
|
if not enriched_data:
|
|
enriched_data = {'enrichment_info': {'type': 'no_data', 'info': 'The enrichment API found no data for the email provided.'}}
|
|
return enriched_data
|
|
|
|
def _find_existing_company(self, email):
|
|
"""Find the company corresponding to the given domain and its IAP cache.
|
|
|
|
:param email: Email of the company we search
|
|
:return: The partner corresponding to the company
|
|
"""
|
|
search = self._get_iap_search_term(email)
|
|
|
|
partner_iap = request.env["res.partner.iap"].sudo().search([("iap_search_domain", "=", search)], limit=1)
|
|
|
|
if partner_iap:
|
|
return partner_iap.partner_id
|
|
|
|
return request.env["res.partner"].search([("is_company", "=", True), ("email_normalized", "=ilike", "%" + search)], limit=1)
|
|
|
|
def _get_company_data(self, company):
|
|
if not company:
|
|
return {'id': -1}
|
|
|
|
fields_list = ['id', 'name', 'phone', 'mobile', 'email', 'website']
|
|
|
|
company_values = dict((fname, company[fname]) for fname in fields_list)
|
|
company_values['address'] = {'street': company.street,
|
|
'city': company.city,
|
|
'zip': company.zip,
|
|
'country': company.country_id.name if company.country_id else ''}
|
|
company_values['additionalInfo'] = json.loads(company.iap_enrich_info) if company.iap_enrich_info else {}
|
|
company_values['image'] = company.image_1920
|
|
|
|
return company_values
|
|
|
|
def _create_company_from_iap(self, email):
|
|
domain = tools.email_domain_extract(email)
|
|
iap_data = self._iap_enrich(domain)
|
|
if 'enrichment_info' in iap_data:
|
|
return None, iap_data['enrichment_info']
|
|
|
|
phone_numbers = iap_data.get('phone_numbers')
|
|
emails = iap_data.get('email')
|
|
new_company_info = {
|
|
'is_company': True,
|
|
'name': iap_data.get("name") or domain,
|
|
'street': iap_data.get("street_name"),
|
|
'city': iap_data.get("city"),
|
|
'zip': iap_data.get("postal_code"),
|
|
'phone': phone_numbers[0] if phone_numbers else None,
|
|
'website': iap_data.get("domain"),
|
|
'email': emails[0] if emails else None
|
|
}
|
|
|
|
logo_url = iap_data.get('logo')
|
|
if logo_url:
|
|
try:
|
|
response = requests.get(logo_url, timeout=2)
|
|
if response.ok:
|
|
new_company_info['image_1920'] = base64.b64encode(response.content)
|
|
except Exception as e:
|
|
_logger.warning('Download of image for new company %s failed, error %s', new_company_info.name, e)
|
|
|
|
if iap_data.get('country_code'):
|
|
country = request.env['res.country'].search([('code', '=', iap_data['country_code'].upper())])
|
|
if country:
|
|
new_company_info['country_id'] = country.id
|
|
if iap_data.get('state_code'):
|
|
state = request.env['res.country.state'].search([
|
|
('code', '=', iap_data['state_code']),
|
|
('country_id', '=', country.id)
|
|
])
|
|
if state:
|
|
new_company_info['state_id'] = state.id
|
|
|
|
new_company_info.update({
|
|
'iap_search_domain': self._get_iap_search_term(email),
|
|
'iap_enrich_info': json.dumps(iap_data),
|
|
})
|
|
|
|
new_company = request.env['res.partner'].create(new_company_info)
|
|
|
|
new_company.message_post_with_view(
|
|
'iap_mail.enrich_company',
|
|
values=iap_data,
|
|
subtype_id=request.env.ref('mail.mt_note').id,
|
|
)
|
|
|
|
return new_company, {'type': 'company_created'}
|
|
|
|
def _get_partner_data(self, partner):
|
|
|
|
fields_list = ['id', 'name', 'email', 'phone', 'mobile', 'is_company']
|
|
|
|
partner_values = dict((fname, partner[fname]) for fname in fields_list)
|
|
partner_values['image'] = partner.image_128
|
|
partner_values['title'] = partner.function
|
|
partner_values['enrichment_info'] = None
|
|
|
|
try:
|
|
partner.check_access_rights('write')
|
|
partner.check_access_rule('write')
|
|
partner_values['can_write_on_partner'] = True
|
|
except AccessError:
|
|
partner_values['can_write_on_partner'] = False
|
|
|
|
if not partner_values['name']:
|
|
# Always ensure that the partner has a name
|
|
name, email = request.env['res.partner']._parse_partner_name(
|
|
partner_values['email'])
|
|
partner_values['name'] = name or email
|
|
|
|
return partner_values
|
|
|
|
def _get_contact_data(self, partner):
|
|
"""
|
|
method used to return partner related values, it can be overridden by other modules if extra information have to
|
|
be returned with the partner (e.g., leads, ...)
|
|
"""
|
|
if partner:
|
|
partner_response = self._get_partner_data(partner)
|
|
if partner.company_type == 'company':
|
|
partner_response['company'] = self._get_company_data(partner)
|
|
elif partner.parent_id:
|
|
partner_response['company'] = self._get_company_data(partner.parent_id)
|
|
else:
|
|
partner_response['company'] = self._get_company_data(None)
|
|
else: # no partner found
|
|
partner_response = {}
|
|
|
|
return {
|
|
'partner': partner_response,
|
|
'user_companies': request.env.user.company_ids.ids,
|
|
'can_create_partner': request.env['res.partner'].check_access_rights(
|
|
'create', raise_exception=False),
|
|
}
|
|
|
|
def _mail_content_logging_models_whitelist(self):
|
|
"""
|
|
Returns all models that emails can be logged to and that can be used by the "log_mail_content" method,
|
|
it can be overridden by sub modules in order to whitelist more models
|
|
"""
|
|
return ['res.partner']
|
|
|
|
def _get_iap_search_term(self, email):
|
|
"""Return the domain or the email depending if the domain is blacklisted or not.
|
|
|
|
So if the domain is blacklisted, we search based on the entire email address
|
|
(e.g. asbl@gmail.com). But if the domain is not blacklisted, we search based on
|
|
the domain (e.g. bob@sncb.be -> sncb.be)
|
|
"""
|
|
domain = tools.email_domain_extract(email)
|
|
return ("@" + domain) if domain not in iap_tools._MAIL_DOMAIN_BLACKLIST else email
|
|
|
|
def _translation_modules_whitelist(self):
|
|
"""
|
|
Returns the list of modules to be translated
|
|
Other mail plugin modules have to override this method to include their module names
|
|
"""
|
|
return ['mail_plugin']
|
|
|
|
def _prepare_translations(self):
|
|
lang = request.env['res.users'].browse(request.uid).lang
|
|
translations_per_module = request.env["ir.http"].get_translations_for_webclient(
|
|
self._translation_modules_whitelist(), lang)[0]
|
|
translations_dict = {}
|
|
for module in self._translation_modules_whitelist():
|
|
translations = translations_per_module.get(module, {})
|
|
messages = translations.get('messages', {})
|
|
for message in messages:
|
|
translations_dict.update({message['id']: message['string']})
|
|
return translations_dict
|