Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment
By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).
There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).
We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.
To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.
This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
(for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
in order to see only one at once
- a floating select input to switch visibility of a particular logical
branching
Task-27033
X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
Purpose
=======
Purpose of this commit is to add a new group for the mail template designer.
Goal is to make roles clearer: managers edit templates, users use them. This
commit allow some designers / managers to make email template and to let
others users use those email templates.
Specifications
==============
When this feature is enabled in the Settings page, a new group is required to
modify email templates in a composer like wizard or to make dynamic content.
This allows to separate managers editing / composing templates from standard
users that use them.
If the current does not have this group, the email body will be in readonly
mode if he selected an email template. That way we force him to use the email
template that the manager made.
Technical
=========
New Group
---------
Only users in this group will be able to create / write email template or
to write Jinja code in the mail composer (including other fields like subject
in mailing).
By default, all internal users have this group. Mass mailing users also have
this group as writing mailings is about the same management level as writing
templates.
Mail Composer Mixin
-------------------
In comment mode, the template is rendered and then saved on the body field
so non-"Mail Template Editor" users can load email templates.
But in mass mode, the body of the template is saved and then rendered and
many things change the body (HTML sanitizer, web editor move inline CSS
properties, add / remove spaces...). So in this case, we can not know if
the user changed the body or not. That is why we put the body field in
readonly mode so, it is not modified by the web editor.
Jinja code detection
--------------------
To detect dynamic Jinja content, we compile the template, and we browse the
AST. If we do not have a single "Template Data" node, we assume that the
template is dynamic.
When we detect the template as static, we do not render it. That way we
avoid unnecessary rendering.
Code cleaning
-------------
Move Jinja import into tools so that it is outside of mail framework code.
Task-2187263
closesodoo/odoo#75840
Related: odoo/enterprise#20547
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When calling send on SMS, sent SMS are unlinked. If a ``delete_all`` parameter
is true, failed SMS are also unlinked.
In this commit we allow to control that behavior with two boolean, one for
sent SMS and one for failed SMS. This allows to ask to keep sent SMS and
update their status accordingly.
More control is necessary notably to enable IAP feedback on sent SMS as
an error could happen after considering it as sent. Some flows will have to
be updated to decide whether this status is necessary and if SMS are kept.
This commit prepares ground for that feature by already improving methods
and API.
Task-2634957
Prepares Task-2535005 (SMS view pimp) and Task-2560666 (IAP feedback)
PR odoo/odoo#75798
Only SMS not in error should be considered as sent instead of being
unlinked. This has no effect on current tests but will be used in
future cleaning.
Task-2634957
Prepares Task-2535005 (SMS view pimp) and Task-2560666 (IAP feedback)
PR odoo/odoo#75798
PURPOSE
Prepare code cleaning and optimization in mail, mass_mailing and SMS by
cleaning models for readability and code complexity and footprint reduction.
SPECIFICATIONS
Be aligned with mail_mail and mass_mailing naming where failure_type is used
instead of error_code.
LINKS
Task ID-2377974
Community PR odoo/odoo#61467
Enterprise PR odoo/enterprise#14633
Upgrade PR odoo/upgrade#1907
PURPOSE
Purpose is to have more tests when sending sms mailings, notably about
canceled or failed mails or sms as well as jinja and links rendering.
SPECIFICATIONS
In this commit we improve SMS Marketing mailing tests. We notably
* add tests for void and invalid email and numbers. It allows to check they
correctly update their trace status;
* add tests for unsubscribe and view links embedded in sms marketing;
* add tests to simulate a click on links sent through sms marketing and
ensure click statistics are effectively updated;
* ensure content of sent sms are checked;
Default content of mailings used in test is updated to ensure jinja is
correctly rendered, including links and some corner cases. This will also
helps ensuring behavior is kept when converting to QWeb.
In event we now correctly use sms gateway mock to ensure SMS finding.
In link_tracker a helper is added to check url tracking in plaintext content.
LINKS
Task ID-2508643
Followup of odoo/odoo#68874 (improve mail tests)
Prepares Task ID-27033 (support QWeb in templates)
Prepares Task ID-2377974 (clean trace and status management in mass mailing)
COM PR odoo/odoo#69461
ENT PR odoo/enterprise#17780
X-original-commit: f1182b9da9aca9e67518fc10c240f3b21895a7ab
In this commit we backport some of 14.1+ improvements done in mail tools
in order to keep a coherent definition through sub versions. We also
improve docstring and add some explanations on available toold and asserts.
Task ID-2500615
COM PR odoo/odoo#68874
X-original-commit: d44c47697389866603f28ff2d5da60a97574ec5f
As no global rule is defined for system, people belonging to both system
and a functional group may be limited in their rights about sms templates.
For example install event_sms -> admin is member of system and event manager
groups. He cannot edit templates other than related to event.
With this commit members of system may write, create or unlink all templates
independently from their functinal groups.
Task ID-2495426
Followup of odoo/odoo#64626closesodoo/odoo#68535
X-original-commit: e0c1563993da918a0fbc2e31a13f3fe6a7ea2916
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
sms.template model has several record rules to give access to templates
linked to models managed by certain groups (like crm.lead for sales managers)
These record rules were meant to restrict access to certain model to create,
write and unlink, but not read.
This is leading to issues when trying to read a template on other models.
Indeed people should always be able read sms.template content.
Unit test were also added to the sms module to ensure that a member of
group_user can always read a sms template.
Unit test is added to ensure admin always has full control on sms.templates.
Task ID-2191254
COM PR odoo/odoo#68445
ENT PR odoo/enterprise#17340
X-original-commit: 6a00157f79be30a6efd6d039504c57be07c449fd
Fix a crash when calling assertSMSStatistics without a number but taking the
number from a partner.
Also fix "sent" sms helper, content should be optional like other helpers.
Task ID 2247037
Community PR #50384
Modules holding tests and helpers
* link_tracker: mainly mock, asserts and tools for link tracker tests
(MockLinkTracker);
* mail: mainly gateway mock and base for mail tests
* MockEmail -> mocks for mail gateway;
* MailCase -> tools and asserts for mail tests;
* MailCommon -> base for mail functional tests);
* sms: mainly SMS gateway mock and base for sms tests
* MockSMS -> mocks for SMS gateway;
* SMS Case -> tools and asserts for mail / SMS tests;
* SMSCommon -> update of MailCommon with SMS capabilities);
* mass_mailing: mainly asserts and tools for mass mailing tests
* MassMailCase -> update of MailCase for mass mailing tools and asserts;
* MassMailCommon -> update of MailCommon with mass mailing);
* mass_mailing_sms: mainly asserts and tools for mass SMS tests
* MockMassSMS -> update of MockSMS for mass SMS tools and asserts;
* MassSMSCommon -> update of MassMailCommon with SMS capabilities);
Modules for tests
* test_mail: module for mail app tests (TestMailCommon);
* test_mass_mailing: module for mass mailing app tests (TestMassMailCommon);
* test_mail_full: tests integrating all discuss features, currently mainly
mail and SMS (TestMailFullCommon);
Enterprise: update test_mail_enterprise and test_marketing_automation
Task ID 2247037
Community PR odoo/odoo#50384
Enterprise PR odoo/enterprise#10266
Upgrade PR odoo/upgrade#1122
Since January 2020, users are required to validate their IAP account via
SMS code validation. This new behaviour needed to be properly ported to
the client to correctly inform the user.
This commit adds support of a new failure_type: 'Unregistered Account'.
Task #2209567closesodoo/odoo#48512
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When no valid phone number was found on a record, and when using _message_sms
directly, no SMS and no notification was created. You could therefore think
everything was ok while it was actually not.
Instead we now create failed notifications. It allows to be notified of it
and fix numbers through cancel / resend wizards.
Task ID 2244192
closesodoo/odoo#50486
X-original-commit: baecdfaa2d31714a20d1fb8af7799af1805cd939
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When no valid phone number was found on a record, and when using the composer
in single recipient mode, you had a crash at sending as composer tried to
write on a field called False.
Instead we just take the first available phone field of the record. As they
are all void we can update them safely.
Task ID 2244192
X-original-commit: 2306d7d79d8e64597912f15365cede7473589731
PURPOSE
SMS are a powerful marketing tool. For instance it is perfect to announce a
sale or to communicate a coupon code, to welcome a new customer in a fidelity
program, ...
Purpose of this task is to integrate SMS sending in batch in mass mailing. It
will use same mailing objects but sending SMS instead of emails. Some metrics
and flows will have to be slightly updated at the same time.
SPECIFICATIONS
Clean the use and options of sms composer after FP feedback :
* see https://s.nimbusweb.me/share/3167586/ap1xpy95rz29a5cys576 as basis;
* globally, do not display invalid recipients, only valid / invalid count
as well as current selection / active domain counts;
* consider logging a note as default behavior when using the composer;
* simplify code: when doing a mass SMS, send SMS and attach a simple note
to the document;
Some other points
* make name of sms templates translatable;
* improve various wording, notably in sms widget;
* display error code in sms list view;
* update sms composer actions accordingly by correctly setting active id
or ids;
LINKS
Task 1997464
PR #34424
Original SMS addition: Task 1922163 (4287481)
PURPOSE
SMS are a powerful marketing tool. For instance it is perfect to announce a
sale or to communicate a coupon code, to welcome a new customer in a fidelity
program, ...
Purpose of this task is to integrate SMS sending in batch in mass mailing. It
will use same mailing objects but sending SMS instead of emails. Some metrics
and flows will have to be slightly updated at the same time.
SPECIFICATIONS
Purpose of this commit is to add a blacklist mechanism for phone numbers
used to send SMS like what already exists for email addresses when sending
emails.
Define a new phone.blacklist model, holding a number and the state of the
blacklist (active field), as well as tools methods to access it. Make it
as private as possible, accessing it in sudo once access are granted.
Also clean phone validation tools: lessen number of tool functions and update
caller to simplify code readability. Some fixes are also included in this
commit, notably blank spaces cleaning in phone numbers.
Improve phone.validation.mixin to add a tool method computing a sanitized
number, in addition to formatting it to national / international.
Define a new mail.thread.phone mixin computing the blacklist status of a
record. This mixin
* inherit from phone.validation.mixin in order to have access to some
base phone number parsing capabilities;
* computes a sanitized phone number based on ´´_phone_get_number_fields´´.
It takes first sanitized value, trying each field returned by the
method. That means one sanitized phone number is available per record
even if several fields are available;
* compute blacklist state of records. It is based on phone.blacklist
model and give an easy-to-use field and API to manipulate blacklisted
records;
* give some API methods :
* ``_phone_set_blacklisted``: set recordset as blacklisted;
* ``_phone_reset_blacklisted``: reactivate recordset (even if not blacklisted
this method can be called safely);
Put menus in technical in order to have access to it. Add a Phone / SMS
menu below "Email" and use it to store SMS / Phone actions.
Finally prepare tests addition by performing some light cleaning while adding
blacklist tests. Purpose is to ease future tests related to SMS.
LINKS
Task 1997464
PR #34424
Original SMS addition: Task 1922163 (4287481)
PURPOSE
Followup of merge 4287481 .
SPECIFICATIONS
wrong_format_number is actually now called wrong_number_format. This commit
propagates this renaming through SMS code and tests.
LINKS
Task 1922187
closesodoo/odoo#35025
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Purpose of this commit is to add options and improve sms composer behavior.
Followup of merge 4287481bf0 .
SPECIFICATIONS
* mass mode: add an option to keep archives when doing mass sms. This mode
is actually a _message_sms in batch using the note subtype to speedup the
process;
* improve _message_sms_schedule_mass to allow more fine-tuning of options
when calling it;
* do not block sending SMS in batch if some recipients are invalid. Indeed
using notifications there will be traces of failed SMS;
* avoid reload of form view;
Linked to task 1925950 and 1935280
Part of PR #34864
Purpose of this commit is to provide users templates to use when sending
SMS. It is inspired from what already exists for mail templates. This commit
* adds templates for SMS
* users can now create template for SMS Text messages similar to mail
templates;
* jinja syntax is supported for body like mail templates, which is why
templates are linked to a given model;
* refactor the SMS composer
* templates are supported in composer like the message composer. This is
supported only in mass mode to avoid bloating the interface in standard
composition mode;
* composer code is rewritten to support various use cases (comment, mass
sms, numbers, ...) and better fits all use cases;
* composer is extended to support notably mass SMS without posting messages;
Templates will also be used in a near future in mass sms sending (mass
mailing application improvement) and in marketing automation (for enterprise).
Tests are updated and added to ensure feature works.
Related to task 1922163
Linked to PR #33510
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
Co-Authored-By: Pierre Rousseau <pro@odoo.com>
Purpose of this commit is to better include SMS notifications when posting a
message. SMS is now just another way of notifying people along with Inbox and
email. Following recent mail merge improving notification mechanism [1] we
have to define a _notify_record_by_sms method on mail.thread.
When a message_post is done using message_type being ``sms`` notification type
of customers is set to sms. Customers can be computed on model (generally based
on partner_id field) or directly set usign partner°ids. Notification model is
updated to store this information directly inside the notification itself.
An new ``_message_sms`` helper method is introduced in SMS module allowing
to send messages using sms type and notification with a reduced parameters
number. It is just a shortcut to message_post, easier to use. Either it
computes default recipients on the record set, either it is based on given
partners and numbers to notify.
The following use cases are notably supported
* default computation: find customer, notify by sms;
* force recipients to notify by sms (partner_ids);
* give a set of numbers to notify by sms (sms_nubmers), not necessarily
linked to existing partners;
* force number / customer relationship independently of mobile number defined
on customer (for example when sending an SMS directly from a mobile field
on a lead linked to a customer);
Tests are updated accordingly. Performance tests are added in order to have
some insights on queries generated when sending SMS, like already done for
mail.thread alone.
Related to task 1922163
Linked to PR #33510
[1] see be27955136: performance and notification code improvements
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
Co-Authored-By: Pierre Rousseau <pro@odoo.com>
Purpose of this commit is to improve and add tools, methods and data to test
mail and SMS features. We also add tests for current implementation of SMS
feature, allowing to better understand future changes.
[ADD] test_mail_full
* have a module depending on mail sub-applications like sms or snailmail.
Its purpose is to check that standard mail features work effectively with
all overrides and extra behaviors activated;
* add a new model specific for SMS gateway, with default recipient
computation;
* add SMS tests for SMS module adding SMS capabilities linked to mail
feature. This commit tests SMS feature before the upcoming refactoring
and improvement of SMS module in community: posting with SMS and sms
composer usage;
In test_mail
* add necessary mobile information on test partners;
* improve assertBusNotification that was not correctly asserting all items
in message of bus notifications;
In sms
* add mock for SMS sending. Purpose is to mock the connection to IAP
services by mocking the call to IAP server. It allows to perform SMS
tests without having to contact (and pay) for this service;
* allow some customization when calling the SMS gateway mock to simulate
errors and test corner cases;
Related to task 1922163
Linked to PR #34516
Nicolas Bayet