As werkzeug.utils.unescape is deprecated in werkzeug 2.0 and the
markupsafe version of unescape just uses the built-in html.unescape [0],
we can use directly the same built-in method.
[0] pallets/markupsafe@c35603a903
Part-of: odoo/odoo#91927
Purpose
Fix bugs in link replacement (html and text version)
1/ The replacement of urls in text (not html) was buggy when
* a string includes multiple urls, and
* `base_url` is one of them and not the first
* Also in SMS marketing when adding the sms `id`
When it came to base_url's turn, the replace function for the content would
replace the `base_url` part of a previously shortened url instead of the
distinct `base_url` link.
2/ Ampersand character
A) The ampersand character prevented replacement of an url inside a `Markup`.
Previous tests passed as the tracker was created but the url was not replaced.
B) The ampersand character was not recognized as part of an url for simple
strings.
3/ Replacement of already short links
A faulty logic made it possible to replace "/r/" urls when no "blacklist" was
passed.
Commit
* Ensures that only the base_url link would be replaced instead in this case.
* Adds support for urls with "&"
* Does prevent shortening short urls
* Improves link conversion performance (regex + avoid duplication when there
are several occurrences of the same url)
* Adds multiple unit tests
Task-2783844
closesodoo/odoo#91164
X-original-commit: 61026806d831b49a7cf5c44ffef3d762986781ce
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This commit modifies most of the usages of read_group and uses
_read_group instead. _read_group doesn't join automatically on the
many2one fields when no order_by is specified, making it more performant
when the "name" of the many2one is not relevant, which is the case for
most back-end cases
closesodoo/odoo#84908
Task-id: 2479334
Related: odoo/enterprise#24877
Signed-off-by: Raphael Collet <rco@odoo.com>
In 2e3075693b06d71222b446bd8a805dacb639d0e4 we used a head request to
determine type of page before trying to get a title out of it.
But requests.head does not allow redirect by default:
https://docs.python-requests.org/en/v2.9.1/user/quickstart/#redirection-and-history
so if an URL was a redirect, we would not longer get the page title.
opw-2457640
closesodoo/odoo#84832
X-original-commit: 63346cf4b77f760121f88762ab2911c37603b548
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
PURPOSE
This commit consolidates UTM usage across all applications.
Global purpose is to avoid having undesired side-effects, such as unlinking an
utm.source/utm.medium/utm.campaign and at the same time cascading the deletion
to various records without noticing.
SPECS
ALLOW MORE PEOPLE TO CLEAN UTM RECORDS
Currently, not even the system administrator can delete utm.mediums and
utm.sources (he can only delete campaigns).
These were considered as "technical records", but allowing some cleanup is
a good idea since these records are often automatically generated and can
create a lot of unnecessary noise in the database.
That's why we now allow the following groups to delete all UTM records
(sources, mediums and campaigns):
- group_system
- group_mass_mailing_user
- group_social_manager (enterprise)
PREVENT DELETION
For some use cases, removing an utm.source/utm.medium/utm.campaign would
cascade delete the related record, which was unintended / hidden side effect.
These combinations were secured by preventing to unlink:
- mailing.mailing source_id field
Trying to delete the utm.source will throw an error message
- mailing.mailing medium_id field
Trying to delete the utm.medium will throw an error message
- hr.recruitment.source source_id field
Trying to delete the utm.source will throw an error message
ADDING CLEAN ERROR MESSAGES
When trying to delete an UTM record that is linked with ondelete="restrict", we
improved the error message to give a clear explication to the user, e.g:
"You can't delete these UTM sources as they are linked to the following
mailings in the Mass Mailing APP, and deleting the source would break the
statistics: Newsletter"
SPECIFY 'ondelete' strategy
For a lot of uses of sources/mediums/campaigns, the 'ondelete' strategy was not
specified, leading to the confusion of "is this really how we want to handle
this?".
A lot of ondelete="set null" have been added in various field definitions to
ensure that this is the desired and logical strategy we want for that
specific model.
PREVENT REMOVING HARDCODED UTM RECORDS
In some functional flows, UTM records are hardcoded using their direct
record reference.
This is notably the case for the recruitment process and its creation of
aliases, and for the Email / SMS Marketing flows.
As deleting them would break these flows, we prevent their deletion in a
"api.ondelete" method.
ENFORCE NEW RULES WITH TESTS
A lot of python tests have been added to make sure we enforce the decisions
taken here above.
LINKS
ENT PR odoo/enterprise#19048
Task-2459480
closesodoo/odoo#72239
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Bug
===
If you send a URL (which will be transformed into a link tracker) to
an external website (e.g. in mass mailing, social, ...), the UTM values
will be added in the GET parameters of the URL.
The external website might crash if those parameters are not supported
by it.
We want to add a system parameter `link_tracker.no_external_tracking`
to be able to not add those UTM values in the redirected URL.
This is done only for external website, because it will never be an
issue if the URL redirect to Odoo.
Task-2657413
closesodoo/odoo#78616
X-original-commit: c8363b7bb74ace4e77b593efc3a9104143240141
Signed-off-by: awa-odoo <awa-odoo@users.noreply.github.com>
- Create 3 websites and set a website domain for each
- Install the website_links module
- Go to Website > Configuration > Settings
- Select the website 3
- Go to website
- Menu Promote > Link tracking
The URL field uses the domain you are logged into the database
rather than the domain of the active website
The expected behavior is:
With an ICP set to URL1, a website 1 set to URL2 and a website 2 set to
URL3, going to the link tracker website page from URL3 but with website
1 selected should:
Show the current website domain as placeholder of the input -> URL2
Create link for that current website the user selected -> URL2
opw-2638494
closesodoo/odoo#77705
X-original-commit: d094d76354d6f46db3dc7f8e716b5f778ca26569
Signed-off-by: agr-odoo <agr-odoo@users.noreply.github.com>
Currently, when we try to shorten the links from content with help of
`_shorten_links_text` method, it throws traceback if the body(content)
we pass is `False`(might happen when we directly pass a model field of
textual type but the field doesn't have a value yet). It's because
the `re` expects the content to be string / bytestring.
This commit fixes the issue by adding a check in this method, which
will simply return `False` if there's no body.
Task-2628586
closesodoo/odoo#76041
X-original-commit: 3cc5217f1196108d5c7a52202054263a480963b2
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Make link tracker handle other types than many2one for tracking fields.
Currently, when a field of type char is returned, an error occurs.
odoo/addons/utm/models/utm_mixin.py already handles other types than
many2one for tracking_fields(). This aligns to that behaviour in the
link tracker.
opw-2494686
closesodoo/odoo#71780
X-original-commit: a2068f9696917d7fa4848bc09d05eced84d0cc2f
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When trying to filter links by a tracked URL code, the following warning
is shown:
WARNING dbname odoo.models: Cannot execute name_search, no _rec_name
defined on link.tracker.code
and the applied filter gives all records.
closesodoo/odoo#71398
X-original-commit: 3687dabbc373e57e24ca4596c18ef1dacfde8273
Signed-off-by: agr-odoo <agr-odoo@users.noreply.github.com>
Purpose
=======
Add a unicity constraint (on the url, UTM campaign, medium and source).
As we must care about null values, we can not implement it in SQL
(possible in pure SQL, but we will need to create an index on the
columns, which is not possible in Odoo if we want to customize the
error message).
Clean the code, before the "search or create" behavior was implemented
in the create, causing strange behavior (creating a link tracker with
the same values as an existing one didn't raise an error, but open
the form view of the existing link tracker).
Task-2090344
closesodoo/odoo#48845
Related: odoo/enterprise#9608
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
`href` is `href="...`, so if the input is a Markup object it will get
escaped on `html.replace`, which means it will not actually match the
text we're looking for.
Therefore mark it as `Markup`, because that's essentially what it
is. This should have no effect if `html` is an `str`.
link tracker tries to get title from HTML, but the url may be not an html page
or too big html to process. It's a waste of bandwidth, but may also lead to a
Server Memory Limit error.
As a solution, make HEAD request and don't proceed to GET request if it's not an
html page or it's too big. Also, limit page downloading to 50KB.
STEPS:
- Have a standard database with link_tracker and mass_mailing.
- Create a new mass mail MM
- Add a link to a large file in MM Mail Body
- Click "SEND"
- Go to Settings / Technical / Automation / Scheduled Actions
- Open the "Process Mass Mailing Queue" or "Email Marketing: Process queue"
- Click "RUN MANUALLY"
---
opw-2457640
closesodoo/odoo#67948
X-original-commit: bdec63cf97e6904604e9f1e0bf3b6f453b6add60
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Ivan Yelizariev // IEL <yelizariev@users.noreply.github.com>
/web/action/load is the public controller that Should be used by the
webclient to fetch actions
_for_xml_id is the default access method on actions that implements
fields filtering to avoid leaking server action code or other
information not needed by the webclient
Implementing whitelist of fields that can be access per model
In this commit we correctly use mailing variable inside a loop
on self to enable multi-mode on convert_links method.
We also update link_tracker shortening methods docstrings. We
add some explanation, notably about parameters. A variable is
fixed to better indicate parameter content (html -> content).
closesodoo/odoo#55452
X-original-commit: 3e1633d5b3590ec5657b851c21cbae059944d964
Related: odoo/enterprise#12218
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Provide a follow-up/overview of the Mailing 24 hours after its been sent.
Specifications
==============
Send an email to the responsible of the mailing 24 hours the last email
of the mass mailing.
During the link trackers creation, extract the button label if exists
(so we can display them in the statistics email).
Technical remarks
=================
The statistics email is sent to the responsible of each mailing in the
CRON of mass mailing.
Task-2227411
closesodoo/odoo#49836
Related: odoo/upgrade#1094
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Clarify the use case in which a specified email address is set as replies may
not be traceable. Make the %Click KPIs visible for Emails as well. Allow users
to report on the links that have (or not) been clicked from their
mailing.
SPECIFICATIONS
Display list of link trackers linked to the mailing on 'click' stat button
click.
Task ID: 2202495
PR #48339
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Several counter fields are performed inefficiently. They use compute methods
using length of 2many fields instead of making use of more efficient ORM
methods like read_group.
SPECIFICATIONS
There may be a lot of clicks linked to a given link tracker. Compute the count
of clicks in batch with a read_group to improve performance.
Same for utm campaign mailing count performances.
Some missing indexes are added to speedup queries.
LINKS
Task ID 2196182
PR #44662
Co-Authored-By: Victor Feyens <vfe@odoo.com>
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
PURPOSE
Move links shortening tools on mail.render.mixin to have rendering and
post procesing tools available on that mixin.
SPECIFICATIONS
Move html and text links shortening methods on mail.render.mixin class.
That way rendering and post processing is done in that mixin instead of
being specific to link.tracker model. Indeed those methods are tools and
not really business related methods.
Future commits will improve the way mail and sms composer use the render
mixin. This commit is therefore both a cleaning and preparing commit.
LINKS
Task ID 1963529
Community PR odoo/odoo#32397
URLs with parameters were not correctly supported by the regular
expression.
For example, URLs with parameters were truncated before the '?'.
This patch adds support for a wider range of URLs, and checks in a test
that the parameters are correctly handled.
closesodoo/odoo#39762
X-original-commit: 616e145635eac06a0a44ab3d899f4a9e707b3ba8
Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>
The link tracker model (link.tracker) contained favicon (store)
and icon_src computed fields. Computing favicon took
{3-6.5%} of installing odoo (with demo data) due to syncronous http
call to a external link (google).
Moreover, these fields had no sense to be store in odoo
and only used in one template. This template was modified to
display the same information (icon) as before but without storing fields
and with a lazy loading of icons.
TASK_ID: 2083715
closesodoo/odoo#38405
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This method being needed in the new social app, it made sense to move it to
the link_tracker module to make it available for mass_mailing_sms and social
to avoid unnecessary duplicated code.
LINKS:
closesodoo/odoo#34973
Taskid: 2045210
Pr: 37143
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This commit cleans up the utm.campaign form view:
* Don't display mailing related things if no mailings on the campaign or
if the campaigns were not activated in mass_mailing settings;
* Stat button for the mass mailings was wrong, it was displaying the number
of mails instead of the number of mailings;
* Remove unused global click ratio field all other fields/methods related
to its computation;
* Remove ab testing cta on campaign form;
* Add groups on mailing related fields to avoid access errors;
LINKS
Task ID 2074813 (FP request UTM refactor)
PR: #37185
Having a global click ratio on the campaign was pointless considering the
nature of the social.app and its posts. Therefore, we decided to have a
global click count which gives a better idea of the popularity of the campaign
itself.
Task ID 2074813 (FP request UTM refactor)
PR: #37185
This branch is the combination of several optimizations in the ORM:
* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;
* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;
* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);
* make method `modified` take advantage of inverse fields to inverse
dependencies;
* filter records by evaluating a domain on records in Python;
* a computed field with `readonly=False` behaves like a normal field
with an onchange method;
* computed fields are computed in superuser mode by default.
Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.
closesodoo/odoo#35659
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
PURPOSE
This commit removes the mass_mailing.campaign model. Instead of having a fully
fledged model, we will simply inherit utm.campaign. We will also add relevant
statistics on utm campaign model in order to use it in various applications.
SPECIFICATIONS
This commit removes the mass_mailing.campaign model. Instead of having a fully
fledged model, we will simply inherit utm.campaign. This change implies that
mass_mailing.tag and mass_mailing.stage have to move to the utm model along
their associated views/data.
These changes were made so that campaigns could be used in the future
by social, mass_mailing and mass_sms and available in the same view
This commit also removes the source_id and the medium_id
fields on the campaign.
This commit also moves the unique_ab_testing field from the mass_mailing_campaign
to the mass_mailing model
Task ID: 2002029
PR: #34015
Multi is the default api for methods, it is not necessary to explicitly
decorate methods with it, adds clutter and most people use it because
they see that the rest of the code uses it.
Done with `find . -type f -name '*.py' | xargs sed -i '/@api.multi/d'`
Before this commit, ordering by `create_date DESC` and `write_date DESC` could
lead to undeterministic order if multiple records had the exact same date.
This is the case during tests as `crate_date` is set to transaction start time.
(That's what NOW() in PSQL does)
Now, we enforce that order to avoid randomness.
Related to #31829
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Purpose of this commit is to remove some unnecessary code in link tracker.
* remove unnecessary code / parameters:
* remove unused field click_date on click as create_date is already used for
that purpose;
This commit is linked to task ID 1904277 and PR #28242.
In this commit we rewrite a bit add_click in order to remove the inlined sudo
and ease inheritance and parameter management inside the method.
Access through controllers is sudo-ed as the main API method is now done
with current user access rights.
This commit is linked to task ID 1904277 and PR #28242.
Purpose is
* to clean a bit views;
* to improve link.tracker form view to have information better displayed;
* to remove unused views;
This commit is linked to task ID 1896681 and PR #27975.
This was not correctly done when splitting the module at fd5361c0f7.
Future commits will clean a bit the code in order to better understand how
this module works.
This commit is linked to task ID 1896681 and PR #27975. No functional change
should occur with this commit as it contains only renaming / move.
This module still holds old code and deserves some love. This commit cleans a
bit the init and manifest files as well as renaming view file according to
guidelines.
This commit is linked to task ID 1896681 and PR #27975. No functional change
should occur with this commit as it contains only renaming / move.
When using only mass mailing two issues arise when trying to send mass
mailings containing links
* a link.tracker is created for each link to shorten; however rights are
given currently only in website_links for website designers and not to
marketing users that use mass mailing. This commit adds the rights for
link.tracker model to those people;
* a link.tracker.code is created to hold the shortened versions of the
links. As it is a technical model it is now done as superuser as rights
on link.tracked model has already been granted and adding rights on such
a technical model is not necessary;
A test is added that checks what is done with links when sending mass
mailings. It helped spotting the hereabove issue.
This commit is linked to task ID 1889703 and PR #27526.
Purpose of this commit is to give description more "business oriented"
because those descriptions appears in Odoo Studio which is supposed to be used by end users, not only by developers.
Related Task ID : 37311
This commit factorizes the link.tracker.click values computation out of
the add_click method. This method is called when customers click on links
in email that are tracked in Odoo.
Purpose of this commit is
* to effectively put mass mailing related code into mass mailing.
Currently link_tracker holds code related to mass mailing which is
not in its dependency tree. Let us therefore try to be a bit modular
and add a proper inheritance in mass mailing;
* allow addons to inherit from the click values prepare method to add
specific values depending on their behavior like mass mailing adding
links to a mass mailing and a mass mailing campaign;
- Create a mass mailing including a 'tel' or 'sms' link, e.g.
`<a href="tel:555-000-0000">555-000-0000</a>`
- When sent, the link is converted to a tracked linked (`http://...`)
We do not want to convert these links, they should be handled by the
browser directly.
opw-765885
Rémy Voet (ryv)