Improve link tracker tests :
* add missing test for trace tracking in mass mailing. We generate a trace
and find a short link in it, in order to test the tracking in mass
mailing;
* add tests for side effects of mass mailing routes: update of trace status,
clicks, ...
* wrap base url locally instead of globally in MockTracker class to ease
specific setup in classes;
Prepares Task-2150462 (Mass Mailing: Unsubscribe flow improvement)
closesodoo/odoo#94660
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
As werkzeug.utils.unescape is deprecated in werkzeug 2.0 and the
markupsafe version of unescape just uses the built-in html.unescape [0],
we can use directly the same built-in method.
[0] pallets/markupsafe@c35603a903
Part-of: odoo/odoo#91927
Purpose
Fix bugs in link replacement (html and text version)
1/ The replacement of urls in text (not html) was buggy when
* a string includes multiple urls, and
* `base_url` is one of them and not the first
* Also in SMS marketing when adding the sms `id`
When it came to base_url's turn, the replace function for the content would
replace the `base_url` part of a previously shortened url instead of the
distinct `base_url` link.
2/ Ampersand character
A) The ampersand character prevented replacement of an url inside a `Markup`.
Previous tests passed as the tracker was created but the url was not replaced.
B) The ampersand character was not recognized as part of an url for simple
strings.
3/ Replacement of already short links
A faulty logic made it possible to replace "/r/" urls when no "blacklist" was
passed.
Commit
* Ensures that only the base_url link would be replaced instead in this case.
* Adds support for urls with "&"
* Does prevent shortening short urls
* Improves link conversion performance (regex + avoid duplication when there
are several occurrences of the same url)
* Adds multiple unit tests
Task-2783844
closesodoo/odoo#91164
X-original-commit: 61026806d831b49a7cf5c44ffef3d762986781ce
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Commit "[IMP] core: don't save visitor default session" moved the geoip
from the session to the request with a deprecation warning. This commit
adapts the remaining modules to use `request.geoip` instead of
`request.session.geoip`.
Geoip is always set on the request but it can be an empty dictionnary in
case the geolocalization failed.
closesodoo/odoo#86015
Task: 2789035
Related: odoo/enterprise#25192
Signed-off-by: Julien Castiaux <juc@odoo.com>
This commit modifies most of the usages of read_group and uses
_read_group instead. _read_group doesn't join automatically on the
many2one fields when no order_by is specified, making it more performant
when the "name" of the many2one is not relevant, which is the case for
most back-end cases
closesodoo/odoo#84908
Task-id: 2479334
Related: odoo/enterprise#24877
Signed-off-by: Raphael Collet <rco@odoo.com>
In 2e3075693b06d71222b446bd8a805dacb639d0e4 we used a head request to
determine type of page before trying to get a title out of it.
But requests.head does not allow redirect by default:
https://docs.python-requests.org/en/v2.9.1/user/quickstart/#redirection-and-history
so if an URL was a redirect, we would not longer get the page title.
opw-2457640
closesodoo/odoo#84832
X-original-commit: 63346cf4b77f760121f88762ab2911c37603b548
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Purpose
=======
Avoid counting requests from social bots (twitter, facebook, linkedin...)
when tracking a link.
Specifications
=============
Social media platforms have a specific user agent in the HTTP headers that
can be used to detect them and to not increment the click count in that case.
Task-2578902
closesodoo/odoo#78806
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
This commit consolidates UTM usage across all applications.
Global purpose is to avoid having undesired side-effects, such as unlinking an
utm.source/utm.medium/utm.campaign and at the same time cascading the deletion
to various records without noticing.
SPECS
ALLOW MORE PEOPLE TO CLEAN UTM RECORDS
Currently, not even the system administrator can delete utm.mediums and
utm.sources (he can only delete campaigns).
These were considered as "technical records", but allowing some cleanup is
a good idea since these records are often automatically generated and can
create a lot of unnecessary noise in the database.
That's why we now allow the following groups to delete all UTM records
(sources, mediums and campaigns):
- group_system
- group_mass_mailing_user
- group_social_manager (enterprise)
PREVENT DELETION
For some use cases, removing an utm.source/utm.medium/utm.campaign would
cascade delete the related record, which was unintended / hidden side effect.
These combinations were secured by preventing to unlink:
- mailing.mailing source_id field
Trying to delete the utm.source will throw an error message
- mailing.mailing medium_id field
Trying to delete the utm.medium will throw an error message
- hr.recruitment.source source_id field
Trying to delete the utm.source will throw an error message
ADDING CLEAN ERROR MESSAGES
When trying to delete an UTM record that is linked with ondelete="restrict", we
improved the error message to give a clear explication to the user, e.g:
"You can't delete these UTM sources as they are linked to the following
mailings in the Mass Mailing APP, and deleting the source would break the
statistics: Newsletter"
SPECIFY 'ondelete' strategy
For a lot of uses of sources/mediums/campaigns, the 'ondelete' strategy was not
specified, leading to the confusion of "is this really how we want to handle
this?".
A lot of ondelete="set null" have been added in various field definitions to
ensure that this is the desired and logical strategy we want for that
specific model.
PREVENT REMOVING HARDCODED UTM RECORDS
In some functional flows, UTM records are hardcoded using their direct
record reference.
This is notably the case for the recruitment process and its creation of
aliases, and for the Email / SMS Marketing flows.
As deleting them would break these flows, we prevent their deletion in a
"api.ondelete" method.
ENFORCE NEW RULES WITH TESTS
A lot of python tests have been added to make sure we enforce the decisions
taken here above.
LINKS
ENT PR odoo/enterprise#19048
Task-2459480
closesodoo/odoo#72239
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Bug
===
If you send a URL (which will be transformed into a link tracker) to
an external website (e.g. in mass mailing, social, ...), the UTM values
will be added in the GET parameters of the URL.
The external website might crash if those parameters are not supported
by it.
We want to add a system parameter `link_tracker.no_external_tracking`
to be able to not add those UTM values in the redirected URL.
This is done only for external website, because it will never be an
issue if the URL redirect to Odoo.
Task-2657413
closesodoo/odoo#78616
X-original-commit: c8363b7bb74ace4e77b593efc3a9104143240141
Signed-off-by: awa-odoo <awa-odoo@users.noreply.github.com>
- Create 3 websites and set a website domain for each
- Install the website_links module
- Go to Website > Configuration > Settings
- Select the website 3
- Go to website
- Menu Promote > Link tracking
The URL field uses the domain you are logged into the database
rather than the domain of the active website
The expected behavior is:
With an ICP set to URL1, a website 1 set to URL2 and a website 2 set to
URL3, going to the link tracker website page from URL3 but with website
1 selected should:
Show the current website domain as placeholder of the input -> URL2
Create link for that current website the user selected -> URL2
opw-2638494
closesodoo/odoo#77705
X-original-commit: d094d76354d6f46db3dc7f8e716b5f778ca26569
Signed-off-by: agr-odoo <agr-odoo@users.noreply.github.com>
We clean various graph archs taking into consideration that:
- the default type of a graph is "bar".
- a bar chart is by default stacked.
- the field attributes type="row" and type="col" does not make sense for
a graph view (since its implementation was separated from the pivot
implementation a long time ago))
- the boolean attributes should now take 1 or 0 as value (but the other
values are accepted for retrocompatibility).
Part-of: odoo/odoo#76065
Currently, when we try to shorten the links from content with help of
`_shorten_links_text` method, it throws traceback if the body(content)
we pass is `False`(might happen when we directly pass a model field of
textual type but the field doesn't have a value yet). It's because
the `re` expects the content to be string / bytestring.
This commit fixes the issue by adding a check in this method, which
will simply return `False` if there's no body.
Task-2628586
closesodoo/odoo#76041
X-original-commit: 3cc5217f1196108d5c7a52202054263a480963b2
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The license is missing in most enterprise manifest so
the decision was taken to make it explicit in all cases.
When not defined, a warning will be triggered starting from
14.0 when falling back on the default LGPL-3.
closesodoo/odoo#74245
Related: odoo/design-themes#48
Related: odoo/enterprise#19862
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This branch adds request.redirect on all requests.
In case of a front end request, we do an url_for to the location.
We removed redirect_with_hash that was only for retro compatibility
local_redirect has been renamed to redirect_query, and param keep_hash has been
removed and moved.
Default code for redirect is 303 now instead of 302.
Now redirect and redirect_query make local redirect by default, you need to
pass local=False to make external redirect.
All werkeug.utils.redirect has been replaced by request.redirect.
Http.redirect now use an http.Response type, and it become easy to add an
override like 'set_cookies' e.g.
Dispatch of a website.page return an http.response too, so we first need to
check if it is a cached version before to check if it is an Odoo Response.
Migrate your code:
http.redirect -> request.redirect(location, code, local)
http.local_redirect -> request.redirect_query(location, query, code, local)
http.redirect_with_hash -> request.redirect
Courtesy of odony for help and review ;)
closesodoo/odoo#72599
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Make link tracker handle other types than many2one for tracking fields.
Currently, when a field of type char is returned, an error occurs.
odoo/addons/utm/models/utm_mixin.py already handles other types than
many2one for tracking_fields(). This aligns to that behaviour in the
link tracker.
opw-2494686
closesodoo/odoo#71780
X-original-commit: a2068f9696917d7fa4848bc09d05eced84d0cc2f
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
When trying to filter links by a tracked URL code, the following warning
is shown:
WARNING dbname odoo.models: Cannot execute name_search, no _rec_name
defined on link.tracker.code
and the applied filter gives all records.
closesodoo/odoo#71398
X-original-commit: 3687dabbc373e57e24ca4596c18ef1dacfde8273
Signed-off-by: agr-odoo <agr-odoo@users.noreply.github.com>
Purpose
=======
Add a unicity constraint (on the url, UTM campaign, medium and source).
As we must care about null values, we can not implement it in SQL
(possible in pure SQL, but we will need to create an index on the
columns, which is not possible in Odoo if we want to customize the
error message).
Clean the code, before the "search or create" behavior was implemented
in the create, causing strange behavior (creating a link tracker with
the same values as an existing one didn't raise an error, but open
the form view of the existing link tracker).
Task-2090344
closesodoo/odoo#48845
Related: odoo/enterprise#9608
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
`href` is `href="...`, so if the input is a Markup object it will get
escaped on `html.replace`, which means it will not actually match the
text we're looking for.
Therefore mark it as `Markup`, because that's essentially what it
is. This should have no effect if `html` is an `str`.
PURPOSE
Purpose is to have more tests when sending sms mailings, notably about
canceled or failed mails or sms as well as jinja and links rendering.
SPECIFICATIONS
In this commit we improve SMS Marketing mailing tests. We notably
* add tests for void and invalid email and numbers. It allows to check they
correctly update their trace status;
* add tests for unsubscribe and view links embedded in sms marketing;
* add tests to simulate a click on links sent through sms marketing and
ensure click statistics are effectively updated;
* ensure content of sent sms are checked;
Default content of mailings used in test is updated to ensure jinja is
correctly rendered, including links and some corner cases. This will also
helps ensuring behavior is kept when converting to QWeb.
In event we now correctly use sms gateway mock to ensure SMS finding.
In link_tracker a helper is added to check url tracking in plaintext content.
LINKS
Task ID-2508643
Followup of odoo/odoo#68874 (improve mail tests)
Prepares Task ID-27033 (support QWeb in templates)
Prepares Task ID-2377974 (clean trace and status management in mass mailing)
COM PR odoo/odoo#69461
ENT PR odoo/enterprise#17780
X-original-commit: f1182b9da9aca9e67518fc10c240f3b21895a7ab
Some incoming tests crash if parser is not the HTML parser when trying to find
a node given its ID. We also remove an unnecessary encapsulation, allowing
to avoid issues like embedding a body or head tag within a div.
Task ID-2508643
Prepares Task ID-27033 (support QWeb in templates)
Prepares Task ID-2377974 (clean trace and status management in mass mailing)
COM PR odoo/odoo#69461
ENT PR odoo/enterprise#17780
X-original-commit: a442057f161610e7869d0863acd067cbadd18195
link tracker tries to get title from HTML, but the url may be not an html page
or too big html to process. It's a waste of bandwidth, but may also lead to a
Server Memory Limit error.
As a solution, make HEAD request and don't proceed to GET request if it's not an
html page or it's too big. Also, limit page downloading to 50KB.
STEPS:
- Have a standard database with link_tracker and mass_mailing.
- Create a new mass mail MM
- Add a link to a large file in MM Mail Body
- Click "SEND"
- Go to Settings / Technical / Automation / Scheduled Actions
- Open the "Process Mass Mailing Queue" or "Email Marketing: Process queue"
- Click "RUN MANUALLY"
---
opw-2457640
closesodoo/odoo#67948
X-original-commit: bdec63cf97e6904604e9f1e0bf3b6f453b6add60
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Ivan Yelizariev // IEL <yelizariev@users.noreply.github.com>
Purpose of the task, is to prevent users from inadvertently
creation/opening countries from country_id fields.
Countries should be managed from their dedicated menu item.
so in this commit, we have set both no_open and no_create to True
so user should not update and create a country from the many2X fields.
closesodoo/odoo#63773
Taskid: 2241677
Related: odoo/enterprise#15464
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
/web/action/load is the public controller that Should be used by the
webclient to fetch actions
_for_xml_id is the default access method on actions that implements
fields filtering to avoid leaking server action code or other
information not needed by the webclient
Implementing whitelist of fields that can be access per model
PURPOSE
When displaying an email in a list the mail client (gmail, outlook...) computes
a preview based on the content. This preview is generally not well computed as
it contains a lot of garbage and tags while it should contain only relevant
text.
SPECIFICATIONS
To build this preview, all mail clients read the content of the email.
The only way to be able to customize the preview is to add an invisible
HTML element at the beginning of the email with the wanted preview text.
We add at the end of the preview `‌` (zero-width non-joiner) to
fill the end of the preview in order to not have the beginning of the
mail at the end of the preview. It doesn't work with simple space as
the mail clients trim each HTML element content.
Task ID-2172125
PR #49886
In this commit we correctly use mailing variable inside a loop
on self to enable multi-mode on convert_links method.
We also update link_tracker shortening methods docstrings. We
add some explanation, notably about parameters. A variable is
fixed to better indicate parameter content (html -> content).
closesodoo/odoo#55452
X-original-commit: 3e1633d5b3590ec5657b851c21cbae059944d964
Related: odoo/enterprise#12218
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This commit sets attribute sample="1" in a bunch of list and
kanban views, to enable the new sample data feature when views are
empty.
Task 2232801
X-original-commit: 5471309b35619f307242c2704c7681ba63d2be42
Purpose
=======
Provide a follow-up/overview of the Mailing 24 hours after its been sent.
Specifications
==============
Send an email to the responsible of the mailing 24 hours the last email
of the mass mailing.
During the link trackers creation, extract the button label if exists
(so we can display them in the statistics email).
Technical remarks
=================
The statistics email is sent to the responsible of each mailing in the
CRON of mass mailing.
Task-2227411
closesodoo/odoo#49836
Related: odoo/upgrade#1094
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Clarify the use case in which a specified email address is set as replies may
not be traceable. Make the %Click KPIs visible for Emails as well. Allow users
to report on the links that have (or not) been clicked from their
mailing.
SPECIFICATIONS
Display list of link trackers linked to the mailing on 'click' stat button
click.
Task ID: 2202495
PR #48339
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This commit adapts several views to make them use newly defined
widgets, the new decoration-xxx mechanism on fields, and to adapt
them to the new design of buttons.
Part of task 2195254
Purpose
=======
We should not be able to duplicate the link trackers (as it
makes no sense from a functionnal POV).
Task-2090344
closesodoo/odoo#50303
X-original-commit: c236b976f6a1bfa92fa6cd6325f652eec461c805
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
URPOSE
Prepare code cleaning and optimization in mail, mass_mailing and SMS by
cleaning models for readability and code complexity and footprint reduction.
SPECIFICATIONS
Add tool methods and asserts for link tracker tests.
Add tool methods and assets for mass mailing tests, notably about mailing
traces and their link with emails.
Merge existing and duplicated tests and rewrite them a bit.
Rename and improve test_mass_mailing models.
LINKS
Prepares task ID 2238597 (notification and trace models cleaning)
Task ID 1906925 (mass mailing tests cleaning)
PR #50169
Related: odoo/enterprise#10180
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
PURPOSE
Several counter fields are performed inefficiently. They use compute methods
using length of 2many fields instead of making use of more efficient ORM
methods like read_group.
SPECIFICATIONS
There may be a lot of clicks linked to a given link tracker. Compute the count
of clicks in batch with a read_group to improve performance.
Same for utm campaign mailing count performances.
Some missing indexes are added to speedup queries.
LINKS
Task ID 2196182
PR #44662
Co-Authored-By: Victor Feyens <vfe@odoo.com>
Co-Authored-By: Thibault Delavallee <tde@odoo.com>
PURPOSE
Move links shortening tools on mail.render.mixin to have rendering and
post procesing tools available on that mixin.
SPECIFICATIONS
Move html and text links shortening methods on mail.render.mixin class.
That way rendering and post processing is done in that mixin instead of
being specific to link.tracker model. Indeed those methods are tools and
not really business related methods.
Future commits will improve the way mail and sms composer use the render
mixin. This commit is therefore both a cleaning and preparing commit.
LINKS
Task ID 1963529
Community PR odoo/odoo#32397
Without demo data, for the odoo-master transifex project
closesodoo/odoo#41935
X-original-commit: dab7670b73506fb3a835695ee3bd735e0c5e5c2b
Related: odoo/enterprise#7287
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Followup of a425695e
The terms were back in 12.0
Courtesy of Juan José Scarafía
closesodoo/odoo#41624
X-original-commit: 85d0c7001a997748d7691205bbb8d066597591a5
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
URLs with parameters were not correctly supported by the regular
expression.
For example, URLs with parameters were truncated before the '?'.
This patch adds support for a wider range of URLs, and checks in a test
that the parameters are correctly handled.
closesodoo/odoo#39762
X-original-commit: 616e145635eac06a0a44ab3d899f4a9e707b3ba8
Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>
The link tracker model (link.tracker) contained favicon (store)
and icon_src computed fields. Computing favicon took
{3-6.5%} of installing odoo (with demo data) due to syncronous http
call to a external link (google).
Moreover, these fields had no sense to be store in odoo
and only used in one template. This template was modified to
display the same information (icon) as before but without storing fields
and with a lazy loading of icons.
TASK_ID: 2083715
closesodoo/odoo#38405
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Romeo Fragomeli