We have our own html2plaintext, already used in lot of use cases instead of
just a few for the html2txt library.
Notably for emails: most emails going through Odoo stack use our simple
html2plaintext to format the body alternative. When no body alternative
is given to ``build_email`` an alternative is built using the library to
remove. Using our own parser allows to have the same results compared to
using ``MailMail.send()``. Difference lies in spaces and new lines as well
as markdown. Our html2plaintext is a bit simple and does not try to generate
Markdown but generates a simple plaintext version.
This also helps solving some issues with depending on that library.
Task-2702034
closesodoo/odoo#82486
X-original-commit: b3b9627b655cd7cb928925affed6cc8d92661e8d
Related: odoo/enterprise#23364
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Mako is not used anymore for a long time.
closesodoo/odoo#78781
X-original-commit: fb9f89afbc7a22e82309150617e8b5de5c995ff9
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
With the release of Debian Bullseye the time has come for the balancing
act by trying to update the requirements.
The constraints are the following:
* Stick as close as possible to python3-* Debian packages versions
of the current Debian stable.
* Same but for the Ubuntu LTS version.
* When one of the above package is patched by Debian or Ubuntu
maintainers, set the upstream version that includes the patch if any.
Also, as support for python < 3.7 is dropped, some cleanup can be done.
The `reportlab / pillow` combo is a special case:
* Pillow has to be updated to 8.1.2 as this version includes the
security patches that were added to Ubuntu package 7.0.0 (Focal).
* Reportlab crashes with 8.1.2 with version prior to 3.5.54 [0].
The problem does not occur on Ubuntu Focal as both versions from
the Ubuntu packaging are compatible.
So the reportlab 3.5.59 is chosen as it's the Debian Bullseye version
and to avoid multiple lines for a few minor versions.
[0] https://hg.reportlab.com/hg-public/reportlab/rev/0cf382dab63b
X-original-commit: 794677fb6a3391379200eb2144a6ed372e89c17a
Part-of: odoo/odoo#78781
PURPOSE
=======
We want to be able to authenticate our servers with a certificate
for the entire domain name instead of using a username and a password.
SPECIFICATIONS
==============
Add 2 fields on the `ir.mail_server`, which are
- the SSL certificate
- the SSL private key
When we uploaded both files, we use them to authenticate the client of
the SSL connection.
Add 2 options on the Odoo binary, so we can provide the filenames of both
files (like we do for the SMTP username/password).
SETTINGS
========
Note that this type of authentication doesn't work locally for Microsoft
office 365. It seems like Microsoft is blocking non-static IP address
(not able to ping the host locally, but it works on the server).
The host name of the server is defined in the MX DNS record. Then, on
Office 365 you must create an SMTP relay based on a certificate and not
based on a hard coded IP address. The certificate must be valid for your
domain name.
e.g.
Host: openerp-org.mail.protection.outlook.com
Port: 25
Username: <keep it blank>
Password: <keep it blank>
Security: STARTTLS
Email: admin@odoobe.com
New Python dependence
=====================
The standard SSL python library only takes a filename to the certificate
/ private key.
But, we do not want to use attachments and take the full path to the
file (in the filestore) or to create temporary file.
So, we need to use a new library "PyOpenSSL" which allows you to load
a certificate / private key from a byte array.
To make this library work with SMTPLIB we use a wrapper developed
in urllib3 (PyOpenSSLContext).
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
As Fedora 32 was the current release when Odoo 14.0 was released, this
should be the supported version.
Also, a few old libs were still in mentioned in the packaging files.
They flew under the radar because they never broke the packaging.
This is not the case anymore, those libs disappeared from the Fedora
repos.
It seems that pyparsing is not used anymore since 5a1c06a19 and thus can
be safely removed from `requirements.txt` too.
pychart is not used anymore since 3425752ea.
While at it, remove mix of tabs and spaces in package.dffedora, also add
missing packages to avoid installation at test time.
Now that I started down the slippery slope, also removed some `-dev`
packages in package.dfsrc as wheel's are available.
Finally, the rpm install script now detects the python ABI version in
order to avoid update this file at each ABI change in Fedora.
Fixes#63719closesodoo/odoo#65288
X-original-commit: a8deb1dd433e3a1690d593e83ade6af46326a26b
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When Odoo is started in worker mode with Python 3.8.5, and
gevent/greenlet installed from requirements, the following error message
is thrown:
`RuntimeWarning: greenlet.greenlet size changed, may indicate binary
incompatibility`
As a gevent developper stated [1] that gevent 1.5 is not compatible with
Python 3.8, this commit bumps the version to 20.9.0 (current version for
the next Debian and Ubuntu releases [2] [3])
This commit should not impact those who use the Debian/Ubuntu packages
of gevent and greenlet. The error does not appear with those versions.
In Ubuntu Focal, the packaged version is 1.4.0 [4] but the problem was
not reported with this version and python 3.8.
For reference, it was bumped to 1.5.0 for Python 3.7 in [5].
And greenlet was bumped too for issues with Python 3.8 and 3.9 in [6].
As a result, the requirements for greenlet/gevents gains even more
complexity and should be cleaned when python 3.6 support will be
dropped.
[1] https://github.com/gevent/gevent/issues/1260
[2] https://packages.debian.org/bullseye/python3-gevent
[3] https://packages.ubuntu.com/hirsute/python3-gevent
[4] https://packages.ubuntu.com/focal/python3-gevent
[5] odoo/odoo@bb0b32bd1a
[6] odoo/odoo/@648635deca67df09417ae55c6eb181c98524b74d
Fixes#64106closesodoo/odoo#65180
X-original-commit: 1622aa755bb67e2d9e6efd18366be49b8e4783cb
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Only updates outdated requirements which actively cause issues:
* freezegun broken in 3.8 (removal of time.clock)
* xlrd broken in 3.8 (removal of time.clock)
* also monkeypatches xlrd.xlsx for 3.9 (removal of
Element.getiterator, breaks because of defusedxml)
* jinja triggers DeprecationWarning in 3.8
* pillow triggers warning in 3.9
* lxml, greenlet don't compile in 3.9
* reportlab doesn't work in 3.9
New versions try to match those of Debian Bullseye.
Also adds a script to more easily compare dependency versions between
the requirements files and what's in various distributions (currently
supports checking against debian and ubuntu).
Furthermore updates warnings filtering:
* removes xlrd (mischeck was monkeypatched as noted above)
* removes setuptools (was for older versions, one would hope this
isn't an issue anymore)
* adds babel: python-babel/babel#684 fixes the deprecation warning but
is not part of any release yet
* ignores error related to `random.sample` on a set, this is a
diagnostics bug because recordsets implement both Sequence and Set,
and the stdlib checks for Set first (bpo-42470)
See #59980Closes#61103closesodoo/odoo#62510
X-original-commit: 648635deca67df09417ae55c6eb181c98524b74d
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Since the usage of gevent 1.3.7 with python 3.6 the CPU usage exploded
on runbot running builds.
Before a better solution is found, I revert to 1.1.2 as before.
closesodoo/odoo#57281
X-original-commit: b1236c731da4c59a211c506b164a5d934ef6bc4a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
It has been a recurrent request from customers to be able to send email
messages to email addresses containing non-ascii characters. [IDNA] is a
domain extension to allow unicode characters in domain names. [SMTPUTF8]
is a SMTP extension to allow unicode in any header.
IDNA defines the [punycode] encoding which translates unicode to an
ascii representation. This encoding MUST be used to encode domains.
SMTPUTF8 is an SMTP extension that allow utf-8 in all headers on the
envelope.
[IDNA] https://tools.ietf.org/html/rfc5890
[SMTPUTF8] https://tools.ietf.org/html/rfc6531
[punycode] https://tools.ietf.org/html/rfc3492
Task: 2116928
opw-2229906
opw-2248251
closesodoo/odoo#47709
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, a lot of leftover import shims existed in the
codebase for py2-py3 compatibility, these are no longer needed since
Odoo 13.0+ doesn't support Python 2 anymore and is (finally) in EOL.
With this commit, these shims are dropped, making the code cleaner,
easier to read and with one less dependency.
Queue -> queue -> py2-py3 compatibility
xmlrpclib -> xmlrpc.client -> py2-py3 compatibility
ConfigParser -> configparser -> py2-py3 compatibility
itertools.izip_longest -> itertools.zip_longest -> py2-py3 compatibility
urllib -> urllib.request -> py2-py3 compatibility
__builtins__ -> builtins -> py2-py3 compatibility
_winreg -> winreg -> py2-py3 compatibility
mock -> unittest.mock -> merged into CPython
The debian/fedora packages and requirements.txt have been updated accordingly
closesodoo/odoo#44601
Related: odoo/enterprise#8141
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
As pdfminer does not have a Debian package in Ubuntu Bionic, it cannot
be declared as a strong requirement.
With this commit, a warning is logged if the library is not installed.
It does not prevent to index other types of documents.
closesodoo/odoo#44327
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
PyPDF performs badly on many types of PDF documents.
We add a text extraction with pdfminer, which is designed for this task.
Because pdf content extraction was so flaky, it was completely
deactivated by 1b753b0d53. We revert that :-)
closesodoo/odoo#38508
Task: 2152494
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
psycopg2 2.7 not be installed on python 3.8, needs at least psycopg2 2.8
use the same version as windows to avoid complicated rules if windows
AND python 3.8
Note that psycopg2 3.8.4 is currently the only one released after the
release of python 3.8 but reported compatibilty issued seems to be
fixed since 3.8 at psycopg/psycopg2#854Fixesodoo/odoo#42660closesodoo/odoo#44143
X-original-commit: f615826486ff8128c00b9a5ed5fbb8f86e6d492a
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
At 795c7b0a94 the external dependencies was changed from trying
to import 'ldap' to checking than 'pyldap' package was installed.
The problem is that pyldap is a unmaintained library that should no
longer be used, as explained on the package page:
https://pypi.org/project/pyldap/
"The pyldap fork was merged back into python-ldap, and released as
python-ldap 3.0.0."
Having pyldap version >= 3.0 installs python-ldap automatically and
will not cause any issue.
The Debian control file package name is adapted to use the latest.
The "ldap" externalm dependency defined in __manifest__.py will cause
pkg_resources.get_distribution() to fail in both case ("python-lap" or
"pyldap"), but the "import" fallback will succeed. For that reason, the
log warning is turned into a log info.
closesodoo/odoo#43769
Note: This library should be replaced by the pure python "ldap3" library.
X-original-commit: 1afd0ccf20881ba97e3c07dffb33e9a3a0b2cda4
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Some library versions are outdated since the release of Debian Buster.
With this commit the required libraries versions will match as close as
possible the versions available in the current Debian stable release
(Buster).
Also, the requirements were tested against a Windows Python 3.7 to
ensure that a "pip install -r" can be used without the need of a CPP
compiler.
As Babel format_time now returns 'HNE' (Heure Normale de l'EST) for Fr
locale instead of the zone offset, the test is adapted.
Finally the babel.dates is explicitely imported, otherwise the proper
import of this submodule is relying on a side effect.
closesodoo/odoo#43106
X-original-commit: 32e455bf72980e6330871aa9cd99c26c6e1225d7
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When installing requirements on MS Windows platform with Python 3.8, the
Pillow requirement is defined two times. This leads to a pip crash.
With this commit, the Pillow requirement is only defined once.
Fixes#40080closesodoo/odoo#40272
X-original-commit: cce9660c2969cc2715ff29b6dfa12e1b726bce25
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
https://packages.debian.org/buster/python-pil
As of today, python-pillow is at 5.4 in the latest debian release
This allows to deprecate the older versions progessively
Newer versions have new features like the exif_transpose method in 6.0
as discussed at #37448closesodoo/odoo#38245
X-original-commit: 4ad7a99df0a25122cf6872f880e07b6148c82ea8
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Repeat #22988 before v13 is released.
Until #35085 is fixed and we can use 0.15.x, at least with this patch people installing Odoo v13 with these requirements will no longer hit #18052.
closesodoo/odoo#36553
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Recommended by GitHub's repository alerts.
We normally stick as close as possible to the version we depend
on in the official DEB packages. This in turn depends on the version of
Debian stable at the time of release - for 11.0 that would be Debian 9
(stretch) and thus Jinja 2.8 (with security backports).
However Jinja2 before 2.10.1 suffers from a few issues that could lead
to crashes of Odoo processes.
It seems it's worth an exception to our rule for pip users, similarly to
previous bump up at d2605bccdb.
closesodoo/odoo#32601
Signed-off-by: Christophe Simonis <chs@odoo.com>
Commit cf853a785b removed all yml tests
and the yml import engine from Odoo forever, however PyYAML remains a
dependency even though it's not used anymore.
This commit removes any reference to this lib that could be found.
closesodoo/odoo#27563
Recommended by GitHub's repository alerts.
We normally stick as close as possible to the version we depend
on in the official DEB packages. This in turn depends on the version of
Debian stable at the time of release - for 9.0 that would be Debian 8
(jessie) and thus Pillow 2.6.1.
However Pillow versions before 3.3.2 and Jinja2 before 2.8.1 suffer
from a few issues that could lead to crashes of Odoo workers.
The bugfixes have been backported in the DEB packages for Pillow,
so users of Debian/Ubuntu LTS versions won't be affected if they are
keeping their systems updated.
However it's worth an exception to our rule for pip users.
Thibault Delavallée