Now that the Debian 12 ("Bookworm") is out with Python 3.11 as the
default, it's time to update our requirements.
Reminder of the constraints for our requirements:
We try choose the smallest version from the Ubuntu/Debian corresponding
package (python3-...).
Also, if we find that one of the package was patched by the
Debian/Ubuntu maintainer, we choose the version from which the patch is
coming.
So, before this commit, the version were choose between Debian 11 and
Ubuntu 22.04. With this commit, we can simplify the requirements because
of a better matching between "Jammy" and "Bookworm".
About the choice of the python version:
* Ubuntu 22.04 ("Jammy") provides 3.10
* Debian 12 ("Bookworm") provides 3.11
* Some features that only exists in 3.9 will be needed in a near future
* 3.9 is a small release
Part-of: odoo/odoo#136904
The C implementation of the JS minification gives speedups between 6
and 55 times faster than the regex-based Python port, depending on
how compressed the input it (which is what our default implementation
does).
This is measurable when generating compiled assets bundle from scratch,
e.g. after installing/updating modules or source code.
As an illustration, the minification of a 2MB JS bundle can be 50x
faster:
```py
import rjsmin
from odoo.addons.base.models.assetsbundle import rjsmin as rjsm
js_source = open("web.assets_common_lazy.js").read() # 2MB JS
%timeit rjsm(js_source)
# -> 339 ms ± 495 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
%timeit rjsmin.jsmin(js_source)
# -> 6.88 ms ± 213 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
```
It's also a drop-in replacement, as long as you rjsmin 1.1.0 or better
is available (to support format strings properly, a.o.).
See also the documentation of rjsmin: http://opensource.perlig.de/rjsmin/closesodoo/odoo#104283
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
On Debian based systems, the `tzdata` package is maintained to reflect changes
in timezones and there is no need to upgrade the `python3-tz` package.
On the other hand, for those who are using `pip` and thus our `requirements.txt`,
the package needs to be up to date. By unpinning it in the requirements.txt:
- new installations based on pip will be up to date
- older installations based on pip can easily upgrade
- debian based installations have to maintain the tzdata package
- mixed installs like on runbot will rely on Debian tzdata
closesodoo/odoo#117527closesodoo/odoo#120155closesodoo/odoo#120205
X-original-commit: bb0fe71388c04cf26884eba89d2e0d9d0c00a185
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
All major systems (debian stable, ubuntu lts, windows) support py3.8
and all dependencies used by Odoo come with wheels for that version.
Most developers at Odoo SA uses 3.8 already and runbot is using ubuntu
jammy (which comes with py3.10) to test the current 16.0/master.
closesodoo/odoo#119492
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
As the original PR was forgotten, let's achieve the JCVD style big gap
beteween Ubuntu stable and Debian stable 11.6 Bullseye.
closesodoo/odoo#118889
X-original-commit: f21c159d9f0c1a3f3f1f3a013c7bf0fc8fa8b554
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
The pinned version of markupsafe for python > 3.10 is incompatible with
the pinned version of jinja.
With this commit, the Jinja version is pinned to follow Debian bookworm
version when python > 3.10 which should fix the issue.
closesodoo/odoo#117191
X-original-commit: 548d177e16f364b35f61f861df132dfcedc71e28
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Although Odoo 16.0 was already adapted to work with python 3.11 in
67c7cea , the requirements were left untouched. The purpose of this
previous work was to prepare for the next Debian Bookworm release.
With this commit, the requirements are updated to match the python
packages versions from Bookworm. That way, it will allow people who
installs Odoo in Python 3.11 environments by using the requirements
should be able to do so.
Closes#114195
X-original-commit: fbe4932cd583a2cc019644807a7ea248d34eec80
Part-of: odoo/odoo#115618
Maxmind offers multiple ip-geolocalization databases, historically we
have been using the City database which contains records on a
city-basis. Many years later it turns out we are primary using geoip to
know the country of the user. Geolocalization in the City database is
considered slow by our standard and we have been clever in order not to
geolocate each request by saving the info in the session.
On the other hand, the Country database that is offered by Maxmind is
much more lightweight and geoip using that country is considered a fast
operation by our standard.
In this work we make Odoo compatible with both the City and the Country
databases. Using multiple database at the same time, we can be smart and
only query each of the two on-demand. If a user ask for its country,
we'll use the fast Country db. If a user ask for its city/timezone we'll
use the slower City db.
By default it loads both database from the `/usr/share/GeoIP/` folder,
respectively the files `GeoLite2-City.mmdb` and `GeoLite2-Country.mmdb`,
you can provide alternative paths using the `--geoip-city-db` and
`--geoip-country-db` CLI options.
In the same mindset as #86015, geoip is still lazy. It is done on-demand
and the result is cached on the current request. The different with the
related PR is that as we know consider geoip to be fast, we no longer
cache the result in the session.
Task: 2848206
Part-of: odoo/odoo#91337
Installed by dependency with another lib, but the version 3.4.8 is
required to sign the DmfA declaration.
closesodoo/odoo#95495
Related: odoo/enterprise#29192
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
Update the default requirements according to latest security risks in
relevant dependencies. The baseline required version is kept in
comments, and it perfectly safe to use when security backports are
present. In other words, using the official Debian/Ubuntu packages
on a supported LTS version of these operating systems, with
unattended upgrades turned on, is a simpler safe option.
closesodoo/odoo#87397
X-original-commit: b488bd8f88a56af553c090351ff6b2ecdfb411dc
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Olivier Dony <odo@odoo.com>
We have our own html2plaintext, already used in lot of use cases instead of
just a few for the html2txt library.
Notably for emails: most emails going through Odoo stack use our simple
html2plaintext to format the body alternative. When no body alternative
is given to ``build_email`` an alternative is built using the library to
remove. Using our own parser allows to have the same results compared to
using ``MailMail.send()``. Difference lies in spaces and new lines as well
as markdown. Our html2plaintext is a bit simple and does not try to generate
Markdown but generates a simple plaintext version.
This also helps solving some issues with depending on that library.
Task-2702034
closesodoo/odoo#82486
X-original-commit: b3b9627b655cd7cb928925affed6cc8d92661e8d
Related: odoo/enterprise#23364
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Mako is not used anymore for a long time.
closesodoo/odoo#78781
X-original-commit: fb9f89afbc7a22e82309150617e8b5de5c995ff9
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
With the release of Debian Bullseye the time has come for the balancing
act by trying to update the requirements.
The constraints are the following:
* Stick as close as possible to python3-* Debian packages versions
of the current Debian stable.
* Same but for the Ubuntu LTS version.
* When one of the above package is patched by Debian or Ubuntu
maintainers, set the upstream version that includes the patch if any.
Also, as support for python < 3.7 is dropped, some cleanup can be done.
The `reportlab / pillow` combo is a special case:
* Pillow has to be updated to 8.1.2 as this version includes the
security patches that were added to Ubuntu package 7.0.0 (Focal).
* Reportlab crashes with 8.1.2 with version prior to 3.5.54 [0].
The problem does not occur on Ubuntu Focal as both versions from
the Ubuntu packaging are compatible.
So the reportlab 3.5.59 is chosen as it's the Debian Bullseye version
and to avoid multiple lines for a few minor versions.
[0] https://hg.reportlab.com/hg-public/reportlab/rev/0cf382dab63b
X-original-commit: 794677fb6a3391379200eb2144a6ed372e89c17a
Part-of: odoo/odoo#78781
PURPOSE
=======
We want to be able to authenticate our servers with a certificate
for the entire domain name instead of using a username and a password.
SPECIFICATIONS
==============
Add 2 fields on the `ir.mail_server`, which are
- the SSL certificate
- the SSL private key
When we uploaded both files, we use them to authenticate the client of
the SSL connection.
Add 2 options on the Odoo binary, so we can provide the filenames of both
files (like we do for the SMTP username/password).
SETTINGS
========
Note that this type of authentication doesn't work locally for Microsoft
office 365. It seems like Microsoft is blocking non-static IP address
(not able to ping the host locally, but it works on the server).
The host name of the server is defined in the MX DNS record. Then, on
Office 365 you must create an SMTP relay based on a certificate and not
based on a hard coded IP address. The certificate must be valid for your
domain name.
e.g.
Host: openerp-org.mail.protection.outlook.com
Port: 25
Username: <keep it blank>
Password: <keep it blank>
Security: STARTTLS
Email: admin@odoobe.com
New Python dependence
=====================
The standard SSL python library only takes a filename to the certificate
/ private key.
But, we do not want to use attachments and take the full path to the
file (in the filestore) or to create temporary file.
So, we need to use a new library "PyOpenSSL" which allows you to load
a certificate / private key from a byte array.
To make this library work with SMTPLIB we use a wrapper developed
in urllib3 (PyOpenSSLContext).
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
As Fedora 32 was the current release when Odoo 14.0 was released, this
should be the supported version.
Also, a few old libs were still in mentioned in the packaging files.
They flew under the radar because they never broke the packaging.
This is not the case anymore, those libs disappeared from the Fedora
repos.
It seems that pyparsing is not used anymore since 5a1c06a19 and thus can
be safely removed from `requirements.txt` too.
pychart is not used anymore since 3425752ea.
While at it, remove mix of tabs and spaces in package.dffedora, also add
missing packages to avoid installation at test time.
Now that I started down the slippery slope, also removed some `-dev`
packages in package.dfsrc as wheel's are available.
Finally, the rpm install script now detects the python ABI version in
order to avoid update this file at each ABI change in Fedora.
Fixes#63719closesodoo/odoo#65288
X-original-commit: a8deb1dd433e3a1690d593e83ade6af46326a26b
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When Odoo is started in worker mode with Python 3.8.5, and
gevent/greenlet installed from requirements, the following error message
is thrown:
`RuntimeWarning: greenlet.greenlet size changed, may indicate binary
incompatibility`
As a gevent developper stated [1] that gevent 1.5 is not compatible with
Python 3.8, this commit bumps the version to 20.9.0 (current version for
the next Debian and Ubuntu releases [2] [3])
This commit should not impact those who use the Debian/Ubuntu packages
of gevent and greenlet. The error does not appear with those versions.
In Ubuntu Focal, the packaged version is 1.4.0 [4] but the problem was
not reported with this version and python 3.8.
For reference, it was bumped to 1.5.0 for Python 3.7 in [5].
And greenlet was bumped too for issues with Python 3.8 and 3.9 in [6].
As a result, the requirements for greenlet/gevents gains even more
complexity and should be cleaned when python 3.6 support will be
dropped.
[1] https://github.com/gevent/gevent/issues/1260
[2] https://packages.debian.org/bullseye/python3-gevent
[3] https://packages.ubuntu.com/hirsute/python3-gevent
[4] https://packages.ubuntu.com/focal/python3-gevent
[5] odoo/odoo@bb0b32bd1a
[6] odoo/odoo/@648635deca67df09417ae55c6eb181c98524b74d
Fixes#64106closesodoo/odoo#65180
X-original-commit: 1622aa755bb67e2d9e6efd18366be49b8e4783cb
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Only updates outdated requirements which actively cause issues:
* freezegun broken in 3.8 (removal of time.clock)
* xlrd broken in 3.8 (removal of time.clock)
* also monkeypatches xlrd.xlsx for 3.9 (removal of
Element.getiterator, breaks because of defusedxml)
* jinja triggers DeprecationWarning in 3.8
* pillow triggers warning in 3.9
* lxml, greenlet don't compile in 3.9
* reportlab doesn't work in 3.9
New versions try to match those of Debian Bullseye.
Also adds a script to more easily compare dependency versions between
the requirements files and what's in various distributions (currently
supports checking against debian and ubuntu).
Furthermore updates warnings filtering:
* removes xlrd (mischeck was monkeypatched as noted above)
* removes setuptools (was for older versions, one would hope this
isn't an issue anymore)
* adds babel: python-babel/babel#684 fixes the deprecation warning but
is not part of any release yet
* ignores error related to `random.sample` on a set, this is a
diagnostics bug because recordsets implement both Sequence and Set,
and the stdlib checks for Set first (bpo-42470)
See #59980Closes#61103closesodoo/odoo#62510
X-original-commit: 648635deca67df09417ae55c6eb181c98524b74d
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Since the usage of gevent 1.3.7 with python 3.6 the CPU usage exploded
on runbot running builds.
Before a better solution is found, I revert to 1.1.2 as before.
closesodoo/odoo#57281
X-original-commit: b1236c731da4c59a211c506b164a5d934ef6bc4a
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
It has been a recurrent request from customers to be able to send email
messages to email addresses containing non-ascii characters. [IDNA] is a
domain extension to allow unicode characters in domain names. [SMTPUTF8]
is a SMTP extension to allow unicode in any header.
IDNA defines the [punycode] encoding which translates unicode to an
ascii representation. This encoding MUST be used to encode domains.
SMTPUTF8 is an SMTP extension that allow utf-8 in all headers on the
envelope.
[IDNA] https://tools.ietf.org/html/rfc5890
[SMTPUTF8] https://tools.ietf.org/html/rfc6531
[punycode] https://tools.ietf.org/html/rfc3492
Task: 2116928
opw-2229906
opw-2248251
closesodoo/odoo#47709
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, a lot of leftover import shims existed in the
codebase for py2-py3 compatibility, these are no longer needed since
Odoo 13.0+ doesn't support Python 2 anymore and is (finally) in EOL.
With this commit, these shims are dropped, making the code cleaner,
easier to read and with one less dependency.
Queue -> queue -> py2-py3 compatibility
xmlrpclib -> xmlrpc.client -> py2-py3 compatibility
ConfigParser -> configparser -> py2-py3 compatibility
itertools.izip_longest -> itertools.zip_longest -> py2-py3 compatibility
urllib -> urllib.request -> py2-py3 compatibility
__builtins__ -> builtins -> py2-py3 compatibility
_winreg -> winreg -> py2-py3 compatibility
mock -> unittest.mock -> merged into CPython
The debian/fedora packages and requirements.txt have been updated accordingly
closesodoo/odoo#44601
Related: odoo/enterprise#8141
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
As pdfminer does not have a Debian package in Ubuntu Bionic, it cannot
be declared as a strong requirement.
With this commit, a warning is logged if the library is not installed.
It does not prevent to index other types of documents.
closesodoo/odoo#44327
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
PyPDF performs badly on many types of PDF documents.
We add a text extraction with pdfminer, which is designed for this task.
Because pdf content extraction was so flaky, it was completely
deactivated by 1b753b0d53. We revert that :-)
closesodoo/odoo#38508
Task: 2152494
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
psycopg2 2.7 not be installed on python 3.8, needs at least psycopg2 2.8
use the same version as windows to avoid complicated rules if windows
AND python 3.8
Note that psycopg2 3.8.4 is currently the only one released after the
release of python 3.8 but reported compatibilty issued seems to be
fixed since 3.8 at psycopg/psycopg2#854Fixesodoo/odoo#42660closesodoo/odoo#44143
X-original-commit: f615826486ff8128c00b9a5ed5fbb8f86e6d492a
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
At 795c7b0a94 the external dependencies was changed from trying
to import 'ldap' to checking than 'pyldap' package was installed.
The problem is that pyldap is a unmaintained library that should no
longer be used, as explained on the package page:
https://pypi.org/project/pyldap/
"The pyldap fork was merged back into python-ldap, and released as
python-ldap 3.0.0."
Having pyldap version >= 3.0 installs python-ldap automatically and
will not cause any issue.
The Debian control file package name is adapted to use the latest.
The "ldap" externalm dependency defined in __manifest__.py will cause
pkg_resources.get_distribution() to fail in both case ("python-lap" or
"pyldap"), but the "import" fallback will succeed. For that reason, the
log warning is turned into a log info.
closesodoo/odoo#43769
Note: This library should be replaced by the pure python "ldap3" library.
X-original-commit: 1afd0ccf20881ba97e3c07dffb33e9a3a0b2cda4
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Some library versions are outdated since the release of Debian Buster.
With this commit the required libraries versions will match as close as
possible the versions available in the current Debian stable release
(Buster).
Also, the requirements were tested against a Windows Python 3.7 to
ensure that a "pip install -r" can be used without the need of a CPP
compiler.
As Babel format_time now returns 'HNE' (Heure Normale de l'EST) for Fr
locale instead of the zone offset, the test is adapted.
Finally the babel.dates is explicitely imported, otherwise the proper
import of this submodule is relying on a side effect.
closesodoo/odoo#43106
X-original-commit: 32e455bf72980e6330871aa9cd99c26c6e1225d7
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
When installing requirements on MS Windows platform with Python 3.8, the
Pillow requirement is defined two times. This leads to a pip crash.
With this commit, the Pillow requirement is only defined once.
Fixes#40080closesodoo/odoo#40272
X-original-commit: cce9660c2969cc2715ff29b6dfa12e1b726bce25
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
https://packages.debian.org/buster/python-pil
As of today, python-pillow is at 5.4 in the latest debian release
This allows to deprecate the older versions progessively
Newer versions have new features like the exif_transpose method in 6.0
as discussed at #37448closesodoo/odoo#38245
X-original-commit: 4ad7a99df0a25122cf6872f880e07b6148c82ea8
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Repeat #22988 before v13 is released.
Until #35085 is fixed and we can use 0.15.x, at least with this patch people installing Odoo v13 with these requirements will no longer hit #18052.
closesodoo/odoo#36553
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Recommended by GitHub's repository alerts.
We normally stick as close as possible to the version we depend
on in the official DEB packages. This in turn depends on the version of
Debian stable at the time of release - for 11.0 that would be Debian 9
(stretch) and thus Jinja 2.8 (with security backports).
However Jinja2 before 2.10.1 suffers from a few issues that could lead
to crashes of Odoo processes.
It seems it's worth an exception to our rule for pip users, similarly to
previous bump up at d2605bccdb.
closesodoo/odoo#32601
Signed-off-by: Christophe Simonis <chs@odoo.com>
Christophe Monniez