* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
fields) are out of scope for the lint but my editor catches
closesodoo/odoo#74604
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
PURPOSE
=======
We want to increase the score of the emails sent by Odoo and we want to
avoid them to be marked as spam by the mail clients (gmail, outook...).
SPECIFICATIONS
===============
From filter
-----------
Add a new field on the "ir.mail_server" which is "from_filter". This
field defines the email address for which the outgoing email server can
be used.
The "from_filter" can either define an email address or a domain name.
Use the system parameter "mail.default.from" which allow us to define
a default email address which is used to encapsulate the emails
(default: notifications@<catch.all.domain>).
Mail server priorities
----------------------
When sending an email, we read the FROM header and,
- We first look for a mail server which match the entire mail FROM
in that case, we do not change the email header (not needed)
- If not found, we search a mail server which matches the domain name of
the mail from (do not need to change the headers in that case)
- If not found, find the mail server linked to the "notifications"
email (defined in the system parameter). Then change the FROM header
to the notification email, and put the old one in the name part of
this header.
E.g.
Initial mail from: "Admin" < admin@example.com >
Final mail from: "Admin (admin@odoo.com)" < notifications@odoo.com >
- If no notification email is configured or if no mail server are
found for the notification email, fallback to the old system and
spoof the FROM header. In that case we do not have the choice if we
want to send the email, he will probably be marked as spam.
Sending method priority
-----------------------
In the mail server models, we defined some priorities,
1. Forced SMTP session
2. Forced mail server
3. Try to find the best mail server (see "Mail server priorities")
4. If not found, read the odoo-bin arguments
Bounce
------
As there's no standard for bounce address, we put it in the envelope
(smtp_from). But in some case, it might be considered as spoofing. So,
we use the bounce address ONLY if the mail server is configured for the
entire domain name.
One behavior which might be broken is the following; we send an email as
"std@gmail.com" and the bounce address is on the domain "odoo.com".
Before we received the bounce notifications but we were spoofing the
local part and the domain.
Now
- if a mail server is configured for GMAIL, we do not use the bounce
address (and we might not receive the bounce notification)
- if no mail server is configured for GMAIL, but one is configured for
"odoo.com"
- the FROM header will be "notifications@odoo.com"
- the FROM envelope will be the bounce address
=> In this situation we are spoofing only the local part of the email
but it's allowed as the mail server is configured for the entire
domain name
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
PURPOSE
=======
We want to be able to authenticate our servers with a certificate
for the entire domain name instead of using a username and a password.
SPECIFICATIONS
==============
Add 2 fields on the `ir.mail_server`, which are
- the SSL certificate
- the SSL private key
When we uploaded both files, we use them to authenticate the client of
the SSL connection.
Add 2 options on the Odoo binary, so we can provide the filenames of both
files (like we do for the SMTP username/password).
SETTINGS
========
Note that this type of authentication doesn't work locally for Microsoft
office 365. It seems like Microsoft is blocking non-static IP address
(not able to ping the host locally, but it works on the server).
The host name of the server is defined in the MX DNS record. Then, on
Office 365 you must create an SMTP relay based on a certificate and not
based on a hard coded IP address. The certificate must be valid for your
domain name.
e.g.
Host: openerp-org.mail.protection.outlook.com
Port: 25
Username: <keep it blank>
Password: <keep it blank>
Security: STARTTLS
Email: admin@odoobe.com
New Python dependence
=====================
The standard SSL python library only takes a filename to the certificate
/ private key.
But, we do not want to use attachments and take the full path to the
file (in the filestore) or to create temporary file.
So, we need to use a new library "PyOpenSSL" which allows you to load
a certificate / private key from a byte array.
To make this library work with SMTPLIB we use a wrapper developed
in urllib3 (PyOpenSSLContext).
LINKS
=====
Task-2367946
odoo/odoo#61853odoo/upgrade#1903
`./odoo-bin -h` prints unnecessary spaces between sentences.
---
task-2431630
Finetuning of #71130closesodoo/odoo#71364
X-original-commit: fcc60e219fd024394f8642509a20755f964546b6
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
Show full path to not found addons folder to ease debugging.
Before:
error: option --addons-path: no such directory: '../non-existing'
After:
error: option --addons-path: no such directory: '/home/user/Odoo/non-existing'
This message is local to developer and never displayed to the enduser
closesodoo/odoo#68959
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
- fixes some typos
- adds command parameters which are present when running odo-bin --help
- updates some descriptions, e.g. regarding configuration file and data-dir
closesodoo/odoo#67778
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
In the odoorc file, set a `logfile` path, but disable it via the command
line with `--logfile=`. The logs are output to the logfile configured in
the config file instead of stdout.
Parsing `--logile=` yield an empty string which was interpreted as
argument not set and skipped.
Closes#3852closesodoo/odoo#60992
X-original-commit: bae0d99b8d654283be4d265e3b40a83247a2299c
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
There is no reason for this constraint to be. Languages with a code
longer than 5 do exists in standard, (e.g. ar_001, sr@latin)
Introduced at 004a0b996fFixesodoo/odoo#52558closesodoo/odoo#52569
X-original-commit: 54340487adaed962750c9719fb50fc7e000da476
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
TL;DR: remember `osv` and `except_orm` ? You can forget about them.
* Deprecated `except_orm` dropped.
* `UserError` elevated as super type of all user-related
errors.
* Unused `DeferredException` dropped.
* Unused `QWebException` dropped (real one is in `qweb.py`).
* `MailDeliveryException` made a python exception.
* `name` legacy exception attribute made an alias of the python standard
`args[0]` attribute and deprecated.
* `value` legacy exception attribute dropped.
* `exception_type` RPC error response key dropped.
* Deprecated `osv` module dropped.
* `--osv-memory-age-limit` cli option made an alias of
`--transient-age-limit` and deprecated.
The `odoo.exceptions.Warning` have long been a deprecated alias to
`UserError`. It is going to be removed in a future version but first we
explicitly deprecate it with a warning.
The `odoo.exceptions.DeferredException` was a very old internal
exception, it has been removed without deprecation notice as it is never
raised.
The `odoo.exceptions.except_orm` has been a deprecated exception type
with deprecation warning for 5 years, it has been removed in favor of
UserError which becomes the super class of all user-related errors.
The `odoo.base.models.ir_mail_server.MailDeliveryException` was
inheriting `except_orm`. As it is not related to a user error but is
more of a problem an admin much take care of, the exception has been
made a Python error.
The `exception_type` JSON key in RPC error responses was holding an
hardcoded value derived from the exception type. Its usage has been
dropped in favor of the `name` JSON key that holds the precise exception
name. Again as it was hardly used in the source code (beside the crash
manager) it has been dropped without deprecation warning.
Since we are here trying to clean odoo custom exceptions, we are also
deprecating the `name` exception attribute in favor of the more standard
`args[0]` attribute.
The `name` (along with `value`) were two attributes used to raise
`except_orm` exceptions before the introduction of `UserError`,
`AccessError` and related exceptions. The `name` attribute, at the time,
was holding the exception type/title. Nowadays it contains the error
message. The `value` attribute, at the time, was holding the error
message. Nowadays it is no more used.
The `osv` module contains very old deprecated aliases. There is no
simple way to log a deprecation warning for osv, osv_memory and
osv_abstract but as they have not been in use for ages, they have been
removed too. To be consistent, the `--osv-memory-age-limit` cli option
has been made a deprecated alias to the `--transient-age-limit`.
closesodoo/odoo#45723
Task: 2187728
Related: odoo/enterprise#9162
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, a lot of leftover import shims existed in the
codebase for py2-py3 compatibility, these are no longer needed since
Odoo 13.0+ doesn't support Python 2 anymore and is (finally) in EOL.
With this commit, these shims are dropped, making the code cleaner,
easier to read and with one less dependency.
Queue -> queue -> py2-py3 compatibility
xmlrpclib -> xmlrpc.client -> py2-py3 compatibility
ConfigParser -> configparser -> py2-py3 compatibility
itertools.izip_longest -> itertools.zip_longest -> py2-py3 compatibility
urllib -> urllib.request -> py2-py3 compatibility
__builtins__ -> builtins -> py2-py3 compatibility
_winreg -> winreg -> py2-py3 compatibility
mock -> unittest.mock -> merged into CPython
The debian/fedora packages and requirements.txt have been updated accordingly
closesodoo/odoo#44601
Related: odoo/enterprise#8141
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Some use case like testing performance or upgrade scripts required a database
with prefilled data, covering basic corner cases. A solution can be to
create data a procedural way.
This commit proposes an API to easily populate a model, usually by giving a
list of possible values for each field or by giving a compute method that will
be based on raw values of other fields.
The basic way to define how to populate a new field is to override `_populate_factories`,
a method that returns a sequence of pairs `(field_name, factory)`.
The definition of a field is a "factory", a function that returns a neverending iterator
combining its value(s) with the values of the iterator given in parameter.
Some factory helpers are given in `tools.populate.py`:
- `iterate(vals, weighs)` ensures that one record is created for each value
by iterating on them, then resumes as `random.choice` on those vals following weights
once the first iteration is finished.
- `cartesian(vals, weights)` makes a cartesian product of its own values with the values
of its input iterator, then resumes as a randomized generator.
- `compute(function)` calls the given function with the current values dict and a random object,
and assigns the current field to the returned value.
- ...
Each iterator yields dictionaries of field values, and the factory should add a
value for the current field(s). The yielded dictionaries also contain a pseudo_field
`"__complete"`, that indicates whether this step is some randomized data
to reach the expected count of records. A falsy value indicates that the iterator
is still covering mandatory cases. This indicates whether a cartesian product is
finished, or an `iterate` has consumed all its values.
The order of the factories is quite important, since some computed fields may need
other fields to be defined, and `cartesian` factories should always be at the beginning
to avoid having too many combination. That is why the factories are given as a list of
pairs instead of a dictionary; this makes it easier to insert elements at any place.
Example:
field A: cartesian([T, F])
field B: cartesian([0, 1])
field C: iterate([a, b, c, d, e])
field D: compute(1-B)
_c is shortcut for __complete
_ is a random value, or result of a random value
```
iter | root | field A | field B | field C | field D | result
0 {_c:F} {... A:T} {... B:0} {...C:a} {...D:1} T,0,a,1 complete:False
{... B:1} {...C:b} {...D:0} T,1,b,0 complete:False
{... A:F} {... B:0} {...C:c} {...D:1} F,0,c,1 complete:False
{... B:1} {...C:d} {...D:0} F,1,d,0 complete:False
1 {_c:T} {... A:_} {... B:_} {...C:e,_c:F} {...D:_} _,_,e,_ complete:False
2 {_c:T} {... A:_} {... B:_} {...C:_} {...D:_} _,_,_,_ complete:True
```
X-original-commit: 4c0182dafa584853ed83a166096f45c33c06a245
Performances from a general point of view can be difficult to track.
This commit proposes to improve logs in two ways:
The current logs only use the sql_counter, wich will only be updated
when a cursor is closed. In a test-enable install, this counter
is actually the queries of the tests wince the install cursor is
open untill the end. The first fix is to use bot sql_counter and
sql_log_count to have total queries untill now on closed cursor,
but also the current number of queries of the current cursor.
This means that the new log format will be
{nb} modules loaded in {time}, {loading_querie} (+{test_cr_queries}) queries
instead of
{nb} modules loaded in {time}, {tests_cr__queries}queries
Nothe that in the current version, {nb} is actually the total number of
loaded modules until now.
This commit also add an equivalent end log by module and change the
loglevel of module start on install (mainly usefull if an error occurs
before anything else is logged hidding the module causing this error.)
A cleaner runbot logger is also added, in order to be abble to call
_logger.runbot( instead of _logger.log(25. This will clarify the purpose
of such a log level.
closesodoo/odoo#47283
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Before this the `--unaccent` flag does double duty to specify whether
to create new databases with the extension *and* to try to use the
corresponding function.
The latter is further gated on the function existing at all in the
database.
Given postgresql does not have unaccent installed by default it seems
only the latter is really useful and we can ignore the flag to decide
whether to enable unaccent features, if the extension is installed
consider that it should be enabled and move on.
Merge this in master rather than previous versions as it can change
things like index access / use.
closesodoo/odoo#47377
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
- small mistake when you want to import a bank statement file:
it is written:
SEPA recommanded Cash Management format (CAMT.053)
while it should be:
recommended
- comma separated -> comma-separated
Task : 2200054
closesodoo/odoo#46645
Related: odoo/enterprise#8940
Signed-off-by: Quentin De Paoli (qdp) <qdp@openerp.com>
Issue
- Remove ~/.odoorc
- Launch a server
- Try to dump the db in db manager
Ok
- Restart the server
- Retry
pg_dump not found
Cause
When the .odoorc file is not there,
the pg_path option is not present so
the normalize method set it to None
When the server is restarted, it re-checks
the options (which are strings) and in the
normalize method we check "if not pg_path"
but 'None' is truthy so it thinks this is
a real path.
Solution
Check if not pg_path or if pg_path string
is None
OPW-2189789
closesodoo/odoo#45369
X-original-commit: f080964694c01a79c52e05c0ceb42c18d4c77df5
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Since this option is soon to be used, this rename is a last tweek
to make it more logical to use since it will point to a single
upgrade dir most of the time.
closesodoo/odoo#44593
X-original-commit: 1c8e2809fb296abce6114b7da906d48a240df418
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Migration have long been only accessible thanks to a symlink from
`odoo.base.maintenance` to our private migration repository. Thank to
the change of bbb1a8f it is now possible to give a load the
migrations scripts from a path given in options.
The `initialize_sys_path` function has been updated to hooks the new
paths or the legacy symlink and to provide aliases to the previous
import logic to ensure backward compatibility.
`odoo.upgrades` (`community/odoo/upgrades`) is a new namespace that hook
all `--upgrades-paths` directories or the
`community/odoo/base/maintenance/migrations` symlink if none is previded.
`odoo.addons.base.maintenance.migrations` has been made an alias to
`odoo.upgrades`.
The `odoo.upgrades` is the desired method for accessing migrations
scripts and should be used by all new scripts.
closesodoo/odoo#44117
Task: 2178274
X-original-commit: d963cc05acd882729c4eb5ab940dae2a2197e55a
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When you deploy Odoo for the first time, it is not explicit that you
must enter a value in bytes for these specific parameters.
closesodoo/odoo#39831
Signed-off-by: Richard Mathot (rim) <rim@openerp.com>
Previously, the argument
--logfile=~/logs/odoo.log
Would create the <cwd>/~/log/ directories with the odoo.log file in it,
which is typically not the behaviour one would expect.
closesodoo/odoo#37194
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit adds a new way to use upgrades scripts folders
whithout needing to symlink them to an hardcoded path.
The folders specified in --upgrades-paths is then being used by
migration.py to find and execute migrations scripts per module
specified in the -u CLI option.
The folder needs to have the following structure:
- <upgrades_paths folder 1>
- <module1 name>
- <version1>
- <script1>
- <script2>
- ...
- <scriptn>
- <version2>
- <scripts>
- <module2 name>
- <versions>
- <scripts>
- ...
- <upgrades_paths folder 2>
- ...
Update odoo/tools/config.py
Co-Authored-By: Olivier Dony <odony@users.noreply.github.com>
Actually, when a browser_js test is running, screencasts and screenshots
are saved only if a "--logfile" parameter is specified.
This is problematic with the runbot because '--logfile' arg is not
provided when running tests. Also, in most situations, developpers
prefer to have logs on stdout.
With this commit, two CLI args are added to specify screencasts and
screenshots directories. If no screenshots dir is provided, it defaults
to "{tmp_dir}/odoo_tests/{db_name}/screenshots".
It means that screenshots are always enabled during browser_js tests.
If screencasts arg is given, it enables screencasts and save them in
"DIR/{db_name}/screencasts". If the screencasts parameter is "1", DIR
will be the same as for screenshots.
The ffmpeg tools is used to encode frames into a video. If the ffmpeg
tool is not found on the system, the frames are kept as pngs files
instead.
closesodoo/odoo#34640
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Add the support to be able to specify exlicitely module, class and method name in test_tags
Before this commit, there is no distinction between module tag and test tag. Meaning
that --test-tag module_name will replace the default +standard tag and execute all test
of the module.
This commit tries to keep the initial behaviour, while addind new features, like explicit
modules tag with /module_name, as well as class (:class) and method (.method)
Some usage examples:
--test_tags /module will execute all standard test of module
--test_tags :class will execute all standard test with class name 'class'
--test_tags .method will execute all standard test with method name 'method'
--test_tags external/module will execute all external test of module
--test_tags */module will execute all tests of module,
--test_tags */module,-standard will execute all non standard tests of module,
--test_tags -/website:TestUiTranslate.test_admin_tour_rte_translator will disable only rte translator test
closesodoo/odoo#34756
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
The test_tags is stored as None by default in the config file,
but 'None' is interpreted as a string when read.
This is a dirty fix in order to avoid to launch tests when using
a config file, we should do something cleaner in master and refactor
part of this code (substitute 'None' in the load method as it is
done for boolean)
closes#27498closesodoo/odoo#27564
Commit 76c5389 removed the option dest for test-enable leading to a
traceback when trying to save the configuration.
This was due to the fact that the options dict was sorted while
containing a None key because of the missing dest.
The goal here is to keep the default behaviour of test-enable and test-tags
except that test-tags will set test-enable to true if set.
- +standart must stay the default test-tags if none are set
(only usefull if test-enable is set)
- should work no matter the order of "test-tags" and "test-enable"
options in args
- knowing the fact that for strange reason, _parse_config is called twice
PR: #27255
As a follow-up to commit 276ea817f4,
we change the default template database used for CREATE DATABASE
operations, and we avoid setting the LC_COLLATE option unless
this default template is used (to avoid locale compatibility errors).
Rationale for the default template choice:
PostgreSQL uses `template1` by default, and recommends that users alter
`template1` if they need a custom template, while keeping template0
virgin. This is likely to be the reason why we had `template1` as the
default previously (from a1e5c645d9).
However, for new Odoo databases, it's actually better to use `template0`
as template, for multiple reasons:
- We do not customize the template database, and the system will work
fine when starting from a completely virgin database
- Since template0 does not allow connections by default, we have a
stronger guarantee that no user will be connected to the template
database (which would otherwise block database creations!)
- We use the same logic for creating new databases and for creating
placholder databases for restoring dumps. In the latter case,
PostgreSQL recommends using a virgin template, to obtain a perfect
copy of the database being restored (unaffected by any template
customization)
- Using `template0` as a template lets us choose any encoding and locale
settings (LC_COLLATE/LC_CTYPE) upon creation. Any other database
template might cause errors unless we select compatible encodings
and locale settings.
As the locale setting is system-dependent, we can't just hope
that there will be no compatibility problem, we must explicitly
default to template0. Therefore the change in
276ea817f4 was prone to causing
locale compatibility errors.
Finally, if users have a reason to prefer a different template, they
should select it explicitly with the --db_template config.
Note: existing environments will likely continue to use `template1`
as a default due to presence of older config files.
Complements #25196
Xavier Morel