When the value of fields is getting None. And no value is assigned to the
`field`. So the traceback will be generated.
In this commit, we will assign a default value to the field if the field gets a
None value. So that it won't get a None value in any case.
sentry-4211843089
closesodoo/odoo#123193
X-original-commit: 3e652808305d3ef4674963195f3c7cdf91b37654
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Archana Vaghasiya (arva) <arva@odoo.com>
Before this PR:
The only way to neutralize the database is running cli command neutralize
After this PR:
There is a new checkbox "neutralize database" in Duplicate database and Restore database dialog that neutralize the database after duplication/restore.
I also moved the neutralization code to the external module so it can be called also outside the cli .
closesodoo/odoo#122185
X-original-commit: 616740e9d09b3d0376be43ed1489e390f6f5823e
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
To reproduce the issue:
1. Install `mrp`
2. Send a RPC to create a new BoM:
- `{"product_tmpl_id": 1, "product_qty": -1}`
Error: It will generate a traceback
Because of the negative product qty, the RPC triggers a SQL constraint
that we try to return. However, the context does not have any `lang`,
hence the traceback
sentry-4088426130
closesodoo/odoo#120614
X-original-commit: badc554b9dd7a299aac8ebca24bec6b90bef779a
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Signed-off-by: Rémy Voet <ryv@odoo.com>
Before this commit:
When uploading a file, if the transaction fails due to a serialization
failure, Odoo will retry the request. However, if a file upload is read
during the transaction, the file pointer will be at the end of the file,
and calling `.read()` again returns an empty bytes object.
After this commit:
Upon retrying the request, rewind uploads to the beginning of the file,
if the file supports it.
opw-3228200
closesodoo/odoo#120180
X-original-commit: ac59ef0668122ad71dffbb5575250c767a0a56ec
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
According to Wiktionary, French spacing is "the archaic practice (though
still current in French) of inserting a space around colons, semicolons,
question marks, and exclamation marks". This is not standard practice in
English and most languages of the world.
The purpose of this commit is to start purging the code from this typo,
as it may reflect poorly on the software for some people.
closesodoo/odoo#114533
Related: odoo/enterprise#37853
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
With the special support for postmortem debugging removed, the
likelihood of needing / wanting the werkzeug remote debugger seems
even more remote (as it works in strictly less situations, only for
frontend non-json requests).
So remove that as well.
closesodoo/odoo#115176
X-original-commit: a2022783b652299155c460294c00dbced9b619ac
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Because Werkzeug supports ipv6 natively (and has since ~2010
pallets/werkzeug@13a76262a4), running
Odoo in threaded mode allows binding to an ipv6 address.
However the prefork server is part of Odoo, and only creates
AF_INET (ipv4) sockets. Thus switching from threaded to worker mode
breaks the bind.
Add a family switch on the `http-interface`, so binding to ipv6
addresses also works in ipv6. Gevent already does that switching
internally, so the evented worker works out of the box, it was only
the prefork / workers which did not.
Looking at both the Werkzeug and Gevent implementations:
- https://github.com/pallets/werkzeug/blob/f9906fa6d83dd668f9214acef458419756bbc062/src/werkzeug/serving.py#L607-L614
- https://github.com/gevent/gevent/blob/1e412d35526183b26c1abf2eb658cbef661f5f70/src/gevent/baseserver.py#L415-L433
Both primarily check whether the host contains `:`. Werkzeug also
checks if `socket.AF_INET6` exists, however gevent doesn't bother with
that. Looking at the
source (https://github.com/python/cpython/blob/7d801f245e2021d19daff105ce722f22aa844391/Modules/socketmodule.c#L7419-L7421),
it is indeed possible for `AF_INET6` to be missing, however that
requires specifically compiling Python in an environment which doesn't
support IPv6.
Meanwhile it's also possible to disable ipv6 at runtime, in which case
I'd assume the `socket.AF_INET6` constant is present, and creating the
socket fails, which nobody guards against. Therefore just ignore the
entire thing, it doesn't seem worth the hassle (or the questioning) to
check whether the constant is present, especially since this is only a
concern when the user specifically requires an IPv6 address.
Fixes#35782closesodoo/odoo#114666
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Odoo Test environments requires to modify many parts of the unittest
TestCase, Suite and Result.
The main initial reason is to **avoid to postpone result at the end of
the test suite**, because even if it is convenient to have all errors
visible after the tests in some case, odoo logs adds information during
the execution that can be useful to debug when a test fail, to have
context for an error. (see **OdooTestResult**)
We are also fixing the stack trace comming from a unittest and since
there is no proper way to hook inside the TestPartExecutor, a dirty hack
injects anoter result on the outcome to manage the error and complete
the stack trace. This was also a way to avoid to postpone subtest logs
at the end of the test case (see _ErrorCatcher)
`_feedErrorsToResult` was used to test the test suite behavior since
there are many customization and this is quite fragile, especially if
unittest changes behavior in other python version.
**Python 3.11** introduced python/cpython#664448d8 That, in a way, goes
in the same direction of the changed introduced with _ErrorCatcher:
immediately feed errors to resut instead of postponing it. But this also
removes `_feedErrorsToResult` that was used to test this behaviors, as
well as other ones.
Since odoo should remain multi-version, this amount of changes on the
initial behavior become to complicate to keep cross-version and the
(already in our mind for a while) solution to **vendor unittest** will
help to simplify most of our test code base.
This commit modified the vendored unittest files to simplify them as
much as possible to suite our needs.
Since the runner is still the unittest one, we need to inherit from
unittest.Testcase in order to have the right type.
This also means that we still have access to all TestCase methods
without overriding them all. This is convenient for assertion methods as
an example but the initial idea is to vendor our own version of TestCase
to avoid having trouble to adapte our miscommunications to future python
versions. A trade-off must be done to chose what should remain in our
code base. The idea is to keep logic closely linked to our changes in
our code base, mainly around the run method, but also addClassCleanup
wich need to be vendored for python 3.7, but assertions methods are
independent. Any logic can be moved fom unittest to our
vendored version in the future if needed.
X-original-commit: 9a5d1ea54be49e4cc8208c33e76a6bbd2414d5d0
Part-of: odoo/odoo#113850
Pregenerate doesn't work for rtl languages. It shouldn't be a problem
but it was discovered that pregenerate run during upgrades leading to
errors
```
Traceback (most recent call last):
File "/home/odoo/src/odoo/16.0/odoo/service/server.py", line 1314, in preload_registries
env['ir.qweb']._pregenerate_assets_bundles()
File "/home/odoo/src/odoo/16.0/addons/website/models/ir_qweb.py", line 180, in _pregenerate_assets_bundles
_, _, _, id_unique, name = bundle_url.split('/')
ValueError: too many values to unpack (expected 5)tore the set of known environments as it was at setUp
```
This is because the rtl is an extra in the url breaking the logic.
Pregenerate is mainly usefull for tests on runbot
(or running tests locally), to speedup httpcases
avoiding generation at each page load.
IntegrityCases are postinstall but don't requires generating assets, or
at least no so often meaning that we can avoid the pregeneration in this
case. It should also avoid losing some time on pregeneration since it
is not useful.
This new logic only pregenerate if we have at least one HTTPCase in the
test suite. This should also speedup local testing when starting
non http post_install tests.
closesodoo/odoo#107107
X-original-commit: 43a5c17088707eec134d54ccf9341e86859f88c7
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The wsgi application entrypoint moved during the [httpocalypse]. Some
clients don't use the odoo builtin wsgi server and have troubles
upgrading from 15.0 to 16.0 because the `odoo.service.wsgi_server`
module doesn't exist anymore.
[httpocalypse]: https://github.com/odoo/odoo/pull/78857closesodoo/odoo#106187
X-original-commit: 4e5d7d2e93fcd082bd1b3a9e76cedab85b46f456
Signed-off-by: Julien Castiaux <juc@odoo.com>
Start odoo on a specific database, e.g. 'db-example'. Drop it via JSON
or XML RPC. The database is successfully dropped but the RPC fails with
a traceback because it attempts to commit on a database that doesn't
exist anymore.
import requests
admin_passwd = ...
requests.post(
'http://127.0.0.1:8069/jsonrpc',
json={'params': {
'service': 'db',
'method': 'drop',
'args': [
admin_passwd, 'db-example'
]
}}
)
Closes odoo#104527
closesodoo/odoo#105710
X-original-commit: b6e195ccb3a6c37b0d980af159e546bdc67b1e42
Signed-off-by: Julien Castiaux <juc@odoo.com>
The new `borrow_request()` function has been introduced to properly
separate the HTTP layer from the RPC layer. We forgot to protect some
RPC endpoints, mainly inside of `odoo.addons.web.controllers.database`.
This commit moves `odoo.service.dispatch_rpc` to `odoo.http` as we only
permorm RPC from the controllers and that we don't want to import
`borrow_request` inside of `odoo.service` (circular import).
X-original-commit: d0ee8615d8820e02232989c50a6e6ffbc66a9266
Part-of: odoo/odoo#105710
Previously, when gracefully stopping the server, incoming requests got
queued in the socket's backlog until the workers all finished their
requests, then the socket closes and backlogged requests get their
error response.
This would leave the system administrator with the awkward choice
between giving existing requests time to finish and not creating too
big a downtime for new incoming requests before forcefully shutting the
server down.
As the workers will stop accepting new connections on the graceful stop
call, we can close the socket so the binded address is freed as well.
This allows a cleaner graceful restart of the service as we can
gracefully quit the Odoo server, start a new server on the same address
and let the old service finish its requests for as long as we want.
closesodoo/odoo#104952
X-original-commit: 006a5fa1499dc44be9f2e285637dd0b6a771be16
Signed-off-by: Olivier Dony <odo@odoo.com>
During tests on runbot, the main reason tours are slow to start is
because the first loading of "/web" need to generate assets bundles.
Generation can take up to 5 seconds time the number of tour.
Before this, the first requests to /web takes around 7 seconds on
runbot and the next ones less than 1 second.
After this commit, the first request to /web takes around 1.5 seconds
Note that the main difficulty is to choose when to generate the assets.
(With optimisations from following commits) the generation time is
arround 30 seconds (21 css + 12 js) the first time, for all modules.
If the attachements exists the generation time is arround 3 seconds
(2.5 js + 0.5 css) mainly because of globs to find usefull files.
In practice for runbot the ideal would be to generate them at
the end of the install so that it is shared for all post_install builds.
Doint it at the end of an install is not wanted in all cases, saas
pregenerated templates and upgrade may avoid doing that.
Doing it in a special runbot step (subcommand) is not possible for niglty
execution (no split, in one go). This needs to be in the code.
It is also not practical for devs wanting to have the same behaviour on
a local machine.
A solution to make the test conditionnal at install is to base the
condion on an existing test_module. test_assetsbundle is a good
candidate here. On runbot, the at_install step (befor split) will
automatically generate assets with this solution.
Assets are also generated before post_install tests. This should be fast
if they already exists and ensure that assets are up to date after
modifying sources locally or after downloading a database on runbot.
It should be fast enough for small database and may be even faster
in the future for small diffs.
Part-of: odoo/odoo#99176
In order for websocket upgrade to work with firefox, the http version
must be set to `HTTP/1.1`.
Until now, this was done in the `send_response` method. The issue is
that this method is also used when sending an error. This is problematic
because we use environ to know whether or not the version should be changed
which means any error during `BaseHTTPRequestHandler.parse_request` (such as
wrong http version) would have led to an AttributeError being raised.
In order to solve this issue, this modification is done when making environ.
Moreover, we previously used the request uri to know whether or not the version
should be changed, this was not really reliable, we now use the upgrade header
for this purpose.
closesodoo/odoo#99535
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
We added trigram index for char fields since https://github.com/odoo/odoo/pull/83015.
But if `unaccent` is installed in the database (and isn't force to
`False` on field), these new trigram indexes are pointless and cost a
lot for nothing (almost nothing, it still can be used for equality
operator but in this case a btree will be far more efficient).
The simple way to fix it is to add `unaccent(<column>)` in the index
trigram definition, but unfortunately `unaccent` is not immutable and
may therefore not be indexed. In order to make `unaccent` indexable, we
must declare it as immutable (see
https://stackoverflow.com/questions/11005036/does-postgresql-support-accent-insensitive-collations/11007216#11007216
for more information and how to do that).
With this patch, trigram indexes are created with `unaccent(<column>)`
if the function `unaccent` is available in the database, and for the
fields that are not declared with `unaccent=False`. Moreover, we issue
a warning when `unaccent` is available but is not immutable, in which
case most trigram indexes will be useless.
odoo/upgrade#3736
task-2551518
closesodoo/odoo#95943
Signed-off-by: Rémy Voet <ryv@odoo.com>
Currently dbs can only be managed via the UI in order to take
filestores in account: while it's possible to load/copy/rename/drop
databases via `psql`, that will not manage the related filestores so
the result of the operation is incomplete DBs and leftover filestores
littering the disk.
Seems like a good idea to add a CLI to perform the same
tasks. Currently the CLI calls into the corresponding service, rather
than both calling into (possibly better designed) unified APIs, but
that seems fine for an initial version.
The top-level `db` command acts as a db manager, with git-style
sub-sub-commands for the various operations:
- `load` to load a dump file into a database (with a specified name or
not)
- `dump` to dump a local db to a zip dump (pg_dump can be created via
the corresponding command so not a concern)
- `duplicate` and `rename`
- `drop` in order to drop both the database itself and the
corresponding filestore
Notably, `create` is currently left out because a database can
trivially be created by invoking odoo using a dbname which doesn't
exist, so doesn't seem useful.
`list` is also left out, because `psql -l` generally does the job.
closesodoo/odoo#97365
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit is part of the websocket integration in Odoo.
The longpolling port does not make sense anymore: longpolling has been dropped.
This commit deprecate the `--longpolling-port` option and replace it by the
`--gevent-port` option.
closesodoo/odoo#75510
Related: odoo/enterprise#23184
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
This commit is the first commit of the websocket integration in Odoo.
It focuses on the implementation of the websocket protocol as per RFC6455.
The implementation is tested thanks to the autobahn test suite.
A config parameter is available to customize the websocket connection:
- websocket_keep_alive_timeout (default 600): Integer specifying how
many seconds a websocket connection should be kept alive
Part-of: odoo/odoo#75510
It's really not a general purpose utility, it's just a way of forcing
a lazy collection's evaluation.
Alternatively, add a helper for this to / alongside `lazy`? Note that
the lazy object(s) may not be the top-level.
closesodoo/odoo#98023
Related: odoo/upgrade#3776
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
They're not used much and they don't seem much of an advantage
compared to just calling `subprocess` with the discovery utility
functions.
While at it, fix `dump_db` to not unnecessarily have an open stdin to
`pg_dump`.
Part-of: odoo/odoo#98023
Uses a dedicated logger (for easier filtering / silencing) for
results output, and provides rough (module-level) stats in INFO but
detailed (test-level) in DEBUG.
Also updates the global query counter (`odoo.sql_db.query_counter`) to
update after each query rather than on close: with test cursors the
actual underlying counter is only rarely flushed.
closesodoo/odoo#95420
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When attempting to run the Odoo server on macOS Monterey (currently
12.4), it crashes with the following traceback:
```
Traceback (most recent call last):
File "/Users/app/Code/odoo/odoo-bin", line 8, in <module>
odoo.cli.main()
File "/Users/app/Code/odoo/odoo/cli/command.py", line 61, in main
o.run(args)
File "/Users/app/Code/odoo/odoo/cli/server.py", line 179, in run
main(args)
File "/Users/app/Code/odoo/odoo/cli/server.py", line 173, in main
rc = odoo.service.server.start(preload=preload, stop=stop)
File "/Users/app/Code/odoo/odoo/service/server.py", line 1342, in start
rc = server.run(preload, stop)
File "/Users/app/Code/odoo/odoo/service/server.py", line 553, in run
self.start(stop=stop)
File "/Users/app/Code/odoo/odoo/service/server.py", line 491, in start
set_limit_memory_hard()
File "/Users/app/Code/odoo/odoo/service/server.py", line 83, in set_limit_memory_hard
resource.setrlimit(rlimit, (config['limit_memory_hard'], hard))
ValueError: current limit exceeds maximum limit
```
Actually, this issue is not specific to Odoo but affects Python on macOS
as a whole (see [1] and [2]).
As our memory management is based on Linux - which is our primary
deployment target - and non-POSIX systems were already excluded, this
commit escapes the rlimit modification on non-Linux systems to prevent
this kind of issue.
References:
[1] https://bugs.python.org/issue34602
[2] https://github.com/python/cpython/pull/14546
Related issue:
https://github.com/odoo/odoo/issues/79112closesodoo/odoo#93381
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Stopping a threaded odoo server would spam the postgresql logs with
multiple:
<...> LOG: could not receive data from client: Connection reset by peer
Let's avoid being rude and not hang up on postgresql connections
unexpectedly.
closesodoo/odoo#89955
X-original-commit: d0da17b167edee82381b6add573742e1cf663a9c
Signed-off-by: Raphael Collet <rco@odoo.com>
Every request comes with a session, a dictionary that is persisted on
the filesystem and that saves various information such as the user
cart on the ecommerce.
When a user simply visits the website, a default session is created and
saved on disk, this bloats the filestore with many sessions. Creating
the session on-the-fly is cheaper than loading it from the filesystem.
With this work the default session is not saved on disk anymore unless
explicitly asked via `session.touch()`.
An exception to the statement "creating the session on-the-fly is
cheaper" is geoip, the ip geolocalization is not cheap. In this work,
geoip have been moved from http_routing/request.session.geoip to a
lazy property core/request.geoip. When requested the info is persisted
on the session. Like other keys from the default session, geoip will not
be persisted unless there is non-default stuff in the session.
Because the CSRF-TOKEN is based on the session-id, it is important the
session-id stays the same across multiples requests even when the
session is not persisted on disk. Even when a session is not persisted
on disk, the session-id cookie is still set so that the next session
created on-the-fly uses the same session-id.
Technical note regarding the session, it has been decided to drop the
session-snapshot protocol and to reintroduce a "modified" flag. It has
been decided not to use werkzeug's session (which natively comes with a
"modified" flag) and to keep our own session object. We decided to
extend MutableMapping instead of dict; using MutableMapping we only
have to override __setitem__ and __detitem__; using dict we would had to
override update()/pop()/... too.
Task: 2789035
Part-of: odoo/odoo#86015
When setting up the PG replication, the slave is considered in recovery
mode. However, `LISTEN / NOTIFY` is not supported in this mode, leading
to an endless loop of crashes.
In recovery mode, we simply deactivate the feature.
closesodoo/odoo#87619
X-original-commit: 6389a64953081130d62f2a1703a27ddfa751098b
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
=> `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
=> `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.
Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.
Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.
PR: odoo#78857
Task: 2571224
This commit is the 9th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals. See also [REF] core: HTTPocalypse (11) ir.http base model.
This is a two-part commit with "(11) ir.http base model". In this commit
we focus on the initialization of the various ORM objects, namely: the
registry, the cursor, the environment, the user and the context. In the
other n°11 commit we focus on the ir.http model and its relation to the
current http.py module.
One of the objectives of this comprehensive refactor was to ease the
cognitive complexity of the http framework, in other words to make it
simplier. One of the problem identified quite early during the
preparation of this work is the way the various ORM objects are
initialized, modified and cleaned during the request lifetime.
Before this work, all the ORM internals were lazily initialized via
properties. It is the first time one uses `request.cr` that a cursor is
opened to `request.db` and stored on `request._cr`. It is the first time
one uses `request.env` that an environment is create with the current
`request.user` and `request.context`. Upon user or context modification,
the current environment is discarded, the next usage of `request.env`
will create yet another environment on the fly using the modified user
and/or context.
Using this model, no ressource is initialized if not necessary. It is
possible for nodb-compatible endpoint to be served via the db-compatible
router and ir.http without ever opening a cursor to the database.
But this model is harder to reason about and ultimately to maintain.
In this work we propose to drop the lazy approach for a greedy one. In
this work the first steps of `_serve_db`, the db-compatible counter-part
of `_serve_nodb`, are dedicated to setup a registry, open a cursor to
the database and create an environment using the session's user and
context. In this work, when one wants to change the environ's user or
context, he must call `request.update_env` or `request.update_context`,
both method will recreate the environment *now* with the given values.
The downside of this approach is that resources are always allocated
even when it is not necessary. We argue that, in general, the
controllers that do not use the ORM are rare thus it is rare we allocate
unecessary resources. We also argue that the cognitive benefits are more
than welcome and that the new APIs will help at writing more robust
applications.
PR: odoo#78857
Task: 2571224
This commit is the 1st commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
Our application framework is quite complicated. There are a few explicit
middlewares: `ProxyFix`, `DisableCacheMiddleware`, `SharedDataMiddleware`.
There are many (13, in total) implicit ones in the form of
`ir.http._dispatch` overrides. There are a few frames that don't have
much values by themselves and are very much implementation details
there are: `application_unproxied`, `_call_function`, `checked_call` and
`EndPoint.__call__`.
The ORM initialisation is quite magical, the cursor, registry and
environment are all setup lazily thanks to class properties. The
context and uid of the environment are aliased on the request. The
abuse of properties makes it impossible to reason about where and when
the environment is actually instantiated and modified.
The dispatch of a request follow the next scheme:
`odoo.http/Root.dispatch` -> `odoo.addons.base.models/ir.http._dispatch`
-> `odoo.http/(Http|Json)Request.dispatch` -> `odoo.http/@route` ->
controller. Because the request becomes http or json specialized quite
late, the many ir.http override all need to parsimony try/except their
code in order to manually call `_handle_exception()` uppon error, they
cannot let the error bubble-up. This back-and-forth between odoo.http
and ir.http is source of some headaches.
Speaking about error, the `odoo.http/WebRequest._handle_exception`
implementation is quite complicated, it basically craft a new exception
out of the passed exception object in order to correct its traceback.
If the error had been bubbled-up instead of handled by
`_handle_exception()` such python hack would not have been necessary.
The objectives of this refactor are about refounding the technical dept:
* We want shorter error reporting;
* We want simpler request and ir.http APIs;
* We want better integration of all the ir.http extensions.
PR: odoo#78857
Task: 2571224
When setting the password in the database manager the config is saved
automatically in a odoorc file.
This can be problematic, especially when testing locally, with
specific options.
Going in the database manager and creating a new database or changing
admin pasword will save all those options.
This commit proposes to only save the admin_passwd when modified.
Part-of: odoo/odoo#82874
A cursor created by a Connection object is not expected to be used with
environments; use registry.cursor() instead.
closesodoo/odoo#78143
X-original-commit: 6aeb73f10953f3547b7cf830718c02a3933df9e0
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The XML-RPC interface has a compatibility shim for binaries as
historically Odoo has returned "binary" data as base64 strings. To
avoid breakages during the Python 3 transition, the shim was
introduced to decode the output binary data (under the assumption that
it'd be ASCII-compatible).
In the case where the data is *not* ascii-compatible, however, it can
generate invalid XML documents: "C0" control codes (with the exception
of tab, LF, and CR) are not valid in XML 1.0 (which XML-RPC is an
application of), however they're perfectly valid string characters and
the standard library's marshaller does not check for them, embedding
them directly in the output document and breaking the client's
decoding.
Work around the issue by replacing such binary data with an empty
string.
While at it, move the bytes shim to the customized marshaller, this
way everything's at the same place and it's not necessary to waste
time trying to understand why the marshaller is just not calling what
it's supposed to call.
Fixes#61919closesodoo/odoo#75973
Forward-port-of: #75952
Forward-port-of: #74699
X-original-commit: 1a0b3f7
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Refactor the Environments object into a Transaction object, which is
bound to one cursor, and is no longer shared among several cursors.
The following methods/properties have been changed:
- Environment.envs no longer works (because of the design change);
- Environment.manage() is deprecated (no longer useful);
- Environment.reset() is now an instance method;
- env.clear_upon_failure() is deprecated in favor of cr.savepoint().
closesodoo/odoo#75598
Related: odoo/enterprise#20451
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
Before this commit the PyInotify filesystem watcher used by the code
autoreload feature (`--dev=reload`) would not get a chance to free
it's inotify watches before the reexec, hence at each reexec triggered
by a code reload the inotify watches where accumulated until potentially
reaching the kernel limit `fs.inotify.max_user_watches`.
This patch ensures that inotify properly closes it's file descriptor
before we reexec:
https://github.com/dsoprea/PyInotify/blob/f77596a/inotify/adapters.py#L79closesodoo/odoo#71302
X-original-commit: 8703ff1e3d9be6f2f5fce2e8c4e62589b05133fb
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Creating a database from the Odoo CLI miserably fails with the error
"psycopg2.ProgrammingError: set_session cannot be used inside a
transaction".
Setting con.autocommit = True fails if some transaction is already
started, which is the case when creating a database. The fix consists
in rolling back the existing transaction (with only a SELECT) before
switching to autocommit.
closesodoo/odoo#68549
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Following 7a235c19ff, use an alternative
API to the method autocommit(). Several functions managing databases
use a connection in autocommit mode to execute some commands outside of
a transaction.
closesodoo/odoo#68491
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
We have some tests in odoo/upgrade that are sensitive to the order on
which they are executed. Specifically: IntegrityCase tests need to be
run after all UpgradeCase tests across all Odoo modules.
To support this we implemented a sorting mechanism for tests based on
the test_sequence class attribute. This is intended to be used by meta
cases, not by individual tests.
closesodoo/odoo#66521
Related: odoo/upgrade#2184
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
Start odoo in threading mode with bus installed. Login in the browser
using any internal user. Make sure the browser call the
/longpolling/poll uri. While the browser is waiting for a response, stop
the server. The server takes up to 50 seconds to stop.
When started in threading mode, a request to /longpolling/poll is served
by a casual http thread. It searches for messages enqueued in the bus
and returns them. If there are no message for the user in the queue yet,
it creates a `threading.Event`, attach it to the user in a shared
dictionnary and `wait()` on it with a timeout of 50 seconds (hardcoded
value). When the bus thread (the one responsible to listen on the
database) receives new messages, it `set()` the events which resume any
http thread that was waiting.
Because when we stop the server, there is no way to server new requests,
there are no way new messages arrive in the bus. All the threads that
were waiting for a new message will just wait until the event timeouts
which slow down the shutdown of the server.
Now we actively `set()` all events in order to resume all those workers
when we stop the server.
The `ImDispatch.poll` signature has been changed too so it is possible
to change (via code) the hardcoded default. The function was using the
object referenced by `TIMEOUT` at the time the function was defined,
using `timeout None` then `if None: timeout=TIMEOUT` ensures we lookup
the variable.
closesodoo/odoo#64530
Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
Introduce a way to schedule the execution of cron jobs *soon*. Triggered
jobs are included in the next execution batch.
Heavy refactor of the `ir.cron` model so the various parallel queries
use the (not so new) `SKIP LOCKED` postgresql select option which skip
rows that are locked instead of throwing an exception like `NOWAIT`
would do. Various methods has been renamed and the overall selection,
execution and update of job records have been re-architectured.
The cron workers can now to wake up early via a notification on the
`cron_trigger` channel of the meta `postgres` database.
closesodoo/odoo#62124
Task: 2368911
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Old naming has been deprecated for a long time, it's been removed
entirely in 3.9 (bpo-37804).
X-original-commit: c4a94bc99285aba085badeb1a41e8fe04562ef50
Before this fix, trying to authenticate via
xml-rpc call from PHP following the documentation at
odoo.com/documentation/14.0/webservices/odoo.html#logging-in
raised an error:
> $uid = $common->authenticate($db, $username, $password, array());
TypeError: 'list' object is not a mapping
Because PHP doesn't have separate array and mapping types, the
XML-RPC encoder disambiguates based on the existence of
key => value pairs, such disambiguation yields an empty xmlrpc
array for an empty PHP array, which is unexpected on the Python
side.
Relax the check on the Python side:
* fixing this on the client side requires adding arbitrary and
meaningless key => value to the empty array to force the
correct disambiguation which is ugly and weird
* the example code has been there for a long time, so there's
probably lots of such PHP code in the wild
opw-2388141
closesodoo/odoo#62101
X-original-commit: 9f97b7435977c81a079004b7c2186ce75526490c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
`registry._init_modules` is a set so its iteration order is
non-deterministic (it's randomised on interpreter initialisation
unless PYTHONHASHSEED is provide through the environment). This can
lead to annoying non-deterministic behavior: while the non-determinism
is only at the module level, it's easy enough for modules to have
python-level side-effects (e.g. patch methods, update globals, ...),
which may only be surfaced by an other module executing after them,
but not if said module executes before.
By sorting the modules we should make this much more reliable one way
or another.
closesodoo/odoo#60028
X-original-commit: f9169a468a2328a691ec4f32233ba3bad3622282
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
While individual addons paths are normalised, module and resource
paths are not.
As a result, when symlinking modules into directories on the
addons path, the path of test modules is only half-normalized: it's
normalised up to the addon path (which likely did not need it in that
setup) but not above that.
This is an issue when using `--test-file`, because that path is fully
normalised, and so the path of the provided test file and that of the
corresponding test module will not match, leading to the tests
unexpectedly not getting run.
Normalize the test module's path before the comparison, using the same
routing used for --test-file.
closesodoo/odoo#59822
X-original-commit: 536809662e542994451be793cd09e87adbf31776
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
odoo/odoo#53938 improved the SQL linter and used psycopg2.sql to
silence the linter where that still made sense. However I forgot to
mark table names (pretty much exclusively) as `sql.Identifier` in a
few somewhat rare callsites, which consequently break when invoked as
a simple string is not a Composable and psycopg2 therefore rejects it
when composing the query.
odoo/odoo#54556 fixed a few mis-updated ones, but apparently I still
managed to miss one here.
closesodoo/odoo#56191
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Archana Vaghasiya