Complement on 1e35315399 (#112450):
alongside the split between forwards and backwards jump we missed that
3.11 has a specialized version of each for the `is None` and `is not
None` cases. A use of that was added in standard in 16.5 (#120446) but
more generally it makes sense that server actions would support
conditional tests against `None`, probably...
closesodoo/odoo#137099
X-original-commit: 3227ae45fb79cd08a102aecb484e4f0a4f2597c1
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
When python expression is evaluated in odoo form an action or qweb, we
are checking the opcodes generated by the evaluation of this code. We do
such a verification, because the code from actions and templates can be
written by someone having not access to the server and we don't want to
let them perform actions out of the scope of their database.
In python 3.11, some opcodes from previous versions of Python have been
renamed, grouped or sepcified. There are also new ones that have been
introduce.
In this PR, we are whitelisting the new ones that are needed by odoo to
properly work in this version of Python.
Part-of: odoo/odoo#112450
When we had a traceback because of a server action, we used to have no
information in the traceback about which server action it was.
With this commit, we will be able to see the server action id in the
traceback. This will help the investigation as we will be able to check
the server action that causes the issue.
Example:
Previous traceback for a server action that executes bad python code:
```
Traceback (most recent call last):
File "/home/odoo/src/version/15.0/odoo/odoo/tools/safe_eval.py", line 330, in safe_eval
return unsafe_eval(c, globals_dict, locals_dict)
File "", line 1, in <module>
NameError: name 'i' is not defined
```
Now:
```
Traceback (most recent call last):
File "/home/odoo/src/version/15.0/odoo/odoo/tools/safe_eval.py", line 330, in safe_eval
return unsafe_eval(c, globals_dict, locals_dict)
File "ir.actions.server(152,)", line 1, in <module>
NameError: name 'i' is not defined
```
closesodoo/odoo#87086
Signed-off-by: Olivia Fantinel (ofa) <ofa@odoo.com>
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.
* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
=> `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
=> `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.
Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.
Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.
PR: odoo#78857
Task: 2571224
Bug
===
The arguments of the lambda function might be the same as other
variables in the QWeb. In that case the template rendering will crash.
To avoid this issue, we encapsulate the name of the argument to be sure
it's not the same as other variable in the QWeb engine (so the name of
the argument can no longer make the QWeb rendering crash)
Task-2674716
Closes#78392.
Forward-port of 96b09b127028e5c9a83c89acce86e1b6df904c6f
Python 3.9 has 10 new bytecode instructions, for now this commit adds 8
* CONTAINS_OP / IS_OP: moved out of COMPARE_OP which is now only used
for *rich* comparisons (bpo-39156)
* JUMP_IF_NOT_EXC_MATCH: also moved out of COMPARE_OP for the sole
purpose of testing exception types
* RERAISE and WITH_EXCEPT_START (bpo-32949) used to simplify the
context manager bytecode, but RERAISE was then used for
try/except/finally, we don't support context managers in safe_eval
so we don't care about the latter
* LIST_TO_TUPLE, LIST_EXTEND, SET_UPDATE, DICT_MERGE and
DICT_UPDATE (bpo-39320) updates to unpacking (* and **) in various
contexts
* LIST_TO_TUPLE we're skipping as it's only used when calling a
function with two `*arg` parameters (aka `foo(*a, *b)`)
* SET_UPDATE is used for set literals of 3 or more items or
unpacking ({*a})
* LIST_EXTEND is used in the same cases as well as function calls
with unpacking
* DICT_MERGE is used for function calls with `**kwargs`
* DICT_UPDATE is used when unpacking in a dict literal (`{**kw}`)
See odoo/odoo#59980closesodoo/odoo#62989
X-original-commit: 3d1c7231a080eebb4f6a7d56cbfa5721bf76befd
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this commit, the function `sorted` wasn't available on
`safe_eval`, even though it's a Python built-in, which mades it
unavailable for Python-code evaluation, e.g. server actions.
After this commit, the above function is now accessible.
closesodoo/odoo#59715
X-original-commit: a725c8927963848c3f8ada3b71897a54b740cce6
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
There are bytecode operations we don't support because we've never
needed them, and there are bytecode operations we don't support
because they're vectors of security issues.
Currently no difference is being made between the two, so when looking
to add a new opcodes it's hard to know whether it's been considered
and rejected or whether it just hasn't been considered (or found
useful) yet.
Add an explicit blacklist, which is explicitly subtracted to all
opcodes lists, to allow motivating the bans of certain
opcodes. Opcodes listed in no lists are the "graylists" of opcodes we
either haven't yet considered or have not had a use for.
Also cleanup the existing lists of opcodes to remove long-gone (or
even never-existing) opcodes, and add a few missing opcodes:
* DUP_TOPX was removed from Python 3, replaced by DUP_TOP_TWO
* STORE_MAP was removed in Python 3.5
* BINARY_DIVIDE and INPLACE_DIVIDE were removed from Python 3 (they
were used to invoke P2's integer division)
* the SLICE+<X> were removed from Python 3, which only uses
BUILD_SLICE
* CALL_FUNCTION_VAR and CALL_FUNCTION_VAR_KW were removed in Python
3.6 and functionally replaced by CALL_FUNCTION_EX (bpo-27213)
* JUMP_IF_FALSE and JUMP_IF_TRUE were removed back in 2.7, replaced by
POP_JUMP_* and JUMP_*_OR_POP (bpo-4715)
* various finally-related bytecode instructions were added to Python
3.8 to improve and speed up the handling of return, break and
continue (bpo-17611).
* INPLACE_REMAINDER, INPLACE_LEFTSHIFT and INPLACE_RIGHTSHIFT were
mentioned in PEP 202 but never actually implemented: INPLACE_
instructions were intended to mirror the BINARY_ instructions, these
mnemonics didn't match the BINARY_ ones, so the actual mnemonics are
INPLACE_MODULO, INPLACE_LSHIFT and INPLACE_RSHIFT
* while at it, BINARY_ and INPLACE_ mnemonics: as noted above they're
mirrors of one another with different prefixes (except for SUBSCR
which doesn't have an INPLACE version), we only supported a fraction
of the INPLACE codes for unknown reason.
Initially looked at simply aligning the BINARY and INPLACE mnemonics
e.g.
'BINARY_POWER', 'BINARY_MULTIPLY', 'BINARY_FLOOR_DIVIDE',
'INPLACE_POWER', 'INPLACE_MULTIPLY', 'INPLACE_FLOOR_DIVIDE',
...
so it would be easier to notice discrepancies, but it seems simpler
and clearer to just move the suffixes to a single list and generate
the BINARY_ and INPLACE_ mnemonics from that
* moved STORE_SUBSCR from EXPR to SAFE, I don't see how `foo[...] = 3`
could be in an expression
* remove pseudo-graylist `_POSSIBLE_OPCODES_P3`, was intended as a
list of new P3 opcodes we might want to add, but the purpose was
hard to understand and a proper blacklist makes way more sense
closesodoo/odoo#41085
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
TL;DR: remember `osv` and `except_orm` ? You can forget about them.
* Deprecated `except_orm` dropped.
* `UserError` elevated as super type of all user-related
errors.
* Unused `DeferredException` dropped.
* Unused `QWebException` dropped (real one is in `qweb.py`).
* `MailDeliveryException` made a python exception.
* `name` legacy exception attribute made an alias of the python standard
`args[0]` attribute and deprecated.
* `value` legacy exception attribute dropped.
* `exception_type` RPC error response key dropped.
* Deprecated `osv` module dropped.
* `--osv-memory-age-limit` cli option made an alias of
`--transient-age-limit` and deprecated.
The `odoo.exceptions.Warning` have long been a deprecated alias to
`UserError`. It is going to be removed in a future version but first we
explicitly deprecate it with a warning.
The `odoo.exceptions.DeferredException` was a very old internal
exception, it has been removed without deprecation notice as it is never
raised.
The `odoo.exceptions.except_orm` has been a deprecated exception type
with deprecation warning for 5 years, it has been removed in favor of
UserError which becomes the super class of all user-related errors.
The `odoo.base.models.ir_mail_server.MailDeliveryException` was
inheriting `except_orm`. As it is not related to a user error but is
more of a problem an admin much take care of, the exception has been
made a Python error.
The `exception_type` JSON key in RPC error responses was holding an
hardcoded value derived from the exception type. Its usage has been
dropped in favor of the `name` JSON key that holds the precise exception
name. Again as it was hardly used in the source code (beside the crash
manager) it has been dropped without deprecation warning.
Since we are here trying to clean odoo custom exceptions, we are also
deprecating the `name` exception attribute in favor of the more standard
`args[0]` attribute.
The `name` (along with `value`) were two attributes used to raise
`except_orm` exceptions before the introduction of `UserError`,
`AccessError` and related exceptions. The `name` attribute, at the time,
was holding the exception type/title. Nowadays it contains the error
message. The `value` attribute, at the time, was holding the error
message. Nowadays it is no more used.
The `osv` module contains very old deprecated aliases. There is no
simple way to log a deprecation warning for osv, osv_memory and
osv_abstract but as they have not been in use for ages, they have been
removed too. To be consistent, the `--osv-memory-age-limit` cli option
has been made a deprecated alias to the `--transient-age-limit`.
closesodoo/odoo#45723
Task: 2187728
Related: odoo/enterprise#9162
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
It's not necessary anymore as all supported Python versions implement
get_instructions, and inlining the usage of that is as readable as
calling _get_opcodes.
Also use the subset/superset predicate for validity testing instead of
difference as it's a fair bit faster:
❯ python3.8 -mtimeit -s 's1 = set(range(10)); s2 = set(range(5))' 's2 - s1'
5000000 loops, best of 5: 88.3 nsec per loop
❯ python3.8 -mtimeit -s 's1 = set(range(10)); s2 = set(range(5, 15))' 's2 - s1'
2000000 loops, best of 5: 159 nsec per loop
❯ python3.8 -mtimeit -s 's1 = set(range(10)); s2 = set(range(5))' 's1 >= s2'
5000000 loops, best of 5: 71.1 nsec per loop
❯ python3.8 -mtimeit -s 's1 = set(range(10)); s2 = set(range(5, 15))' 's1 >= s2'
5000000 loops, best of 5: 53.6 nsec per loop
we're paying double in the failure case but that doesn't super duper matter
because we're raising an exception and bailing out, the 24% gain on the
happy path seems more relevant.
closesodoo/odoo#48948
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
odoo/odoo#28519 removed large parts of pycompat, but left reraise
despite that not having much value.
Remove that helper and replace it by just a `raise` in most cases:
when raising from an except block, the old exception is automatically
chained to the new one, no need to mess around.
There is one exception: in http we have to re-raise an existing
exception explicitly (aka `raise exc` rather than just `raise).
This is less than ideal as Python *concatenates* stacks: the
previously reified stack (from the except clause) is stacked on top of
the new stack (from this raises), this leads to tracebacks "jumping
around" at the break point of the handler and is somewhat confusing.
So we want to use explicit chaining (`raise a from b`) with the
"source" providing the caught exception's original traceback and the
child providing the rest.
However since callers rely on the exception making sense, we need the
re-raised exception to be the original[0]. Copying the exception
doesn't work (see [0]), chaining an exception to itself doesn't
do anything useful, and while we could probably copy exceptions using
the pickle method[1] that's still risky.
So the most reliable option seems to be to create a new "cause"
exception, move the old traceback over to it, then re-raise the
original exception having cleared its traceback, chained to new the
cause.
[0] or a copy thereof but Odoo exceptions don't all work properly with
copy.copy and we don't want that to fail so not really an option,
we can't rely / bet on every new exception being cleanly copy-able
[1] create an "empty" instance using __new__ (or an instance of
something else onto which we re-set the __class__ in case the exctype
actually overrides __new__) then copy the __dict__
closesodoo/odoo#39709
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
In Python 3, SETUP_FINALLY is used if the exception object is bound in
the except block — even if there is no finally block — in order to
unbind and cleanup the exception's binding as part of PEP 3110:
https://www.python.org/dev/peps/pep-3110/#semantic-changes
This allows both using exception objects in Python 3 and using
finally blocks in both version.
opw 1824211
Following the 3.6 wordcode change, EXTENDED_ARGS gets used for jumps
offset by more than 256 bytes (128 opcodes), most commonly
absolute (so many script of more than 128 opcodes with conditionals
arer going to trigger the issue).
In bytecoded Python versions, EXTENDED_ARGS was only necessary above
65k opcodes: the opcode would get a separate 16-bits argument, while
wordcode opcodes get 8-bit intrinsic.
* remove references to basestring & unicode (use relevant pycompat
helpers)
* remove some str calls (either entirely or replaced by relevant
helper, either text or native)
* use better API to avoid unnecessary conversions
* remove some XML declarations in views
* Python 3.4 adds new convenient bytecode-inspection API to dis which
is nice, however Python 3.6 also switches from the old bytecode to a
new wordcode which breaks _get_opcodes and *requires* that we use
the new API
* add various opcodes which either were added in Python
3 (CALL_FUNCTION_EX) or were only used for some small corners of
Python 2 but have been expanded or despecialised in Python
3 (BUILD_SLICE)
In Python 2, to raise a new exception but reuse a traceback requires a
special form of ``raise`` (``raise etype, evalue, tb``).
In Python 3, this is now done via a ``with_traceback`` method on
exception objects.
However this requires a bit of trickery as the former is invalid
syntax in Python 3, hence pycompat bridge created via an exec for
Python 2.
In Python 3:
* various builtins and dict methods were changed to return
view/iterable objects rather than lists
* and the separate Python 2 view/iterable builtins and methods were
removed altogether
This is problematic when using these items as list (which the happens
repeatedly in Odoo), but more viciously when iterating *multiple times*
over them (which also happens, which I've messed up multiple times while
writing this, and which is a pain to debug even when you've just created
the issue).
Convert all code using these to semantics-matching cross-version
helper functions to get the LCD behaviour between P2 and P3, and
forbid the builtins via lint.
issue #8530
Xavier Morel