The XML-RPC interface has a compatibility shim for binaries as
historically Odoo has returned "binary" data as base64 strings. To
avoid breakages during the Python 3 transition, the shim was
introduced to decode the output binary data (under the assumption that
it'd be ASCII-compatible).
In the case where the data is *not* ascii-compatible, however, it can
generate invalid XML documents: "C0" control codes (with the exception
of tab, LF, and CR) are not valid in XML 1.0 (which XML-RPC is an
application of), however they're perfectly valid string characters and
the standard library's marshaller does not check for them, embedding
them directly in the output document and breaking the client's
decoding.
Work around the issue by replacing such binary data with an empty
string.
While at it, move the bytes shim to the customized marshaller, this
way everything's at the same place and it's not necessary to waste
time trying to understand why the marshaller is just not calling what
it's supposed to call.
Fixes#61919closesodoo/odoo#75973
Forward-port-of: #75952
Forward-port-of: #74699
X-original-commit: 1a0b3f7
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
PR #42723 changed the public method context_get on res.users return a
frozendict instead of a classic CPython dict.
Since the default XMLRPC Marshaller does not know how to serialize a
frozendict, this would result in errors when calling the method directly
via the XMLRPC layer.
This commit solves this issue by defining a marshalling procedure for
frozendict objects in our custom OdooMarshaller class, this simply
converts the frozendict into a dict and uses the default dict-to-xml
dump procedure (dump_struct).
Fixes#75412closesodoo/odoo#75478
X-original-commit: e80a299ed910cacac4aa4e64164909af36ce62ad
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
odoo/odoo#74678 fixed the universal traceback which would happen any
time an HTML field would be read, however it's incomplete because:
* it was tested on a field which wasn't HTML in 14.4
* and no markup was put in the field anyway
So the markup being embedded in the XML-RPC document unescaped was
missed, leading to:
* corrupted (partial) responses when the HTML content is XML-valid,
depending on the exact API of the client it might only return the
first or last text segment, or all the text without markup, or
something else
* outright deserialisation error on XML-invalid content (e.g. void
elements like <br>)
The cause being that `Markup` overrides all `str` methods to first
escape their parameters before actually applying on the object. This
means `xmlrpc.client.escape` would basically do nothing, then would
concatenate the `Markup` into the document (stringifying it) and send
the entire thing on its way.
Fixes#74884closesodoo/odoo#74957
X-original-commit: 8a0f7d820cb2296f410be215ffeadd1cefae174c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
In 01875541b1, HTML fields (and various
methods) were made to return markupsafe.Markup objects.
However at the time I didn't consider that XML-RPC serialization
remains based on type *identity*, and thus the `Markup` object would
not serialize outbound through XML-RPC, and would blow up instead.
This should fix the issue, by serializing Markup objects as str.
closesodoo/odoo#74684
X-original-commit: 165bd3bf88becc42416e49ad2f430e90675219fa
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
We provide a new helper class to help redact x2many commands for create
and write methods. To ensure best compatibility with the xmlrpc layer we
do not change the protocole, the commands are still 3-elements tuples
where the first element is still an integer in between 0 and 6. The
helper class provide the cannonical constants and static methods to ease
working with the commands. The new helper class is also available in QWeb.
Developers are encouraged to transition their code so it uses this new
helper class.
Task: 2366606
Also non-browser jsonrpc (as it goes through a similar process): for
internal performance reasons, name_search and read_group have been
converted to a *lazy* name_get, so the "display name" is not
unnecessarily computed.
However this is an issue for the RPC endpoints (/xmlrpc and /jsonrpc)
as they have no support for `lazy` and thus tend to blow up and / or
do the wrong thing when trying to output a lazy:
* xmlrpc has no way to handle lazy at all and straight blows up
* jsonrpc falls back to `json_default` so they try to stringify the
lazy, which might have worked except
*Problematically* both endpoints delegate the actual work to
`dispatch_rpc` which handles dispatching between various services and
ultimately creates a *new* cursor before calling model
methods (`object` service and `execute`/`execute_kw`).
This means by the time the result is serialized to be output, the
lazy's cursor has long been closed, and thus any access to an
unevaluated `lazy` errors out when trying to fetch the underlying
item.
This also means we can't just add a hook to serialize the lazy
in the xmlrpc marshaller, though we do have to do that. We *also* (for
both xmlrpc and jsonrpc) have to force evluation of lazy values before
our cursor is closed, meaning it has to be done right after the method
is invoked, iterating the entire response.
Related to task 2170343
closesodoo/odoo#49286
X-original-commit: e2b5a359c1d5eccbe725c1c3169b4130d7bca49b
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
`method` has no effect, so currently XML-RPC can be called via other
methods, e.g. GET, which is against XML-RPC specification.
closesodoo/odoo#37917
X-original-commit: c2fe13d31f6b55900ad139d4e75b8c1a0c4ac165
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Before this revision, trying to read a Date field using xmlrpc would
result in a marshalling error.
This is due to the fact that 960360afe4
changed the internal value of Date/Datetime fields to hold
datetime.date/datetime objects.
The xmlrpc layer is capable of converting datetimes to the XML
datetime representation, but this is not possible for datetime.date
objects.
In order to solve this, the Marshaller class used by python's
xmlrpc.client is overridden and monkey-patched, allowing to convert
datetime.date / datetime.datetime objects into properly formatted
strings.
opw-1896364
closesodoo/odoo#28022
Endpoints can be explicitly marked as `save_session=False` (default is
true across the board). In that case they will have an in-memory session
(either the existing one or a brand new one) but the session won't be
persisted to disk.
Currently used for non-browser RPC endpoints: the APIs don't use
cookies/sessions and we can't assume the RPC libraries keep cookies
across calls. This means a new session is created and saved to disk for
each RPC calls, for no useful reason.
This way, XMLRPC calls can get request details form the standard
`odoo.http.request` system.
Move jsonrpc to the same controller while at it, for coherence.
Fix#24183
Xavier Morel