P2 and P3 subprocess stdout seem to have very odd/different behaviours
in the face of mixing select and buffering.
Use raw/low-level IO (os.read) with select to try and fix that
up. Also read more than one byte at a time since os.read will not
block *if* the fd is readable, it will just get whatever data is
immediately available.
As of P3 socket.error is replaced by IOError and the underlying error
code must be accessed with `.errno `. This also worked in P2, so simply
use that all the time.
Now that we're closer to switching to P3 for good, these helpers have
outlived their usefulness, and mostly add noise.
All remaining dict.iter*() or dict.view*() must be converted to the
normal keys(), values() or items() calls.
Whenever the result is likely to be used for more than the scope of a
loop, or when the dict needs to be modified during iteration, the calls
must be wrapped in a ``list()``, to protect the new P3 semantics.
Those cases are very exceptional.
Also removed some dead code or improved the API to remove unnecessary
conversions.
No formal decision there, and the stdlib API seems like a PITA as it's
based around native strings, not bytes and not text.
* try to use cross-model API instead of random encode/decode
* unicode all the mail_template things
* formataddr *requires* a text tuple now
In P2, formataddr((False, False)) "worked" (generated "False
<False>"), in P3 it will fail and blow up.
* attachments are quite definitely bytes
* remove try_coerce_ascii because it's garbage
XMLRPC dumps returns text rather than bytes (for reasons unknown),
which does not suit Werkzeug's WSGI server, we need to encode the
body.
For simplicity and future-proofiness, return Werkzeug responses
instead, Werkzeug takes care of encoding text and passing bytes
through.
Otherwise when the subquery is somehow injected in the parent
query (didn't find where but...) it's b-prefixed and treated as a
binary rather than a sub-query somehow, which fails.
* IrValues.value should be a binary field, since pickle encodes
objects to bytes
* Which means the "action" string must be encoded before storage and
decoded before use
* Remove complete garbage mess of random encodings during XML
conversion, this probably still requires serious thinking and fixing
but it seems to run for now (yay)
* unichr -> pycompat (builtin removed from Python 3, ``chr`` has
become unicode-aware)
* Psycopg2 bytea values (binary fields) are returned as memoryviews in
P3 but buffers in P2
* Both bytes and (text) strings are conventionally iterable in
P3 (they have a __iter__ method), so fix up the exclusion pattern
for flatten
* remove references to basestring & unicode (use relevant pycompat
helpers)
* remove some str calls (either entirely or replaced by relevant
helper, either text or native)
* use better API to avoid unnecessary conversions
* remove some XML declarations in views
* StringIO removed from stdlib, replace with io
* try to correctly handle BytesIO/StringIO (one is for bytes the other
is for text)
* fix base64: Python 3 removed bytes-encoding and bytes-bytes
codecs (via #encode) so replace all calls to str.encode('base64'),
also b64encode is a bytes->bytes conversion so attempt to properly
handle that
issue #8530
So that was a fun one: mock.patch calls would regularly fail refusing
to find the addon in odoo.addon (e.g. essentially getattr(odoo.addon,
'account_budget' deep within the bowels of mock).
Turns out the answer is that our import hooks would not be used for
many imports: while in Python 2, sys.meta_path is empty and the
default finders are run after all meta_path finders fail as noted by
the documentation[0].
However when the import system was rewritten in Python 3.3[1]
meta_path was "despecialised" and the default finders were moved to
meta_path rather than be a hidden part of the import machinery[2]:
> sys.meta_path and sys.path_hooks now store all of the meta path
> finders and path entry hooks used by import. Previously the finders
> were implicit and hidden within the C code of import instead of
> being directly exposed.
The result of this change is that ``sys.meta_path.append`` means the
default finders should take priority and the custom ones should be
fallback. This is the exact opposite of what we want.
Fix issue by ``sys.meta_path.insert``-ing our finders at the start of
the path rather than appending them at the end. This should change
nothing in Python 2 but seems to fix the issue in P3.
[0]
https://docs.python.org/2/library/sys.html?highlight=meta_path#sys.meta_path
[1] https://docs.python.org/3/whatsnew/3.3.html#importlib
[2] https://docs.python.org/3/whatsnew/3.3.html#visible-changes
In Python 2, hasattr swallows *any* exception raised during the
access. In Python 3, it only swallows AttributeError.
Accessing an attribute on an unbound threadlocal proxy raises
RuntimeError, while running test_ir_http_mimetype ``request`` is
unbound, in Python 2 ``hasattr`` would return ``False`` on an unbound
threadlocal, in Python 3 it lets ``RuntimeError`` pass through blowing
up the test.
Fix this by just checking if the threadlocal is bound by using it as a
boolean…
In Python 3 round:
* rounds half to even (banker's rounding) rather than up
* returns an int if not given a precision (or given a precision of
None)
* discards the negative sign when rounding to 0
Add a compatibility shim which implements P2 behaviour and at least
allows passing float_utils's tests.
Implementing float_round on top of Decimal was attempted but did not
work: although Decimal.quantize takes a non-integral
Decimal (e.g. ``d.quantize(D('0.001'))``) it only supports integral
powers of 10 and does not support quantizing to e.g. 0.05: while that
will not fail it will just use the exponent for quantization and 0.05
is thus equivalent to 0.01, which is not what we want if e.g. need to
round a value to 5 cents.
This is hinted at but maybe not clearly spelled out by the official
documentation:
> Return a value equal to the first operand after rounding
> and **having the exponent of** the second operand.
Also convert a bunch of hand-rolled currency rounding to just using
the round() method on res.currency.
When deciding to prefetch records (getting records from the cache with
no value for the field being fetched), if the field was computed
`determine_value` would just get all records, not limited by the normal
prefetch limit; for large recordsets this would generate gigantic
prefetch lists for records we may not need at all.
Fix by applying the `PREFETCH_MAX` limit to records from the cache as is
done in `_prefetch_field`.
Complementarily, when traversing related fields the prefetch
environment would be lost and every record would get an empty prefetch
environment, so the values would ultimately be read one by one.
Example: select (search) 1000 product.product records, access a
related field (e.g. categ_id) in a loop, on the first iteration the
system would first read 1000 templates, then it would read each
categ_id individually, resulting in >1000 SQL queries rather than the
~2 we would expect.
Fixes#18511
In the main_flow_tour, we edit a related record of a many2one in a
one2many. A modal is displayed and we make some changes. We
then click on 'Save' in the dialog. The next step is to add a new record
in the one2many, so we click on 'Add an item'. However, we didn't
wait for the modal to be closed before clicking, which is actually not
possible in practice (because of the modal backdrop).
This made the tour fail when run from the browser, since commit
c300e5e, which produces a redraw of the line containing the
many2one that has been edited.
This rev. ensures that we wait for the modal to be closed before
adding a second record.
Christophe Simonis