Commit Graph
230 Commits
Author SHA1 Message Date
Christophe Simonis eeae2b80ea [MERGE] forward port branch 11.0 up to b793e23ae5 2018-05-09 13:55:27 +02:00
xmo-odoo 020906659e [FIX] orm: "relate" binding type on actions
When ir.values was removed and the "action" ir.values were merged
directly into the actions themselves, "client_action_relate" was
discarded as unused (and possibly too similar to
client_action_multi?). *However*:

* it was actually used implicitly as "relate" was the default key2 of
  the <act_window> XML tag
* and it had a crucial difference from client_action_multi:
  client_action_multi is shown on both form and list views by default
  and only on list if multi=True, whereas relate is shown on *either*
  the form or list view (tree if multi else form)

This means without relate the actions which should be only visible on
the form view are now on both list and tree, which leads to
overpopulated `Action` menus and odd behaviours (e.g. actions relying
active_id on lists, which id do they get and why?)

=> reintroduce relate as "action_form_only" for the specific case of
multi=False and either no key2 or a key2 of client_action_relate. If
multi=True then binding_type=action.

Fixes #20124

Followup: Task 1843603 to remove #multi and redundancy
2018-05-09 09:45:29 +02:00
Christophe Simonis 013ce7f889 [MERGE] forward port branch 11.0 up to f2e105eeca 2018-05-08 19:08:18 +02:00
Christophe Simonis 9d6ea8ef3e [FIX] api: do not prefetch new records.
New records (with a NewId) are not necessary in the cache, but should
not be prefetched.
This check has been omitted during optimisation in b0c08a6846.

opw-1843566
opw-1843636
opw-1843683
opw-1843548
opw-1843629
opw-1843657

Closes #24595
2018-05-07 18:58:27 +02:00
Christophe Simonis ac63dfc8f4 [MERGE] forward port branch 11.0 up to 8dade01e2c 2018-05-04 16:10:50 +02:00
SEINLET Nicolas b0c08a6846 [FIX] api: optimize way to retrieve records to prefetch (#24503)
Instead of searching for ids in the whole cache, search for the ones we will potentially prefetch.

Example: imagine you have 1K records in cache, and only 3 records in your prefetch set. Instead of retrieving the 1K records that have a value in cache, retrieve which of the 3 records that have no value in cache.
2018-05-04 09:59:45 +02:00
Christophe Simonis e8bf128318 [MERGE] forward port branch 11.0 up to 3ab25b60bc 2018-04-23 18:15:25 +02:00
David Arnold 71862789f2 [FIX] ORM code comment
Make misleading comment more precise, made debugging harder than needed be.
2018-04-23 09:31:50 +02:00
Xavier Morel b41c4396dd [FIX] handling of empty/absent modules in xids
Restore behaviour of not adding the separating dot if the module field
of an ir.model.data record is empty. This is useful/important for
interop with external systems.
2018-04-20 11:03:33 +02:00
Christophe Simonis 7084a5190c [MERGE] forward port branch 11.0 up to 841f3914cc 2018-04-19 15:04:26 +02:00
Christophe Simonis 841f3914cc [MERGE] forward port branch saas-15 up to 75853066a1 2018-04-19 13:12:55 +02:00
Christophe Simonis 75853066a1 [MERGE] forward port branch saas-14 up to c638cb4d56 2018-04-19 13:10:53 +02:00
Christophe Simonis c638cb4d56 [MERGE] forward port branch 10.0 up to 23431389c3 2018-04-19 13:10:12 +02:00
Raphael Collet c9caf55177 [FIX] models: bad setup of inherited custom fields
Assume a custom field F is defined on model 'res.partner'.  The setup of F may
silently fail because of missing stuff.  In that situation, setting up the
field inherited from F on model 'res.users' should also silently fail.

To reproduce the bug, install Invoicing, create a related custom field F on
'res.partner' with 'property_account_position_id.active', and install another
module.  Setting up F after loading module 'base' will fail because the field
'property_account_position_id' does not exist yet.  The error is not caught by
the inheritance of F on model 'res.users', and the installation crashes.
2018-04-19 10:00:49 +02:00
Raphael Collet 23431389c3 [FIX] models: bad setup of inherited custom fields
Assume a custom field F is defined on model 'res.partner'.  The setup of F may
silently fail because of missing stuff.  In that situation, setting up the
field inherited from F on model 'res.users' should also silently fail.

To reproduce the bug, install Invoicing, create a related custom field F on
'res.partner' with 'property_account_position_id.active', and install another
module.  Setting up F after loading module 'base' will fail because the field
'property_account_position_id' does not exist yet.  The error is not caught by
the inheritance of F on model 'res.users', and the installation crashes.

OPW 1835872
2018-04-18 17:26:05 +02:00
Raphael Collet 87b42ad9a8 [FIX] models: make group by date and order by date work together
The issue occurs when `read_group` is called with a date/datetime field to
group and order on, and the group_by is qualified, such as:

    model.read_group(..., groupby=['date:week'], orderby='date')

The ORDER BY clause in the query should use the same term as the GROUP BY
clause for the corresponding field.

OPW 1834148
2018-04-18 15:47:30 +02:00
Christophe Simonis 5f2d080cf8 [MERGE] forward port branch 11.0 up to ad825b673b 2018-04-16 18:34:56 +02:00
Xavier Morel 1e6da402d7 [FIX] export performance regression from d28f8f7
d28f8f704f carefully clears only the
relevant bit of the cache (only the top-level object which we're
exporting), however the commit did not considert that _export_rows
calls itself recursively for relational fields, and thus when
exporting relational fields the relation would become the new
top-level and get cache-cleared on every iteration, significantly
slowing down cases where such records are shared and would need to be
re-fetched every iteration, even more so if they're somewhat/somehow
expensive.

OPW-1833545
OPW-1836361
2018-04-13 17:42:52 +02:00
Xavier Morel 115fa8f2d7 [FIX] export time regression by b5c660d8
While c059e7b93c64b9dd9bc96eeb6ee5cadf significantly lowers the cost
of generating xids, a safety invalidate cache can regress cases
significantly e.g. 18s -> 430s (7mn) for some recursive exports of
xids as the cache would be completely cleared *after each record*
requiring complete fetching & recomputation of the following records.

Fix by only clearing the cache if we've actually had to create
xids *and* only for ir.model.data records.

It's possible that clearing the cache isn't even necessary at all (?),
check with @rco-odoo

OPW-1835226
2018-04-13 17:42:52 +02:00
Christophe Simonis 860dfb5586 [MERGE] forward port branch 11.0 up to d277adf4d5 2018-04-06 15:36:10 +02:00
xmo-odoo b5c660d8c0 [IMP] base: batch XID generation during export
* add UUIDs to XID sections (4 bytes / 8 hex digits) so it's not
  necessary to handle collisions & add a fallback generator
* use COPY for performances over executemany: executemany just
  performs an implicit loop on all statements (in psycopg2)
* ~pure SQL was possible:

      INSERT INTO ir_model_data (module, model, name, res_id)
      SELECT
          '__export__',
          '{model}',
          '{table}_' || A.id || '_' || uuid_generate_v4(),
          A.id
      FROM {table} A
      LEFT JOIN ir_model_data B
             ON A.id = B.res_id AND B.model = %s
      WHERE B.res_id IS NULL;

  but would have required installing pg extensions (problematic
  especially in stable) and performances are about the same as
  the COPY version

task 36343
Fixes #22493
2018-04-05 10:34:48 +02:00
rco-odoo 875f849018 [FIX] models: backport of 5f660cd
This fix avoids wild prefetching when serializing
onchange results.

We fell on a strange situation in method `onchange`, where
some data was missing from cache when serializing the
result. In that case, the prefetching overwrites so
many stuff in cache that the results of the onchanges
are lost.
Don't let that happen.

This is a backport of 5f660cd, with rco-odoo's blessing.
A use case this commit solve is
  - Install the sale app.
  - Create new quotation and assign new partner
  - From inline editable listview add a new order line
  (created the product inline) then directly save.
  - Click on the order line (NOT IN EDIT MODE). This opens
  the order line in a popup.
  - Close the popup.
  - Click edit again.
  - Now change the order line description.
When unfocusing order line field, the description is reset
to previous one.

Closes #23276
Closes #23277
2018-03-22 09:59:47 +01:00
Christophe Simonis e0345a4a3f [MERGE] forward port branch 11.0 up to 2835d29979 2018-03-20 11:45:11 +01:00
Christophe Simonis c921d94236 [MERGE] forward port branch saas-15 up to 3730a0d2df 2018-03-13 12:05:36 +01:00
Christophe Simonis 3730a0d2df [MERGE] forward port branch saas-14 up to 0e898eae35 2018-03-12 18:48:15 +01:00
Christophe Simonis 0e898eae35 [MERGE] forward port branch 10.0 up to 0440e25380 2018-03-12 18:16:02 +01:00
Raphael Collet 2ea3c333e4 [FIX] models: update non-stored inverse field on several records
Issue: when the inverse method writes on a dependency of the first record, it
invalidates the cache of the field for the other records...  Solution: simply
evaluate the inverse method one record at a time.
2018-03-12 13:35:57 +01:00
Raphael Collet c04685e54e [FIX] models: update non-stored inverse field on several records
Issue: when the inverse method writes on a dependency of the first record, it
invalidates the cache of the field for the other records...  Solution: simply
evaluate the inverse method one record at a time.
2018-03-12 13:30:43 +01:00
Christophe Simonis a0aa939d9b [MERGE] forward port branch 11.0 up to fcb48b7241 2018-03-08 19:00:04 +01:00
Christophe Matthieu 070f1173da [FIX] odoo: onchange load the related field data
Configurations uses an onchange to populate the fields visible to the
user. However, the related fields are by default called sudo, when it is
an onchange, there may be a missmatch in the cache, the cache used being
empty, there is no value to return (cache fix is not currently possible).

Before this fix we must use 'related_sudo=False' to use the good cache but
it's an inconsistent fix because in some case we must use sudo to avoid
access error.

opw-1823363
2018-03-08 15:36:22 +01:00
Christophe Simonis 9a84ba5533 [MERGE] forward port branch saas-11.1 up to e750bb6e11 2018-02-22 10:39:15 +01:00
Christophe Simonis ae6a65753e [MERGE] forward port branch 11.0 up to dbb713c2f8 2018-02-21 20:19:26 +01:00
Christophe Simonis bd16df15ea [MERGE] forward port branch saas-16 up to 98d01e46e5 2018-02-20 11:53:06 +01:00
Christophe Simonis 98d01e46e5 [MERGE] forward port branch saas-15 up to 482370d014 2018-02-20 11:08:54 +01:00
Christophe Simonis 8b527229bf [MERGE] forward port branch saas-14 up to e3d39b0e5f 2018-02-19 19:01:54 +01:00
Christophe Simonis e3d39b0e5f [MERGE] forward port branch 10.0 up to de62e33618 2018-02-19 17:07:29 +01:00
xmo-odoo d28f8f704f [FIX] base: high memory usage in export
Overly attached cache's overhead turns out problematic. Clearing the
cache after batches of records keeps the cache overhead low and does
not change performances.

Explanations:

env.cache is 3 levels of maps {field: record_id: {env: value}}, where
env can be either an Environment or a pair (cr, uid) depending on the
field's dependency on the context or not.

This can be an issue when the current request loads many fields in an
enormous number of records in a single environment. The (PaaS) case
here was the export of a res.partner field from 36358 records in a single
environment[0]:

* prefetch expanded the single field to 68, leading the base `cache`
  to have 68 entries. getsizeof(d<len=68>) == 3360 (3kB, which we will
  soon see we can ignore entirely).
* *each* of these entries would hold a map of 36358
   records. getsizeof(d<len=36358>) = 3146016 (3MB), 68 times = 213MB.
* finally each record entry is also a {(cr, uid): value}, here the
  dicts have a single entry which makes them 280B, and their key is a
  2-tuple "worth" 72B, or 352B/record/field, or 352 * 36358 * 68 ~
  870MB[1].

For a total of ~1GB, which is roughly the issue we can observe.

Future possibilities: extract the cache-clearing iterator to be more generic 
and available on BaseModel directly? Or even make the default iterator 
batched & cache-clearing?

[0] note that sys.getsizeof only provides the size of the object it's
    called on, it is not recursive

[1] slightly more in actuality as there's some variation between the
    leaves depending on the field type e.g. M2O values are a 1-tuple
    adding 60B, ...

Fixes #22475
2018-02-19 13:46:26 +01:00
Xavier Morel fc55d649d6 [FIX] base: don't recompute while performing the bulk of the unlink
If a record is deleted *and* has property fields, the deletion of the
property fields will trigger the various recomputes before the record
itself has actually been deleted, and thus stuff which depend(ed) on
that record will be recomputed under the assumption that the record
still exists, which probably will not work.

Under the assumption that the same issue would occur when deleting
more than one batch of records (unlinking the data, values or
attachment would also trigger a recompute before the proper end of the
function and could lead to incoherence, e.g. some of the records being
deleted being visible to recomputes), lift the entire effective body
of the unlink into a norecompute context, to ensure that only the very
final recompute (from the top-level unlink) is actually triggered.

opw-807036
2018-02-19 10:48:30 +01:00
Xavier Morel a583a56324 [FIX] base, web: don't fold M2M when not in import compatible mode
Before this commit, if the first exported field of an M2M is `id` (the
xid) the entire M2M is folded into a single cell with comma-separated
xids and any following field is ignored. If `id` is any but the first
field, the export behaves normally (with the m2m exported as a "table"
inside the parent record).

This behaviour makes sense for import-compatible exports where the id
is the only thing which can be exported anyway, but it is troublesome
outside of that mode as the behaviour of m2m under export becomes
incoherent/unpredictable (ish) as it depends on the position of the
m2m's `id` in the exports list.

Change it so we only perform folding in import-compatible mode (which
is the default for backwards compatibility with e.g. API calling
export_data directly & the like).

opw-813361

Fixes #22600
2018-02-19 10:43:15 +01:00
rco-odoo 5f660cdb28 [FIX] models: avoid wild prefetching when serializing onchange results
We fell on a strange situation in method `onchange`, where some data was
missing from cache when serializing the result. In that case, the
prefetching
overwrites so many stuff in cache that the results of the onchanges are
lost.
Don't let that happen.
2018-01-29 17:16:27 +01:00
Raphael Collet f50c6b1f6c [FIX] tools: align API of groupby on the one of itertools.groupby
The function `stripped_sys_argv` was broken because of `itertools.groupby`
being hidden by the local `groupby`.  Aligning APIs makes the local one a valid
substitution for `itertools.groupby`.
2018-01-25 11:28:13 +01:00
Raphael Collet 6f30f0af9c [FIX] models: behavior with multiple inverse/compute mixed on same fields 2018-01-23 16:41:45 +01:00
Raphael Collet 2ed137af3e [REF] models: refactor code to update parent_left/parent_right
Put the code to update the MPTT in specific methods, and reduce the number of
queries being made (from 5-6 queries to 2-3 queries).  Add test on MPTT to
validate the refactoring.
2018-01-16 10:47:13 +01:00
Raphael Collet 869f062e08 [REF] models: improve code of _create/_write
Also remove method `self._check_selection_field_value`, and perform the check
in field method `convert_to_column`.
2018-01-16 10:47:13 +01:00
Raphael Collet be0f24a96b [REF] models: make _create/_write deal with stored fields only
Improve methods `create`/`write`, and update parent records in them instead of
methods `_create`/`_write`.
2018-01-16 10:47:13 +01:00
Christophe Simonis 782154da11 [MERGE] forward port branch saas-11.1 up to f6ef0c7760 2018-01-10 20:35:53 +01:00
Christophe Simonis 228ecdbe28 [MERGE] forward port branch 11.0 up to ff0abd214c 2018-01-10 11:41:35 +01:00
Martin Trigaux 7e894aca0b [FIX] models: do not keep the old source in case of copy override
Current behaviour when duplicating a record:

    apply copy override in the user language (e.g. "%s (copy)")
    copy the previous term in all the other languages (including)

So if product was named:

    cheeses (en)
    kaas (nl)
    fromage (fr)

Duplicating the product, while in French, will result into:

    cheeses (en)
    kaas (nl)
    fromage (copie) (fr)

This is a problem if there is a unicity constrain on the field, it will be
raised (there is already a product named 'cheese')

After this PR:

Duplicating the product, while in French, will result into:

    fromage (copie) (en)
    kaas (nl)
    fromage (copie) (fr)

This applies **only** if there is an override of copy

This is detected by comparing the old and new value for the user language

Fixes #7010
Closes #22052
2018-01-09 18:15:47 +01:00
Raphael Collet 160d657704 [IMP] base: make translations unique
With the unique constrain, the slow SELECT DISTINCT query can be removed

Fixes #18679
2018-01-04 14:48:41 +01:00
Christophe Simonis 8ef7af6afe [MERGE] forward port branch 11.0 up to f96a797fe6 2017-12-06 12:02:58 +01:00