Commit Graph
100 Commits
Author SHA1 Message Date
Rémy Voet (ryv) 3353dfdb29 [REM] core,base,*: deprecated some methods to be remove correctly later
- Deprecated `norecompute` (on Environment class) because it is useless
and do nothing.
- Deprecated `cache_restart` (`res.company`) because `clear_caches` do
the stuff.
- Deprecated `write_company_and_print_report` (`res.company`) because
since https://github.com/odoo/odoo/pull/33863/, it is unused
- Deprecated `open_company_edit_report` (`res.company`) because since
25f7040998, it is unused.

Part-of: odoo/odoo#99550
2022-11-02 15:34:30 +01:00
william bebfa9b52c [FIX] base,core: order of self.env.companies
Using a `frozenset` leads to non deterministic issues.
Because when nothing is set in the context key `allowed_company_ids`,
`self.env.companies` will fallback on `self['res.company'].browse(user_company_ids)`
Since the browsing is done on a non ordered set, the recordset doesn't
follow the `_order` set on the model, by definition.

This has the effect of not being deterministic when iterating on
`self.env.companies`.

For instance
https://runbot.odoo.com/runbot/build/20539598

closes odoo/odoo#104538

X-original-commit: bfee55b14c918c44ebbb2563d5dc0e102edf396e
Signed-off-by: Rémy Voet <ryv@odoo.com>
Signed-off-by: William André (wan) <wan@odoo.com>
2022-10-28 20:39:50 +02:00
Xavier Morel 6c6ca16626 [REM] core: deprecated Environment methods
Part-of: odoo/odoo#98138
2022-10-26 19:03:46 +02:00
Victor Feyens 9ded78ede0 [IMP] core: docstring improvements
* clean and improve docstrings in orm
* fix typos found with codespell
* rely on the Environment class docstring instead of doc content (and
therefore move part of the doc inside the class docstring)

closes odoo/odoo#102969

X-original-commit: 8250cd4b210005d223a4cdb8afa4014425ca6fa3
Related: odoo/documentation#2803
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2022-10-10 19:55:37 +02:00
Denis Ledoux 6ed63b9fee [IMP] base: possibility to archive companies
It is technically nearly impossible to delete a company
when it actually dealed with customers
(emitted invoices, received payments, ...).

Hence, if you want to get rid of a company,
for instance because you closed one of your subsidiaries,
giving the possibility to archive your company,
thanks to an active field, would be the best way to go.

We are hesitating to do a related field to the
`partner_id.active`, but we are a bit afraid
some people archive the partner linked to their company
for other valid reasons than get rid of their subsidiary
(such as avoid changing the address of their company
by mistake through the Contacts app),
while still wanting the company itself to be active.

So, we make it an independant column at the moment,
so we have the actual stored column in case we need it,
and will do a related stored to the partner
later on if we change our mind.

Manual forward-port of #102801

closes odoo/odoo#102586

Signed-off-by: Olivier Dony <odo@odoo.com>
2022-10-10 11:56:51 +02:00
ef00294e71 [IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation.  Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table.  The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages).  Empty text is allowed in translation values, but not NULL.

Here are examples for a field with translate=True:

    NULL
    {"en_US": "Foo"}
    {"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
    {"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}

Like before, writing False to the field makes it NULL, i.e., False in all
languages.  However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.

Here are examples for a field with translate=xml_translate:

    NULL
    {"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}

Change for callable(translate) fields: one can now write any value in any
language on such a field.  The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values.  Basically the
structure of the value must remain the same in all languages, like before.

Reading a translated field is now both simpler and faster than the former
implementation.  We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:

    SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...

The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages).  For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.

Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language.  Before this change, only the source value of the
field ('en_US') could be indexed.

Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages.  We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.

Code translations are no longer stored into the database.  They become static,
and are extracted from the PO files when needed.  The worker simply uses a cache
with extracted code translations for performance.  This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker.  Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).

Performance summary:
 (+) reading 'model' translated fields is faster
 (+) reading 'model_terms' translated fields is much faster (no need to inject
     translations into the source value)
 (+) searching translated fields with operator 'ilike' is much faster when the
     field is indexed with 'trigram'
 (+) updating translated fields requires less ORM flushing
 (-) importing translations from PO files is 2x slower

Some extra fixes:
 - make field 'name' of ir.actions.actions translated; because of the PG
   inheritance, this is necessary to make the column definition consistent in
   all models that inherit from ir.actions.actions.
 - add some backend API for the web/website client for editing translations
 - move methods get_field_string() to model ir.model.fields
 - move _load_module_terms to model ir.module.module
 - adapt tests in test_impex, test_new_api
 - because env.lang is injected into SQL queries, its returned value is
   now guaranteed to correspond to a valid active language or None
 - remove wizard to insert missing translations (no longer makes sense)

task-id: 2081307

Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-09-15 22:37:50 +02:00
Rémy Voet (ryv) dfa34aa3fe [IMP] core: reduce cost of modified.
For non-stored computed fields, `_modified_triggers` will traverse the
tree (at the cost of extra queries) only to know which record to
invalidate in cache. But in most cases, these fields have no data in
cache, so they can be ignored from the start, which allows us to prune
entire subtrees from the merged tree.

By example:
With simple write on `show_operations` of one `stock.picking.type`, the
`_modified_triggers` will fetch every ids (from database) of `stock.picking`
and `stock.move` related to this `stock.picking.type`
(For `stock.move`, it is because of the
`show_operations = fields.Boolean(related='picking_id.picking_type_id.show_operations'`))
In that case, there isn't any data of `show_operations` (`stock.move`)
in cache, then there are nothing to invalidate and
the `_modified_triggers` cost
is high (extra queries/processing) for nothing.

Then we cut parts of the tree when we know that
they won't invalidate anything (=> if the cache is empty for the field).
Also refactor the way to merge trees to be more efficient.

Performance improvements:
- For all tests at-install done by the runbot, we gain -+ 3.5% of
queries.
- In the example above, we reduce the number of queries (potentially
bottleneck queries in large DB) from 7 to 4.
- In term of CPU (without counting time in SQL), the new version is -+
20 % faster (on install of stock,purchase,mrp and with
--test-tags=/stock,/purchase).

closes odoo/odoo#76322
task-2780812

closes odoo/odoo#99274

Related: odoo/enterprise#30951
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-09-02 23:16:06 +02:00
Raphael ColletandVincent Schippefilt 384fda2c2a [REF] core: replace towrite by dirty flag in cache
Merging both the memory of field values and suspended updates has
several advantages:
 - avoid inconsistencies between cache and towrite
 - cache updates can be made safer w.r.t. dirty flag

However, the dirty flag in cache does not go well with context-dependent
fields.  When a context-dependent field is dirty in cache, the value to
store in the database is accessible through some context values.  But
when the model is flushed, the context values on the current environment
may be different.  When this happens, the method flush() fails to
retrieve the data to flush.

The proposed solution is to store the "dirty" value in cache under
conventional context values, and to retrieve them under the same
conventional context values to flush them.  For instance, when storing
the value of a binary field, it will be stored once under the context
value `context.get('bin_size')`, and a second time under the context
value `None`.  The flush implementation will then retrieve the value
using the context value `None`.

Translated fields are also problematic when a value is put in cache with
an environment where lang=False, and the value is retrieved with another
environment where lang=None.  This issue is addressed by normalizing the
context key 'lang' to None when the context value is False.

Part-of: odoo/odoo#95325
Co-authored-by: Vincent Schippefilt <vsc@odoo.com>
2022-07-14 22:45:44 +02:00
Raphael Collet 9c3b9a4926 [FIX] core: automatically flush upon invalidation for cache consistency
Because method _read() no longer updates existing values in memory,
those values in memory must be consistent with the database.

On the other hand, if pending updates are not performed on the database
before fetching values, it means that the corresponding database values
cannot be put in cache.  This implies that one cannot empty the cache
without flushing the corresponding fields.

In order to avoid mistakes, flush automatically before invalidating the
cache.  This makes the invalidation methods safe by default, and avoids
cargo-culting which would systematically associate invalidation to
flushing, which may eventually be less performant.

Part-of: odoo/odoo#66938
2022-07-05 11:35:00 +02:00
Raphael Collet a91cb08c5d [IMP] core: avoid flushing fields to read
The idea is to avoid flushing the fields to fetch in method _read().
This delays UPDATE queries, and makes the prefetching mechanism simpler
and more effective.

We do this by not overwriting the cache values by the values fetched
from database.  This simple idea allows to fetch more fields and more
records without having to care about pending computations and updates.
But it requires the cache consistency to be much more strict, because
nothing will "fix" the cache inconsistencies "by chance".  And it also
requires pending updates to be present in cache.

Part-of: odoo/odoo#66938
2022-07-05 11:35:00 +02:00
Raphael Collet 9f6be9321c [IMP] core: reimplement cache check in pure SQL
This allows checking that the ORM keeps the cache consistent without
relying on it for the check.

Part-of: odoo/odoo#66938
2022-07-05 11:34:59 +02:00
Gorash 8c869769e9 [IMP] base: add a cache for company_ids on res.user
Part-of: odoo/odoo#88276
2022-06-03 16:40:39 +02:00
Raphael Collet 32bc28aa66 [IMP] core: better API for flush() and invalidate()
This provides a new API for those operations, in order to make the
distinction between the use cases more explicit.  The former API was
using obscure parameter combinations to correspond to various cases.

In the summary below, `fnames` is an iterable of field names.  If the
parameter is not given, it means "all fields" in the given context.
Note that method recompute() is now mostly private, as it should not be
used in business code.

    # process pending computations and updates
    records.env.flush_all()                # all fields of all models
    records.flush_model(fnames)            # the fields of all records of the model
    records.flush_recordset(fnames)        # the fields of the given records

    # process pending computations, became non-public methods
    records.env._recompute_all()           # all fields of all models
    records._recompute_model(fnames)       # the fields of all records of the model
    records._recompute_recordset(fnames)   # the fields of the given records

    # invalidate the cache of fields
    records.env.invalidate_all()           # all fields of all models
    records.invalidate_model(fnames)       # the fields of all records of the model
    records.invalidate_recordset(fnames)   # the fields of the given records

Part-of: odoo/odoo#87527
2022-05-25 18:00:46 +02:00
Xavier-Do cb9b93b5c4 [FIX] core: ordered to_compute
Set order is not randomized for integers (integer hash is the integers
itself and doesnt use pythonhashseed)
but the output may be ordered or not depending of the values:

In [2]: list(set(range(126,130)))
Out[2]: [128, 129, 126, 127]

In [3]: list(set(range(130,134)))
Out[3]: [130, 131, 132, 133]

This will lead to different results in query count tests depending on
the current value of the sequence.

The proposed solution is to use an Orderedset. We need to keep in
mind that the initialisation/operation on Orderedset may be slower but
this shouldn't represent any significant overhead in this case.

closes odoo/odoo#91962

X-original-commit: da66b8fc008f97680584e266135b7fa7d3dedeca
Signed-off-by: Rémy Voet <ryv@odoo.com>
2022-05-23 08:30:14 +02:00
Rémy Voet (ryv) 8b7c8d0b71 [REF] core: replace _browse() by __init__()
The creation of recordset was done by class method _browse() instead of
a regular call to the model's class.  As we removed the old usage of
method __init__(), we can now reuse it with a normal usage. After this
commit, we can create a recordset by simply calling its class:

    registry['model_name'](env, ids, prefetch_ids)

closes odoo/odoo#79563

Signed-off-by: Raphael Collet <rco@odoo.com>
2021-12-24 14:56:58 +00:00
Rémy Voet (ryv) cbb563fd98 [REM] core: remove deadcode from models.py
- Remove unused imports `AsIs` and `Collector`
- Remove unused logger _schema
- Remove unused function same_name
- Remove unused attribute `_needaction`
- Remove backward compatibility of `__new__` and `__init__`
- Remove backward compatibility of `__export_rows`

Also:
- remove deprecated warning in api.py
- remove the `__init__` from `ir.ui.menu`, it was useless
because the cleaning of cache (`clear_caches`) is global (a call of
`clear_caches` clean all cache method with a ormcache decoration)

Part-of: odoo/odoo#79563
2021-12-24 14:56:57 +00:00
Victor Feyens 57ced812f0 [REF] product: clean pricelist API
* Do not rely on context, everything should be cleary specified through
parameters
  Catch context keys in an unique targeted place to improve code clarity
* Drop strange old API
  * do not provide unused partner parameter anymore
  * do not provide products, qty as a list of tuple, we only request the
same qty for all products anyway
* Clear methods, add/adapt comments and docstrings
* Reduce potential side-effects of context content.
2021-12-23 15:30:23 +01:00
Yannick TivisseandVictor Feyens 18952cdc76 [IMP] *: Convert single create method into multi
Taskid: 2703085
Part-of: odoo/odoo#80824
Co-authored-by: Victor Feyens <vfe@odoo.com>
2021-12-14 19:13:18 +00:00
Xavier Morel bdc9d9d369 [FIX] core; base: lots of docstrings
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
  fields) are out of scope for the lint but my editor catches

closes odoo/odoo#74604

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-12-09 14:36:58 +00:00
Raphael Collet 0011823932 [FIX] core: cr.transaction no longer depends on registry
The cursor features a "transaction" object to manage application-
specific data in relation with cursor operations.  In its initial
implementation, the transaction object was created by the registry.
This created a requirement: in order to be used in environments, a
cursor had to be created by registry.cursor().

We now remove that unnecessary technical requirement by making
environments create the transaction object on demand.

closes odoo/odoo#80644

X-original-commit: e902713648bca329e6821859462b2128aad62c09
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-12-01 08:44:48 +00:00
Rémy Voet (ryv) c3399b5470 [FIX] base: fix performance scalability of create
Issue:
------
The `create` method (models.py) doesn't scale correctly
(with huge number of values) when compute store no-readonly
field(s) are in values (at most one).
It is due to the method `protecting` (api.py) which has a
bad complexity in the `create` situation
(list of (field, records) pairs as argument):

O(r²) with `r` = number of record containing the protected field:
List of `len(r)` send to `protecting`,
loop of this list (`r` factor), loop on fields (constant factor),
create a new frozen set with the previous one (`r` factor).

Fix:
----
Decrease the complexity to O(r) by creating a map of set of ids by
protected field which allows avoiding recreating a new frozenset
for each record (update with tuple of one inside => O(1)).

Performance gain:
----------
For a very simple model with only one compute store no-readonly field,
and all `create` `values` contains this field.
+--------------+---------------+---------------+---------------+----------------+
|   Batch ->   |     1000      |     10000     |    30000      |     80000      |
+--------------+---------------+---------------+---------------+----------------+
| Before (sec) | 0.083 ± 0.006 | 1.433 ± 0.091 | 8.764 ± 0.609 | 94.428 ± 3.021 |
| After (sec)  | 0.069 ± 0.003 | 0.706 ± 0.006 | 2.188 ± 0.076 | 5.875 ± 0.104  |
+--------------+---------------+---------------+---------------+----------------+

closes odoo/odoo#80437

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-11-29 11:22:24 +00:00
Stefan RijnhartandRaphael Collet 8f574a274a [14.0][FIX] Keep original kwargs intact for reuse on retry
closes odoo/odoo#79589

X-original-commit: 6387619d308c8969cbafe6acf77f6d22075ac6ba
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2021-11-10 10:43:17 +00:00
Raphael ColletandXavier Dollé 1595c0ee27 [REF] core: replace thread-local "envs" by cursor-bound "transaction"
Refactor the Environments object into a Transaction object, which is
bound to one cursor, and is no longer shared among several cursors.

The following methods/properties have been changed:
 - Environment.envs no longer works (because of the design change);
 - Environment.manage() is deprecated (no longer useful);
 - Environment.reset() is now an instance method;
 - env.clear_upon_failure() is deprecated in favor of cr.savepoint().

closes odoo/odoo#75598

Related: odoo/enterprise#20451
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
2021-09-03 15:45:46 +00:00
Martin Trigaux 9aedb999f5 [IMP] *: remove xmlid_to_object helper
Use env.ref instead of the xmlid_to_object
To make it more readable and be able to clean old helpers
2021-08-10 14:24:11 +02:00
Martin Trigaux c7bac3dee0 [IMP] *: make ir.model.data helper private
No reason to interfact with them directly in RPC
2021-08-10 13:49:04 +02:00
Raphael Collet 3deafa8ebc [FIX] core: enable callable in api.constrains()
This avoids a call to __init__() which no longer has access to the
model's class' parent classes, and was not even correct...

closes odoo/odoo#70400

Related: odoo/enterprise#18141
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-05-06 07:30:29 +00:00
Raphael Collet 34d6f87d54 [REF] core: put field.depends on registry and make field.recursive explicit
The attributes field.depends and field.depends_context are problematic
for sharing fields across registries, because they depend on the model's
registry class, which may vary from one registry to another.  In order
to make computed fields shareable, we have to move those values away
from fields.

For the same reason, field.recursive should not be inferred, because its
value may depend on the registry, although it is generally not the case.
Moreover, the flag recursive=True is set on a field when field triggers
are determined (on the registry).  A compute method may be called before
the flag is set (if no update has been done yet), and that can lead to
incorrect computations.

This happened in test TestUsers2.test_reified_groups in module 'base'.
The user groups view was apparently determined without the flag being
set, and the view depends on the recursive field 'trans_implied_ids',
which was not correctly computed.

We thus force developers to be explicit about recursive computed fields.
The code now logs a warning when the flag is not set up properly.
2021-05-03 12:33:29 +00:00
Adrian Torres 1c8a958098 [IMP] core: introduce api.ondelete decorator
With this commit, a new ORM api decorator is introduced:
`api.ondelete(*, at_uninstall)`.

This decorator is to be applied to Model methods that check for specific
business conditions when attempting to unlink a record via the
interface.

E.g. trying to unlink a validated journal entry

This decorator allows this logic to exist outside the `BaseModel.unlink`
method and is automatically bypassed when in uninstall mode, this means
that during an uninstall any and all data related to a module can and
will be removed easily and cleanly while still being able to apply
business logic to manual deletion of records.

This feature opens the gates to solving a very big problem with
uninstalls: records and tables that remain in a database despite the
relevant module being uninstalled, because if an override to unlink
raises an error, the data will never be deleted from the database.

Henceforth, overrides of unlink shall solely be used for data-cleaning
purposes, i.e. deletion or modification of data that is related to the
one currently being deleted but cannot be automatically deleted because
there are no proper SQL relations.

In certain very specific, low-level scenarios an unlink may be
overridden to raise an error, but this should only be done if you know
what the fuck you're doing, most of the time you'll want to resort to
`@api.ondelete`.

Note that this new decorator includes a keyword-only, required argument
called `at_uninstall`, in most business cases this argument shall be
False as this argument dictates whether or not this method should be
executed in the `unlink` call during uninstall. It should only be set to
True if you are certain of all the implications which most likely means
that the records of the model in which this ondelete function is defined
will NOT be removed during uninstall, they will forever linger in the DB
until manual intervention, this in turn can mean a wide range of
undefined problems due to crap left on the database.

Following commits will replace any `unlink` overrides that raise
business errors by methods decorated with `api.ondelete`, another commit
will introduce a pylint checker that will raise a warning any time that
an unlink override raises an error.
2020-12-04 09:15:51 +00:00
Victor Feyens f7782f6c76 [FIX] core: consistent environment for env.company(ies)
Before this commit, the company recordset returned by
env.company/env.companies
was sometimes in sudo and sometimes not.

If 'allowed_company_ids' was specified in the context, the returned
recordset wasn't sudoed.
If no company was specified through the context, company/companies was
using env.user.company_id(s)
as fallback, which is always sudoed since env.user is always sudoed.

With this commit, the behavior is now consistent, we enforce that the
returned recordset is only sudoed if the current environment is.
As everybody has read access on res.company records, there is no need
to return the company as a sudoed record.
When a specific operation has to be done on the company, then a sudo
should be called if needed.

closes odoo/odoo#62074

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-12-03 14:25:12 +00:00
Raphael Collet a40511cd29 [FIX] core: env.clear() also invalidates lazy properties
In unit tests, this avoids side effects from one test to another.  In
particular, a test modifying the user's company can make other tests
fail because `env.company` defaults to the user's company when nothing
else is available in the context.
2020-11-24 13:23:32 +00:00
Raphael Collet 9b7a4e1815 [IMP] core: smaller and faster field cache
Consider a context-dependent field and two possible cache structures for
storing the data of that field for N records and K context keys.  The
common use-case is to assume that N is much larger than K.  We look at
how many dicts are used in each cache structure, where we label dicts
with N entries as "large", and dicts with K entries as "small".

     Structure                 | Number of dicts
    ---------------------------+-------------------
     cache[field][id][key]     | 1 large + N small
     cache[field][key][id]     | 1 small + K large

It is known that "small" dicts have a large relative overhead when
compared to "large" dicts.  Given this fact, the second cache structure
will generate less memory overhead.  Here are numbers for N=16, showing
the size in bytes taken by the cache structure.  Larger values of N lead
to larger size differences, always in favor of the second structure.

     Structure             |  K=1 |  K=2 |  K=3 |  K=4 |  K=5 |  K=6
    -----------------------+------+------+------+------+------+------
     cache[field][id][key] | 4488 | 4488 | 4488 | 4488 | 4488 | 6536
     cache[field][key][id] |  888 | 1536 | 2184 | 2832 | 3480 | 4256

Moreover, some memory operations are performed on many records and a
single context key.  Those operations are clearly simpler and faster
with the second cache structure.
2020-07-24 08:30:00 +00:00
Raphael ColletandXavier-Do 6d5da2d2d3 [FIX] core: prefetching of context-dependent fields
Consider a context-dependent field, and successively access it on a
recordset with different contexts.  On the first context, the field is
correctly computed in batch.  After that, the field is always computed
one by one.

The bug is in the method that determines which records in a given set
have no value in cache.  On the first context, the cache is empty for
the field, so all records are returned.  After that, the method
considers that all records have a value in cache: they do, but for
another context key!  Simply using the context key when looking up the
cache fixes the issue.

closes odoo/odoo#52360

X-original-commit: 35d69589d9b43afe6c6fc9779458323f7180153e
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Co-authored-by: Xavier-Do <xdo@odoo.com>
2020-06-03 13:00:36 +00:00
c5d3a109f5 [REF] ir.autovacuum: declarative garbage collector registration
The ir.autovacuum model purpose is to run several garbage collecting
operations like removing files from the filestore when no attachment
references them anymore.

The precedent strategy to register new garbage collection tasks was to
override the `power_on` method and to imperatively execute a vacuum
cleaning method on a given model. All calls were executed in a single
SQL transaction without any error handling, meaning a single fail during
any call resulted in a complete failure of the entire vacuum cleaning
chain.

We introduce a new `@autovacuum` api decorator, its purpose it to
register garbage collecting methods that will be safely executed in
their own transaction by the vacuum cleaner. In order to ensure this
new strategy is used, we deprecate `power_on` extensions.

By the way, garbage-collecting methods can be quite heavy and we don't
want users to directly call them. We now ensure they are private.

closes odoo/odoo#47842

Task: 2154079
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Olivier Dony <odo@odoo.com>
2020-05-19 13:38:19 +00:00
Xavier Morel fe376d50d0 [FIX] core, base: leftover deprecation warnings
* from collections import <ABC> is deprecated, unclear why the
  deprecation warning didn't appear before (possibly only appears in
  3.7/3.8?) either way `collections.abc` should be 3.3+ so switch
  everything to it.
* add some more ignores on third-party packages deprecation
  warnings (meh)
* while at it, mitigate generation of non-breaking space on some
  versions of Babel (in the french locale used by our tests anyway)

closes odoo/odoo#47581

Related: odoo/enterprise#9214
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-03-19 09:32:21 +00:00
Denis Ledoux c5b9bccf6b [FIX] doc: api.model no longer accept traditional style call
Since we dropped the old-api style compatibility layer,
some years ago.

This docstring was added to the 13.0 documentation
thanks to odoo/odoo#44838
So it's best not to mention this old-api style
in our latest documentation

closes odoo/odoo#46223

X-original-commit: 63691c78006fbf7077eb13f3045be21a18bbe753
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2020-02-25 12:06:06 +00:00
Nicolas Martinelli 12af185961 [FIX] api, product: search by pricelist
- Activate variants and pricelists
- Go to Product > Products Variants
- Search for anything on the 'Pricelist' filter

A traceback is raised: 'Can only create cache keys from hashable
values...'.

This comes from the following change:

https://github.com/odoo/odoo/blob/4b06fe19fa68255b7982d15e5847da2f6d6209fd/addons/web/static/src/js/views/control_panel/control_panel_model.js#L962

It returns a list instead of a string. Since a list is not hashable, it
causes the issue.

There are not much solutions since the context key `pricelist` can be a
`list` or an `int` (an ID). We force the cache key conversion to a tuple
to avoid the `TypeError` and handle the `list` use case (which was
broken on top of crashing).

opw-2187757

closes odoo/odoo#45064

X-original-commit: ec0a8767143b4c4818418281803c4f52fefa8d2a
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
2020-02-11 11:05:09 +00:00
Adrian Torres 1d13928764 [IMP] core: improve mapped and filtered performance
Previously, mapped was following a very naive approach, which was simply
calling the field name passed as input for every record in a recordset,
sequentially.

The problem with this approach is that we will potentially recompute the
same fields multiple times for differents records, when this could be
done once per field for ALL records, and store this value in cache for
further access.

Another potential problem is that we don't take advantage of the ORM's
prefetching to fetch all the records that are not in cache at once,
instead of doing the same query for every record in the recordset.

Yet another problem is the conversion of each cache value to a record
format and then combining all of the individual records into a single
recordset, which, depending on the size of the recordset, can take an
unbelievable amount of CPU time.

With this new implementation of `mapped()` we take care of all of these
problems:

This is done by first delegating `mapped()` from the model to the field,
this mapped takes a recordset as input and it will try to batch compute
and prefetch as much as possible for the entire recordset, but it will
not keep these values for the actual output, it just stores everything
in cache and then at the end, retrieves everything from the cache to
guarantee the same order.

After the mapped, the conversion from cache format to record format is
delegated to the new `convert_to_record_multi` which will fetch all the
ids and then perform a single browse to encapsulate all of the records
into a single recordset with the least amount of overhead possible.

Part of Task 2170344

closes odoo/odoo#42611

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-01-21 14:06:50 +00:00
Adrian Torres be01db2f71 [IMP] api: move cache_key to the Environment and cache it
Computing the `cache_key` turned out to be a big factor during the
lifespan of a `BaseModel.mapped` call and a lot of this time is spent
computing the same `cache_key` over and over.

These unnecessary computations can be easily reduced to a couple by
moving the `cache_key` method on the environment (instead of the field)
and by implementing a memo for that method.  The rationale is that the
`cache_key` of a field does not change for a given environment.

The result of this patch is up to 50% faster `Field.__get__` which in
turn means a GLOBAL gain in performance, especially for methods /
functions that rely heavily on `__get__` such as `BaseModel.mapped`.

closes odoo/odoo#42674

Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
2020-01-21 08:28:36 +00:00
Xavier Morel 17ad6be46a [FIX] base: returning a domain from an onchange
Turns out we've got an operator just for the pattern of "match value
if there's one, otherwise match everything".

Also remove an example of domain in onchange doc.

closes odoo/odoo#40938

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2019-11-27 09:59:51 +00:00
Victor Feyens 3455d02189 [IMP] base: remove force_company
From now on, if one wants to force following operations to happen in a given company,
use with_company(company) or with_company(cid) to update the environment.
2019-11-18 12:25:05 +00:00
Victor Feyens ae4855fa1e [IMP] base, doc: orm page refactoring.
closes odoo/odoo#39725

X-original-commit: 7ce7c58592c7c7202a37c73767c477be5a835752
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2019-11-04 11:16:12 +00:00
Victor FeyensandRaphael Collet f76c5a8bbd [IMP] ORM: do not allow invalid allowed_company_ids context key.
Since https://github.com/odoo/odoo/commit/a5b6f31cf28e5381e1c85f66730bcdb55998e643,
the current companies of the user are saved in the context as "allowed_company_ids"
context key.

In case of invalid context content, the api was computing the intersection
between the context content and the user company(ies) and falling back on user
company_id(s) when catching an error.

A sanity check was done, but no feedback was given to the user,
saying that the context change was falsy.

This commits changes this behavior to :

* raise an AccessError when trying to access self.env.company(ies) when
invalid or unauthorized companies are defined in the context.

* take sudo mode into consideration, allowing inter-company impacts,
even when current user doesn't have access to a given company,
if the code is done in a sudoed environment.

Co-Authored-By: Raphael Collet <rco@odoo.com>
2019-10-24 17:12:16 +00:00
Christophe Simonis d74b451805 [MERGE] forward port branch 13.0 up to f4105eb9c7 2019-10-09 02:08:17 +02:00
Adrian Torres af46a5c5a4 [DOC] api: warn about onchange pitfalls
closes odoo/odoo#37836

X-original-commit: a8454381ad2151136ee2a49665f69a12b8dc7daf
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
2019-10-02 17:45:13 +00:00
Raphael Collet 492e4cc5a1 [REF] fields: setup and use of depends_context
closes odoo/odoo#36795

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-12 14:24:58 +00:00
Denis Ledoux 3d1c924cff [FIX] fields.py: improve computation time of records having a different value in cache
Instead of using a filtered with `cache.get`,
use a dedicated method in the Cache class to get
the records having a different value in cache then asked.

This is mainly to avoid the creation of intermediate
`browse` of 1 record, when doing `for rec in self`
in `filtered`.

Creating browses is costly, and avoiding it leads
to performance gains.

The dedicated method `get_records_different_from`
loops on the record ids, instead of on browse records
2019-09-11 07:55:17 +00:00
Denis Ledoux 0e028de72e [IMP] api: lazy_property for user, company, companies
These variable are not supposed to change within a same
environment.

The lazy property will compute these variables only
once, then store the result,
while the property were computing these variables
each time they were called.

e.g. for a 1000 iteration loop,
with `env.company`,
the company was computed 1000 times.
With a lazy property, the company will be computed one time only.
2019-09-11 07:55:17 +00:00
Raphael Collet cfb08e87b0 [FIX] models: use transitive triggers instead of recursion
On average, this reduces the total time spent in method `modified` by
half (times measured on invoice creation and post).
2019-09-09 13:18:48 +00:00
Julien Castiaux 4f03a5f136 [FIX] *: remove old deprecated modules/functions
PEP-594 is deprecating a bunch of modules. As part of the cleanup, we
are also dealing with long deprecated modules, functions and aliases.

* `assert_` -> `assertTrue`
* `assertEquals` -> `assertEqual`
* `assertNotEquals` -> `assertNotEqual`
* `assertAlmostEquals` -> `assertAlmostEqual`
* `assertRaisesRegexp` -> `assertRaisesRegex`
* `assertRegexpMatches` -> `assertRegex`
* `base64.encodestring` -> `base64.encodebytes`
* `base64.decodestring` -> `base64.decodebytes`
* `inspect.getargspec` -> `inspect.signature`
* `inspect.formatargspec` -> `inspect.signature`
* `logging.warn` -> `logging.warning`

closes odoo/odoo#36863

Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-17 11:36:42 +00:00
Fabien Pinckaers 0ec6acc458 [FIX] base: multi-company code 2019-08-26 10:20:13 +00:00