This reverts commit 5f4a945182.
The cure is worse than the original disease.
The main installation cursor often holds exclusive locks (due to DDL
changes) on vital tables such as res_users. As a result, using
another cursor to perform changes while the main cursor is waiting
is extremely deadlock-prone. And these deadlocks can't be detected by
PostgreSQL as they mix Python-SQL locking, which leads to deadlocked
HTTP workers.
Related to:
- opw-1916918
- #29528
- Create a new DB with `base_automation`, without demo data
- Switch to developer mode
- Go to Settings, then 'Load demo data', validate
A traceback occurs.
An error since the demo data in `base_automation_demo.xml` are pointing
to `model_base_automation_lead_test`, which is a test model. It is
therefore not created whithout demo data. The error is:
```
bad query: b'RELEASE SAVEPOINT "5159....."'
```
However, that's not the real crash. Indeed, in case the installation of
demo data crashes, `demo_failure_todo` is supposed to handle the
situation like a boss. But in this case, the DB cursor seems unusable.
Therefore, the following crashes like a gros caca:
``` python
todo = env.ref('base.demo_failure_todo', raise_if_not_found=False)
```
Solution: use a new cursor.
opw-1916918
closesodoo/odoo#29528
After a module upgrade, validate the architecture of the module's views that
are impacted by updates, but have not been checked yet. Before this patch,
views were checked twice on average.
This patch speeds up the installation of modules by about 15% without demo
data. With demo data, the speedup is around 10%.
The registry loading system should not alter databases unless it is
asked to do so, by a module installation or update instruction.
This property should hold true as well for database bootstrap, and this
is what this commit changes..
In order to avoid any behavior change for command-line users, an
implicit `-i base` is assumed when starting the server from the
command-line with `-d <db>`, causing the db boostrap to happen if the
database did not exist yet.
Task 1856935
Currently, if installing a module's demo data fails (because the
module was uninstalled and some were left over and can't be
reinstalled, or because they're broken, or…) the installation of the
module fails and further installations are skipped (?).
Since demo data are non-essential (though required to run tests),
rather than fail everything if they fail just roll them back and
notify the user.
This change generates a warning log *and* shows a notification popup
to the end user.
Try to remove cr.commit (and rollback) from module and db install:
* put a savepoint around test data loading
* remove a bunch of commits sprinkled throughout
* remove rollback on data loading failure (assuming it bubbles up, the
entire module's installation should be rolled back)
* add commit right before the tests are run, so they can run isolated
and still see whatever was done when installing their module
* convert a few explicit closing to context managers
Before this commit:
* Module A defines a field X of model M
* Module B inherits from model M without touching field X
* Module C inherits from model M and extends field X by giving an INDEX
/ NOT NULL constraint.
* Module B and C depend from Module A, but not each other
If all three modules are installed and Module B is updated, the INDEX /
NOT NULL constraint could be dropped.
This happens because Module B can be loaded before Module C is loaded,
if that's the case, then after the upgrade of Module B, during the
schema checking, we verify that the field object we have and the field
on the DB are the same, since Module B doesn't introduce the index then
this check is false and we drop the index. When we get to loading Module
C, we do not do any schema checking because the module is not marked as
`to upgrade`, therefore the index is lost forever.
To solve this, we re-init the models that belong to the set of the intersection
between upgraded and modified models and loaded and modified models.
Fixes#24958
Before this commit:
* Module A defines a field X of model M
* Module B inherits from model M without touching field X
* Module C inherits from model M and extends field X by giving an INDEX
/ NOT NULL constraint.
* Module B and C depend from Module A, but not each other
If all three modules are installed and Module B is updated, the INDEX /
NOT NULL constraint could be dropped.
This happens because Module B can be loaded before Module C is loaded,
if that's the case, then after the upgrade of Module B, during the
schema checking, we verify that the field object we have and the field
on the DB are the same, since Module B doesn't introduce the index then
this check is false and we drop the index. When we get to loading Module
C, we do not do any schema checking because the module is not marked as
`to upgrade`, therefore the index is lost forever.
To solve this, we re-init the models that belong to the set of the intersection
between upgraded and modified models and loaded and modified models.
Fixes#24958
Before this commit:
* Install any module that creates a SQL view and another that extends
this view.
e.g.: sale and pos_sale for `report.all.channels.sales`.
* Uninstall the module that extended the SQL view.
e.g.: uninstall pos_sale.
* Try to access the view from the web client -> Traceback, table not
found.
This happens because when reloading the registry, postgres drops the sql
view and it must be re-initialized.
After this commit:
We solve this issue by calculating all missing tables/views during the
uninstallation process, and re-initializing all of them right before
reloading the registry.
Also remove sql-view hacks in the modules `sale` and `sale_margin`.
Fixes#23528, #23529, #23530
Commit 763d714 introduced cron job locking for databases which had
modules with states set to 'to x', however if an
installation/uninstallation/upgrade fails, the state will stay at 'to
x', and it may stay in that state for an indefinite amount of time,
meaning that cron jobs could stay locked forever.
This commit fixes this in part by adding a cleanup function to loading.py that
will be executed whenever load_modules fails, the function will change
every 'to x' module to their original state, effectively unlocking the
execution of cron jobs.
This however only works to prevent "zombie" transient states for
brand new databases, however for existing databases which already
contain some modules in a zombie state it won't do anything unless
a module is installed/uninstalled/upgraded, which may never happen.
This is where the second part comes in (ir_cron.py), when failing to
execute crons, we check if the failure was due to bad module state
and if an arbitrary amount of time (5 hours as of this commit) has passed
since the last time it was supposed to be executed, if it is the case, it means
that the cron execution failed around 5 * 60 times (1 failure per minute for 5h)
in which case we assume that the crons are stuck because the db
has zombie states and we force a call to reset_module_states.
Previous to this rev., sometimes None could be added
to the list of _init_modules in the registry, this would
then be problematic in ir_http since a sorted would be
performed on this list, which works in py2 but in py3
None and string can't be compared implicitly.
Was only done when updating base.
The cost of scanning the addons path should small in comparison to the update
time itself.
Followup of #9133Closes#9140
Now that we're closer to switching to P3 for good, these helpers have
outlived their usefulness, and mostly add noise.
All remaining dict.iter*() or dict.view*() must be converted to the
normal keys(), values() or items() calls.
Whenever the result is likely to be used for more than the scope of a
loop, or when the dict needs to be modified during iteration, the calls
must be wrapped in a ``list()``, to protect the new P3 semantics.
Those cases are very exceptional.
Also removed some dead code or improved the API to remove unnecessary
conversions.
In Python 3:
* various builtins and dict methods were changed to return
view/iterable objects rather than lists
* and the separate Python 2 view/iterable builtins and methods were
removed altogether
This is problematic when using these items as list (which the happens
repeatedly in Odoo), but more viciously when iterating *multiple times*
over them (which also happens, which I've messed up multiple times while
writing this, and which is a pain to debug even when you've just created
the issue).
Convert all code using these to semantics-matching cross-version
helper functions to get the LCD behaviour between P2 and P3, and
forbid the builtins via lint.
issue #8530
Problem: the update of custom models/fields is not fully transactional, and may
potentially lead to an inconsistent database. An other problem is creating two
custom fields by writing on a model: if the second one fails, the first one has
been committed without notice. Retrying the request will give an unexpected
error (duplicate field name).
Solution: never commit in the middle of a request. If the changes have an
impact on the registry, then mark it as invalid (with a new flag), and signal
registry invalidation after everything has been committed. If the request
fails, reset the registry. Both registry and cache invalidation are handled
the same way.
Move the post-install tests execution outside `Registry.new`, and add a flag on
class `HttpCase` to enable/disable the registry "test mode".
This allows a test to run actual transactions that will reload the registry,
which may be used to test the creation of `ir.model` instances, etc.
Olivier Dony