[FIX] core: avoid losing translations during module upgrade

Consider field F in module X with parameter translate=False, and F is
overridden in module Y with parameter translate=True.  During the
upgrade of module X, module Y hasn't been loaded yet, and the ORM
considers the field to be `translate=False`.  Therefore it converts its
database column from type jsonb to varchar, and accidentally drops
non-en_US values:

    {"en_US": "English value", "fr_FR": "French value"} (jsonb)
        -> 'English value' (varchar)

As a result, translations are lost after upgrade.

This commit fixes the bug by checking whether the field is translated in
database and patches the field accordingly when loading the registry.
This avoids the ORM considering the field as non-translated while
upgrading modules.

In order to "force" translated fields to become non-translated ones, at
the end of the loading process the patch above is discarded, and fields
are checked again.  We then adapt the schema of models that have such
fields.  This extra step handles the uninstallation of modules like
module Y in the example above.

The patching of the fields has one potential issue.  While upgrading
module X, field F is patched with translate=True.  If module Y actually
overrides F with translate=xml_translate or so, this may cause the
behavior of the upgrade to be slightly incorrect.  Because of the
complexity, we have chosen to not support this case.

closes odoo/odoo#112223

X-original-commit: 1e6e482a9763baa5fae70d9b6676d16a46458f4a
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>

odoo/models.py

@@ -2728,6 +2728,17 @@ class BaseModel(metaclass=MetaModel):
                 for field in klass._field_definitions:
                     definitions[field.name].append(field)
         for name, fields_ in definitions.items():
+            if f'{cls._name}.{name}' in cls.pool._database_translated_fields:
+                # the field is currently translated in the database; ensure the
+                # field is translated to avoid converting its column to varchar
+                # and losing data
+                translate = next((
+                    field.args['translate'] for field in reversed(fields_) if 'translate' in field.args
+                ), False)
+                if not translate:
+                    # patch the field definition by adding an override
+                    _logger.debug("Patching %s.%s with translate=True", cls._name, name)
+                    fields_.append(fields_[0].new(translate=True))
             if len(fields_) == 1 and fields_[0]._direct and fields_[0].model_name == cls._name:
                 cls._fields[name] = fields_[0]
             else:

odoo/modules/loading.py

@@ -411,6 +411,11 @@ def load_modules(registry, force_demo=False, status=None, update_module=False):
             _logger.critical('module base cannot be loaded! (hint: verify addons-path)')
             raise ImportError('Module `base` cannot be loaded! (hint: verify addons-path)')
 
+        if update_module and odoo.tools.table_exists(cr, 'ir_model_fields'):
+            # determine the fields which are currently translated in the database
+            cr.execute("SELECT model || '.' || name FROM ir_model_fields WHERE translate IS TRUE")
+            registry._database_translated_fields = {row[0] for row in cr.fetchall()}
+
         # processed_modules: for cleanup step after install
         # loaded_modules: to avoid double loading
         report = registry._assertion_report
@@ -477,6 +482,23 @@ def load_modules(registry, force_demo=False, status=None, update_module=False):
                     ['to install'], force, status, report,
                     loaded_modules, update_module, models_to_check)
 
+        if update_module:
+            # set up the registry without the patch for translated fields
+            database_translated_fields = registry._database_translated_fields
+            registry._database_translated_fields = ()
+            registry.setup_models(cr)
+            # determine which translated fields should no longer be translated,
+            # and make their model fix the database schema
+            models_to_untranslate = set()
+            for full_name in database_translated_fields:
+                model_name, field_name = full_name.rsplit('.', 1)
+                if model_name in registry:
+                    field = registry[model_name]._fields.get(field_name)
+                    if field and not field.translate:
+                        _logger.debug("Making field %s non-translated", field)
+                        models_to_untranslate.add(model_name)
+            registry.init_models(cr, list(models_to_untranslate), {'models_to_check': True})
+
         registry.loaded = True
         registry.setup_models(cr)
 

odoo/modules/registry.py

@@ -113,6 +113,7 @@ class Registry(Mapping):
         self.models = {}    # model name/model instance mapping
         self._sql_constraints = set()
         self._init = True
+        self._database_translated_fields = ()  # names of translated fields in database
         self._assertion_report = odoo.tests.runner.OdooTestResult()
         self._fields_by_model = None
         self._ordinary_tables = None