[FIX] core: propagate update of modifiers attributes from base terms

When we update an inline translated element (like `<span>`) for the base
language `en_US` we should update the modifiers attributes for all
languages.

For example when updating from (16.0)
https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/views/ir_ui_view_views.xml#L127-L129
https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/i18n/fr.po#L7233-L7235
to (17.0)
https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/views/ir_ui_view_views.xml#L126-L128
https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/i18n/fr.po#L12087-L12089

The base term, `en_US`, is updated since the text matches but the
attributes of the translated terms, `fr_FR` for example, are not
updated. Later when the PO file is loaded in non-overwrite mode the
translated terms for `fr_FR` is not updated. This causes all sort of
issues during an upgrade for inline-translated terms -- like `<span>`.
More so since the recent change that converts domain-based attributes
into inline Python expressions.

In this patch we propagate modifiers attributes from inline-translated
items in the new base term into all translated terms when the base term
is updated. In that way we ensure the attributes are correct in all
languages even if later the loading of their corresponding PO file
doesn't update the term.

For a detailed example, let's see what happens when loading the view
above during an upgrade 16->17, right at the first load of the XML file
at https://github.com/odoo/odoo/blob/b1461d28/odoo/fields.py#L1864
```
(Pdb) p old_term
'<span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'soft\')]}">This view has no previous version.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'hard\')]}">This view is not coming from a file.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'other_view\')]}">You need two views to compare.</span>'
(Pdb) p closest_term
'<span invisible="reset_mode != \'soft\'">This view has no previous version.</span>\n                        <span invisible="reset_mode != \'hard\'">This view is not coming from a file.</span>\n                        <span invisible="reset_mode != \'other_view\'">You need two views to compare.</span>'
(Pdb) p translation_dictionary[old_term]
defaultdict(<class 'dict'>, {'fr_FR': '<span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'soft\')]}">Cette vue n\'a pas de version antérieure.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'hard\')]}">Cette vue ne provient pas d\'un fichier.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'other_view\')]}">Vous avez besoin de deux vues pour comparer.</span>'})
```
As we can see the new term will get an updated value for its `invisible`
attribute, while also removing `attrs`. The translated terms will be
still keep the old modifier though.
Now later when the fr_FR.po file is loaded we reach this point
https://github.com/odoo/odoo/blob/b1461d28/odoo/tools/translate.py#L1442
```
(Pdb) p term_en
'<span invisible="reset_mode != \'soft\'">This view has no previous version.</span>\n                        <span invisible="reset_mode != \'hard\'">This view is not coming from a file.</span>\n                        <span invisible="reset_mode != \'other_view\'">You need two views to compare.</span>'
(Pdb) p translation_dictionary[term_en]
defaultdict(<function DeepDefaultDict at 0x7fc9640cdfc0>, {'fr_FR': '<span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'soft\')]}">Cette vue n\'a pas de version antérieure.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'hard\')]}">Cette vue ne provient pas d\'un fichier.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'other_view\')]}">Vous avez besoin de deux vues pour comparer.</span>'})
```
Thus the translated values are NOT updated, keeping the _wrong_
modifiers. This is later fixed during the upgrade in a clumsy way.
C.f. the warnings like this one in runbot:
```
Incomplete conversion for view(id=77, lang=fr_FR) at
<span attrs="{'invisible': [('reset_mode', '!=', 'soft')]}">Cette vue n'a pas de version ant&#233;rieure.</span>
```
Note that such warnings are gone in current PR CI.
The root issue here is that when the fr_FR translation is loaded the
terms are not updated due to a combination of factors:
1. The text content of the term didn't change
2. There is no override flag set for translations

Option 2 is not a valid option during upgrades because we want to keep
custom translations. We could instead of the current patch tweak how
option 1 works and perhaps make the closest term more restricted. This
would lead to the update of the whole translation though while the
actual issue here is _just_ the modifiers. Moreover if the translations
are out of sync the translated terms will still keep the wrong values
that could still be essential for the correct functioning of the record
they belong too (view archs -- for example).

Finally this is a more extreme case (16.0):
https://github.com/odoo/enterprise/blob/1e63b4a8/sale_subscription/views/sale_order_views.xml#L142
In this case during the upgrade we fix the modifier value (refer to
runbot warning above -- it's the same script that fixes it) and set
```
invisible="(subscription_management == 'upsell') or (recurrence_id == False)"
```
for translations, which is wrong. The correct value is (17.0):
```
invisible="not plan_id or subscription_state == '7_upsell'"
```
https://github.com/odoo/enterprise/blob/530ba3ad/sale_subscription/views/sale_order_views.xml#L136

closes odoo/odoo#150152

Signed-off-by: Raphael Collet <rco@odoo.com>

odoo/addons/base/tests/test_translate.py

@@ -1105,6 +1105,58 @@ class TestXMLTranslation(TransactionCase):
         self.assertEqual(view.with_env(env_fr).arch_db, archf % (terms_fr[0], terms_en[1], terms_en[2]))
         self.assertEqual(view.with_env(env_nl).arch_db, archf % (terms_nl[0], terms_en[1], terms_en[2]))
 
+    def test_sync_xml_inline_modifiers(self):
+        """ Check translations of 'arch' after xml tags changes in source terms
+            when source term had updated modifiers attributes
+        """
+        archf = '''<form class="row">
+    %s
+    <div class="s_table_of_content_vertical_navbar" data-name="Navbar" contenteditable="false">
+        <div class="s_table_of_content_navbar" style="top: 76px;"><a href="#table_of_content_heading_1672668075678_4" class="table_of_content_link">%s</a></div>
+    </div>
+    <div class="s_table_of_content_main" data-name="Content">
+        <section class="pb16">
+            <h1 data-anchor="true" class="o_default_snippet_text" id="table_of_content_heading_1672668075678_4">%s</h1>
+        </section>
+    </div>
+</form>'''
+        terms_en = ('Bread and cheese', 'Knive and Fork', 'Knive <span style="font-weight:bold">and</span> Fork')
+        terms_fr = ('Pain et fromage', 'Couteau et Fourchette', 'Couteau <span style="font-weight:bold">et</span> Fourchette')
+        terms_nl = ('Brood and kaas', 'Mes en Vork', 'Mes en Vork')
+        view = self.create_view(archf, terms_en, en_US=terms_en, fr_FR=terms_fr, nl_NL=terms_nl)
+
+        env_nolang = self.env(context={"install_mode": True})
+        env_en = self.env(context={'lang': 'en_US', "install_mode": True})
+        env_fr = self.env(context={'lang': 'fr_FR', "install_mode": True})
+        env_nl = self.env(context={'lang': 'nl_NL', "install_mode": True})
+
+        self.assertEqual(view.with_env(env_nolang).arch_db, archf % terms_en)
+        self.assertEqual(view.with_env(env_en).arch_db, archf % terms_en)
+        self.assertEqual(view.with_env(env_fr).arch_db, archf % terms_fr)
+        self.assertEqual(view.with_env(env_nl).arch_db, archf % terms_nl)
+
+        # modify attributes in source term
+        terms_en = ('Bread and cheese', 'Knife and Fork', 'Knife <span invisible="1">and</span> Fork')
+        view.with_env(env_en).write({'arch_db': archf % terms_en})
+        terms_fr = ('Pain et fromage', 'Couteau et Fourchette', 'Couteau <span style="font-weight:bold" invisible="1">et</span> Fourchette')
+
+        # check whether translations have been kept
+        self.assertEqual(view.with_env(env_nolang).arch_db, archf % terms_en)
+        self.assertEqual(view.with_env(env_en).arch_db, archf % terms_en)
+        self.assertEqual(view.with_env(env_fr).arch_db, archf % terms_fr)
+        self.assertEqual(view.with_env(env_nl).arch_db, archf % terms_nl)
+
+        # modify attributes in source term
+        terms_en = ('Bread and cheese', 'Knife and Fork', 'Knife <span readonly="1">and</span> Fork')
+        view.with_env(env_en).write({'arch_db': archf % terms_en})
+        terms_fr = ('Pain et fromage', 'Couteau et Fourchette', 'Couteau <span style="font-weight:bold" readonly="1">et</span> Fourchette')
+
+        # check whether translations have been kept
+        self.assertEqual(view.with_env(env_nolang).arch_db, archf % terms_en)
+        self.assertEqual(view.with_env(env_en).arch_db, archf % terms_en)
+        self.assertEqual(view.with_env(env_fr).arch_db, archf % terms_fr)
+        self.assertEqual(view.with_env(env_nl).arch_db, archf % terms_nl)
+
     def test_cache_consistency(self):
         view = self.env["ir.ui.view"].create({
             "name": "test_translate_xml_cache_invalidation",

odoo/fields.py

@@ -1854,6 +1854,7 @@ class _String(Field):
                 text2terms[self.get_text_content(term)].append(term)
 
             is_text = self.translate.is_text if hasattr(self.translate, 'is_text') else lambda term: True
+            term_adapter = self.translate.term_adapter if hasattr(self.translate, 'term_adapter') else None
             for old_term in list(translation_dictionary.keys()):
                 if old_term not in new_terms:
                     old_term_text = self.get_text_content(old_term)
@@ -1863,7 +1864,11 @@ class _String(Field):
                         old_is_text = is_text(old_term)
                         closest_is_text = is_text(closest_term)
                         if old_is_text or not closest_is_text:
-                            translation_dictionary[closest_term] = translation_dictionary.pop(old_term)
+                            if not closest_is_text and records.env.context.get("install_mode") and lang == 'en_US' and term_adapter:
+                                adapter = term_adapter(closest_term)
+                                translation_dictionary[closest_term] = {k: adapter(v) for k, v in translation_dictionary.pop(old_term).items()}
+                            else:
+                                translation_dictionary[closest_term] = translation_dictionary.pop(old_term)
             # pylint: disable=not-callable
             new_translations = {
                 l: self.translate(lambda term: translation_dictionary.get(term, {l: None})[l], cache_value)

odoo/tools/translate.py

@@ -292,6 +292,47 @@ def parse_xml(text):
 def serialize_xml(node):
     return etree.tostring(node, method='xml', encoding='unicode')
 
+
+MODIFIER_ATTRS = {"invisible", "readonly", "required", "column_invisible", "attrs", "states"}
+def xml_term_adapter(term_en):
+    """
+    Returns an `adapter(term)` function that will ensure the modifiers are copied
+    from the base `term_en` to the translated `term` when the XML structure of
+    both terms match. `term_en` and any input `term` to the adapter must be valid
+    XML terms. Using the adapter only makes sense if `term_en` contains some tags
+    from TRANSLATED_ELEMENTS.
+    """
+    orig_node = parse_xml(f"<div>{term_en}</div>")
+
+    def same_struct_iter(left, right):
+        if left.tag != right.tag:
+            raise ValueError("Non matching struct")
+        yield left, right
+        left_iter = left.iterchildren()
+        right_iter = right.iterchildren()
+        for lc, rc in zip(left_iter, right_iter):
+            yield from same_struct_iter(lc, rc)
+        if next(left_iter, None) is not None or next(right_iter, None) is not None:
+            raise ValueError("Non matching struct")
+
+    def adapter(term):
+        new_node = parse_xml(f"<div>{term}</div>")
+        try:
+            for orig_n, new_n in same_struct_iter(orig_node, new_node):
+                removed_attrs = [k for k in new_n.attrib if k in MODIFIER_ATTRS and k not in orig_n.attrib]
+                for k in removed_attrs:
+                    del new_n.attrib[k]
+                keep_attrs = {k: v for k, v in orig_n.attrib.items() if k in MODIFIER_ATTRS}
+                new_n.attrib.update(keep_attrs)
+        except ValueError:  # non-matching structure
+            return term
+
+        # remove tags <div> and </div> from result
+        return serialize_xml(new_node)[5:-6]
+
+    return adapter
+
+
 _HTML_PARSER = etree.HTMLParser(encoding='utf8')
 
 def parse_html(text):
@@ -380,6 +421,8 @@ html_translate.term_converter = html_term_converter
 xml_translate.is_text = is_text
 html_translate.is_text = is_text
 
+xml_translate.term_adapter = xml_term_adapter
+
 def translate_sql_constraint(cr, key, lang):
     cr.execute("""
         SELECT COALESCE(c.message->>%s, c.message->>'en_US') as message