[FIX] base: encoding guessing of html module descriptions

I missed a critical issue in #133708: various users had discovered
they could already fix description issues by adding an XML declaration
to their document which is very cool (though technically not really
valid).

What is a lot less cool is that lxml gets *extremely* unhappy when
asked to parse *strings* with an encoding declaration, raising a
ValueError, so the purported fix breaks on any module which does that,
which seems to include a lot of OCA modules.

Gate the encoding guessing by bailing if the document has an XML
declaration, in which case we just assume the author knows what
they're doing and we leave them alone. For extra safety, check the
encoding declaration in ascii and utf16. Could also have checked for
BOMs, but lxml seems to not care about them overly much (in fact it
seems to prefer them decoded which is odd).

Also same as non-utf8 descriptions, mark XML declarations as
deprecated (because it's a hack to make UTF8 descriptions work which
is not necessary anymore).

closes odoo/odoo#133968

Reported-by: @rezak400
X-original-commit: fd353d7d0104431208b91603431e41ef4a6e54bb
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>

odoo/addons/base/models/ir_module.py

@@ -134,6 +134,13 @@ STATES = [
 ]
 
 
+XML_DECLARATION = (
+    '<?xml version='.encode('utf-8'),
+    '<?xml version='.encode('utf-16-be'),
+    '<?xml version='.encode('utf-16-le'),
+)
+
+
 class Module(models.Model):
     _name = "ir.module.module"
     _rec_name = "shortdesc"
@@ -170,15 +177,25 @@ class Module(models.Model):
             if module_path and path:
                 with tools.file_open(path, 'rb') as desc_file:
                     doc = desc_file.read()
-                    try:
-                        contents = doc.decode()
-                    except UnicodeDecodeError:
+                    if doc.startswith(XML_DECLARATION):
                         warnings.warn(
-                            f"Non-UTF8 module descriptions are deprecated since Odoo 17 ({module.name}'s description is not)",
+                            f"XML declarations in HTML module descriptions are "
+                            f"deprecated since Odoo 17, {module.name} can just "
+                            f"have a UTF8 description with not need for a "
+                            f"declaration.",
                             category=DeprecationWarning,
                         )
-                        contents = doc
-                    html = lxml.html.document_fromstring(contents)
+                    else:
+                        try:
+                            doc = doc.decode()
+                        except UnicodeDecodeError:
+                            warnings.warn(
+                                f"Non-UTF8 module descriptions are deprecated "
+                                f"since Odoo 17 ({module.name}'s description "
+                                f"is not utf-8)",
+                                category=DeprecationWarning,
+                            )
+                    html = lxml.html.document_fromstring(doc)
                     for element, attribute, link, pos in html.iterlinks():
                         if element.get('src') and not '//' in element.get('src') and not 'static/' in element.get('src'):
                             element.set('src', "/%s/static/description/%s" % (module.name, element.get('src')))