pengyuthon/odoo_source
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[FIX] mail,account_edi: fix creation of invoice upon email reception

Browse Source 
When receiving an email on a mailbox with an alias that triggers the
creation of invoices, 4 bugs could occur.

1. If the xml received contains replacement characters (U+FFFD �), and
the charset of the part of the email is "US-ASCII" the encoding of the
string will fail, preventing the rest of the flow to be completed. Be
more resilient, encode the string and ignores these characters if this
case occurs.

NB: sometimes, the charset is omitted for a Content-type: text/xml. This
is valid but not recommended (see:
https://www.ietf.org/rfc/rfc2376.txt). In this case, the default used is
"US-ASCII". This means that any non-ascii char will be lost (they are
replaced by the replacement character: �, see:
https://github.com/python/cpython/blob/3.10/Lib/email/contentmanager.py#L67)
when decoding the attachment.

2. When the xml attachment is created in Odoo, the mimetype is
'text/plain' (rather than 'application/xml'). Thus, the
`_decode_attachment` needs to be more flexible when guessing the type of
the attachment (to know which function to use to read the content of the
attachment and create the invoice).

3. When creating an invoice from an email with an xml attachment, the
xml is attached as the `message_main_attachment_id`. It's only later on
that the content of the xml is read and we possibly find the PDF in
base64 inside. When creating the PDF attachment, it was not set as the
`message_main_attachment_id`, so the PDF was not rendered on the right
part of the invoice form view. Add a clause to replace the
`message_main_attachment_id` in such a case.

4. When the xml attachment represents a credit note, the move_type of
the invoice created by the email alias needs to be changed. Indeed, the
invoice is created before decoding the attachment, so we can only change
the `move_type` later.

opw-3144519
opw-3149649

closes odoo/odoo#121076

X-original-commit: 1e193a92b9c84e75b958985f8067873b90f686e0
Signed-off-by: Laurent Smet <las@odoo.com>
Signed-off-by: Julien Van Roy <juvr@odoo.com>
This commit is contained in:
Julien Van Roy
2023-05-11 08:28:42 +02:00
parent a5f8788332
commit e553f0b372
6 changed files with 110 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
addons/account/models/ir_attachment.py
+4 -3
View File
@@ -97,9 +97,10 @@ class IrAttachment(models.Model):
"""
def is_xml(attachment):
# XML attachments received by mail have a 'text/plain' mimetype.
# Therefore, if content start with '<?xml', it is considered as XML.
is_text_plain_xml = 'text/plain' in attachment.mimetype and attachment.raw.startswith(b'<?xml')
# XML attachments received by mail have a 'text/plain' mimetype (cfr. context key:
# 'attachments_mime_plainxml'). Therefore, if content start with '<?xml', or if the filename ends with
# '.xml', it is considered as XML.
is_text_plain_xml = 'text/plain' in attachment.mimetype and (attachment.raw.startswith(b'<?xml') or attachment.name.endswith('.xml'))
return attachment.mimetype.endswith('/xml') or is_text_plain_xml
return [
addons/account_edi_ubl_cii/models/account_edi_common.py
+16 -2
View File
@@ -281,7 +281,14 @@ class AccountEdiCommon(models.AbstractModel):
else:
return
if not new and invoice.move_type != move_type:
return
# with an email alias to create account_move, first the move is created (using alias_defaults, which
# contains move_type = 'out_invoice') then the attachment is decoded, if it represents a credit note,
# the move type needs to be changed to 'out_refund'
types = {move_type, invoice.move_type}
if types == {'out_invoice', 'out_refund'} or types == {'in_invoice', 'in_refund'}:
invoice.move_type = move_type
else:
return
# Update the invoice.
invoice.move_type = move_type
@@ -317,7 +324,7 @@ class AccountEdiCommon(models.AbstractModel):
# (Windows or Linux style) and/or the name of the xml instead of the pdf.
# Get only the filename with a pdf extension.
name = attachment_name.text.split('\\')[-1].split('/')[-1].split('.')[0] + '.pdf'
attachments |= self.env['ir.attachment'].create({
attachment = self.env['ir.attachment'].create({
'name': name,
'res_id': invoice.id,
'res_model': 'account.move',
@@ -325,6 +332,13 @@ class AccountEdiCommon(models.AbstractModel):
'type': 'binary',
'mimetype': 'application/pdf',
})
# Upon receiving an email (containing an xml) with a configured alias to create invoice, the xml is
# set as the main_attachment. To be rendered in the form view, the pdf should be the main_attachment.
if invoice.message_main_attachment_id and \
invoice.message_main_attachment_id.name.endswith('.xml') and \
'pdf' not in invoice.message_main_attachment_id.mimetype:
invoice.message_main_attachment_id = attachment
attachments |= attachment
if attachments:
invoice.with_context(no_new_invoice=True).message_post(attachment_ids=attachments.ids)
addons/l10n_account_edi_ubl_cii_tests/tests/test_xml_ubl_be.py
+15
View File
@@ -369,3 +369,18 @@ class TestUBLBE(TestUBLCommon):
# source: vat-category-E.xml
self._assert_imported_invoice_from_file(subfolder=subfolder, filename='bis3_tax_exempt_gbp.xml',
amount_total=1200, amount_tax=0, list_line_subtotals=[1200], currency_id=self.env.ref('base.GBP').id)
def test_import_existing_invoice_flip_move_type(self):
""" Tests whether the move_type of an existing invoice can be flipped when importing an attachment
For instance: with an email alias to create account_move, first the move is created (using alias_defaults,
which contains move_type = 'out_invoice') then the attachment is decoded, if it represents a credit note,
the move type needs to be changed to 'out_refund'
"""
invoice = self.env['account.move'].create({'move_type': 'out_invoice'})
self._update_invoice_from_file(
'l10n_account_edi_ubl_cii_tests',
'tests/test_files/from_odoo',
'bis3_out_refund.xml',
invoice,
)
self.assertRecordValues(invoice, [{'move_type': 'out_refund', 'amount_total': 3164.22}])
addons/mail/models/mail_thread.py
+4 -1
View File
@@ -2188,7 +2188,10 @@ class MailThread(models.AbstractModel):
if isinstance(content, str):
encoding = info and info.get('encoding')
content = content.encode(encoding or 'utf-8')
try:
content = content.encode(encoding or "utf-8")
except UnicodeEncodeError:
content = content.encode("utf-8")
elif isinstance(content, EmailMessage):
content = content.as_bytes()
elif content is None:
addons/test_mail/data/test_mail_data.py
+50
View File
@@ -276,6 +276,56 @@ SGVsbG8gd29ybGQK
--Apple-Mail=_9331E12B-8BD2-4EC7-B53E-01F3FBEC9227--
"""
MAIL_MULTIPART_INVALID_ENCODING = """Return-Path: <whatever-2a840@postmaster.twitter.com>
To: {to}
cc: {cc}
Received: by mail1.openerp.com (Postfix, from userid 10002)
id 5DF9ABFB2A; Fri, 10 Aug 2012 16:16:39 +0200 (CEST)
From: {email_from}
Subject: {subject}
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="00000000000005d9da05fa394cc0"
Date: Fri, 10 Aug 2012 14:16:26 +0000
Message-ID: {msg_id}
{extra}
--00000000000005d9da05fa394cc0
Content-Type: multipart/alternative; boundary="00000000000005d9d905fa394cbe"
--00000000000005d9d905fa394cbe
Content-Type: text/plain; charset="UTF-8"
Dear customer,
Please find attached the Peppol Bis 3 attachment of your invoice (with an
encoding error in the address)
Cheers,
--00000000000005d9d905fa394cbe
Content-Type: text/html; charset="UTF-8"
<div dir="ltr">Dear customer,<div><br></div><div>Please find attached the Peppol Bis 3 attachment of your invoice (with an encoding error in the address)</div><div><br></div><div>Cheers,</div></div>
--00000000000005d9d905fa394cbe--
--00000000000005d9da05fa394cc0
Content-Type: text/xml; charset="US-ASCII";
name="bis3_with_error_encoding_address.xml"
Content-Disposition: attachment;
filename="bis3_with_error_encoding_address.xml"
Content-Transfer-Encoding: base64
Content-ID: <f_lgxgdqx40>
X-Attachment-Id: f_lgxgdqx40
PEludm9pY2UgeG1sbnM6Y2JjPSJ1cm46b2FzaXM6bmFtZXM6c3BlY2lmaWNhdGlvbjp1Ymw6c2No
ZW1hOnhzZDpDb21tb25CYXNpY0NvbXBvbmVudHMtMiIgeG1sbnM9InVybjpvYXNpczpuYW1lczpz
cGVjaWZpY2F0aW9uOnVibDpzY2hlbWE6eHNkOkludm9pY2UtMiI+DQo8Y2JjOlN0cmVldE5hbWU+
Q2hhdXNz77+977+9ZSBkZSBCcnV4ZWxsZXM8L2NiYzpTdHJlZXROYW1lPg0KPC9JbnZvaWNlPg0K
--00000000000005d9da05fa394cc0--
"""
MAIL_SINGLE_BINARY = """X-Original-To: raoul@grosbedon.fr
Delivered-To: raoul@grosbedon.fr
addons/test_mail/tests/test_mail_gateway.py
+21
View File
@@ -1742,6 +1742,27 @@ class TestMailgateway(MailCommon):
records = self.env['mail.test.gateway'].search([('name', 'ilike', 'Whitelist test alias loop %')])
self.assertEqual(len(records), 10, msg='Email whitelisted should not have the restriction')
# --------------------------------------------------
# Corner cases / Bugs during message process
# --------------------------------------------------
def test_message_process_file_encoding_ascii(self):
""" Incoming email containing an xml attachment with unknown characters (�) but an ASCII charset should not
raise an Exception. UTF-8 is used as a safe fallback.
"""
record = self.format_and_process(test_mail_data.MAIL_MULTIPART_INVALID_ENCODING, self.email_from, 'groups@test.com')
self.assertEqual(record.message_ids.attachment_ids.name, 'bis3_with_error_encoding_address.xml')
# NB: the xml received by email contains b"Chauss\xef\xbf\xbd\xef\xbf\xbde" with "\xef\xbf\xbd" being the
# replacement character � in UTF-8.
# When calling `_message_parse_extract_payload`, `part.get_content()` will be called on the attachment part of
# the email, triggering the decoding of the base64 attachment, so b"Chauss\xef\xbf\xbd\xef\xbf\xbde" is
# first retrieved. Then, `get_text_content` in `email` tries to decode this using the charset of the email
# part, i.e: `content.decode('us-ascii', errors='replace')`. So the errors are replaced using the Unicode
# replacement marker and the string "Chauss������e" is used to create the attachment.
# This explains the multiple "�" in the attachment.
self.assertIn("Chauss������e de Bruxelles", record.message_ids.attachment_ids.raw.decode())
@tagged('mail_gateway')
class TestMailThreadCC(MailCommon):