[FIX] web: fix web_read_group total groups count

Fetching all the groups from the DB causes MemoryError on some DBs
Example Accounting > Accounting > Journals > Miscellaneous menu:
```
 Traceback (most recent call last):
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 176, in crawl_menu
    self.mock_action(action_vals)
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 267, in mock_action
    mock_method(model, view, fields_list, domain, group_by)
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 380, in mock_view_tree
    self.mock_web_read_group(model, view, domain, group_by, fields_list, limit_group=5)
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 425, in mock_web_read_group
    data = model.web_read_group(domain, fields_list, group_by, limit=limit)["groups"]
   File "/home/odoo/src/odoo/14.0/addons/web/models/models.py", line 96, in web_read_group
    all_groups = self.read_group(domain, ['display_name'], groupby, lazy=True)
   File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2248, in read_group
    result = self._read_group_raw(domain, fields, groupby, offset=offset, limit=limit, orderby=orderby, lazy=lazy)
   File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in _read_group_raw
    result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
   File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in <listcomp>
    result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
 MemoryError
```

This issue was observed during the upgrade requests upg-18830, target
14.0; and upg-61934 (legacy), target 13.0

The issue can be reproduced on a clean DB with just account_accountant
installed and ~2 millions account moves on a Misc journal.

closes odoo/odoo#73855

X-original-commit: a9b3d9cf9bfd2bf4c9b0904059f1606ad6e04ee4
Signed-off-by: Christophe Simonis <chs@odoo.com>
This commit is contained in:
Alvaro Fuentes
2021-07-16 11:29:29 +00:00
parent dc7678ea15
commit aa4fad64ce
+9 -2
View File
@@ -102,8 +102,15 @@ class Base(models.AbstractModel):
if not groups:
length = 0
elif limit and len(groups) == limit:
all_groups = self.read_group(domain, ['display_name'], groupby, lazy=True)
length = len(all_groups)
# We need to fetch all groups to know the total number
# this cannot be done all at once to avoid MemoryError
length = limit
chunk_size = 100000
while True:
more = len(self.read_group(domain, ['display_name'], groupby, offset=length, limit=chunk_size, lazy=True))
length += more
if more < chunk_size:
break
else:
length = len(groups) + offset
return {