pengyuthon/odoo_source
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[FIX] stock: performance issue inventory at date

Browse Source 
On a production database with ~10000 products and multiple stock
locations, uunning the 'Inventory at date' tales ~50 minutes. Most of
the time is spent in the loop populating `group_lines` thanks to
multiple search queries.

We get rid of this loop by selecting the stock history lines and making
the match between these lines and the `read_group` result manually. On
the reference database, the duration falls to ~30 seconds.

Based on preliminary work by amoyaux

opw-803460
This commit is contained in:
Nicolas Martinelli
2018-01-29 13:58:18 +01:00
committed by Nicolas Martinelli
parent 6d682cea25
commit 5c5098f740
1 changed files with 28 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
addons/stock_account/report/stock_history.py
+28 -12
View File
@@ -25,22 +25,35 @@ class StockHistory(models.Model):
@api.model
def read_group(self, domain, fields, groupby, offset=0, limit=None, orderby=False, lazy=True):
# Step 1: retrieve the standard read_group output. In case of inventory valuation, this
# will be mostly used as a 'skeleton' since the inventory value needs to be computed based
# on the individual lines.
res = super(StockHistory, self).read_group(domain, fields, groupby, offset=offset, limit=limit, orderby=orderby, lazy=lazy)
if 'inventory_value' in fields:
groupby_list = groupby if not lazy else groupby[:-1]
date = self._context.get('history_date', fieldsDatetime.now())
stock_history = self.env['stock.history']
group_lines = {}
for line in res:
domain = line.get('__domain', domain)
group_lines.setdefault(str(domain), self.search(domain))
stock_history |= group_lines[str(domain)]
# get data of stock_history in one shot to speed things up (the view can be very slow)
# Step 2: retrieve the stock history lines. The result contains the 'expanded'
# version of the read_group. We build the query manually for performance reason
# (and avoid a costly 'WHERE id IN ...').
fields_2 = set(
['id', 'product_id', 'price_unit_on_quant', 'company_id', 'quantity'] + groupby_list
)
tables, where_clause, where_clause_params = self._where_calc(domain).get_sql()
select = "SELECT %s FROM %s WHERE %s "
query = select % (','.join(fields_2), tables, where_clause)
self._cr.execute(query, where_clause_params)
# Step 3: match the lines retrieved at step 2 with the aggregated results of step 1.
# In other words, we link each item of the read_group result with the corresponding
# lines.
stock_history_data = {}
if stock_history:
self._cr.execute("""SELECT id, product_id, price_unit_on_quant, company_id, quantity
FROM stock_history WHERE id in %s""", (tuple(stock_history.ids),))
stock_history_data = {line['id']: line for line in self._cr.dictfetchall()}
stock_histories_by_group = {}
for line in self._cr.dictfetchall():
stock_history_data[line['id']] = line
key = tuple(line[g] if g in line else False for g in groupby_list)
stock_histories_by_group.setdefault(key, [])
stock_histories_by_group[key] += [line['id']]
histories_dict = {}
not_real_cost_method_products = self.env['product.product'].browse(
@@ -56,7 +69,10 @@ class StockHistory(models.Model):
for line in res:
inv_value = 0.0
for stock_history in group_lines.get(str(line.get('__domain', domain))):
# Build the same keys than above, but need to take into account Many2one are tuples
key = tuple(line[g] if g in line else False for g in groupby_list)
key = tuple(k[0] if isinstance(k, tuple) else k for k in key)
for stock_history in self.env['stock.history'].browse(stock_histories_by_group[key]):
history_data = stock_history_data[stock_history.id]
product_id = history_data['product_id']
if self.env['product.product'].browse(product_id).cost_method == 'real':