william-andre
313d7071a9
Summary: ======== 1. Handle the inverse fields of `account.full.reconcile` manually 2. Batch the update of matching numbers 3. Batch the balance consistency test 4. Batch the construction of reconciliation batches 5. Precompute fields on partials 6. Avoid recomputation of same default analytic distributions 7. Cache the accounting date wrt lock date 1: Handle the inverse fields of `account.full.reconcile` manually ----------------------------------------------------------------- In order to update the inverse fields of `reconciled_line_ids` and `partial_reconcile_ids`, the ORM is triggering one query per full reconcile record. In the case of a big batch payment, that is a lot of round trips. This also bypasses the computation for possible tracking, which is expensive to do for 2 records at a time. That field is not tracked anyway. 2: Batch the update of matching numbers --------------------------------------- Basically the same as 1. without the tracking. 3: Batch the balance consistency test ------------------------------------- Use the various context managers around `__reconcile_plan` to check for the balance consistency only at the end in one single query. This avoids checking it at every single intermediate write done on the lines. Also doing it for the sync of dynamic lines. It should be useless but it doesn't cost anything, and allows to remove the context keys in enterprise without a cost. (they shouldn't be used explicitly, the context managers should always be used instead) Using these context managers also allows to remove all the `skip_` context keys, which should not be used: using them will mean that as soon as the journal entry is modified for any reason, some business fields will change because of the synchronisation since the context won't be the same anymore. This commit therefore also makes `_sync_dynamic_line` more resilient and avoids even more useless syncs. 4: Batch the construction of reconciliation batches --------------------------------------------------- Introduce `_reconciled_by_number`, which is basically a `_read_group`. In conjuction with `_filter_reconciled_by_number`, we can find all the reconciled lines in one single query, for as many batches as needed; as opposed to one query per batch before. 5: Precompute fields on partials -------------------------------- Simply avoid doing one batched `INSERT INTO` followed by one `UPDATE` for each record. 6: Avoid recomputation of same default analytic distributions ------------------------------------------------------------- Not really linked to the reconciliation, but when a lot of lines are using the same parameters for the default analytic distribution, it is useless to re-compute it multiple times. Can speed up the population of test data for instance. 7: Cache the accounting date wrt lock date ------------------------------------------ When preparing the create values for the exchange moves, computing the date with regards to the lock date once per move is very expensive. By creating a context dependent field, we can cache the value and avoid doing the same operation multiple times. Benchmark: ========== We are testing 2 use cases: * a lot of small matchings that are producing a full 2 by 2 * one big matching (also producing a full) To reconcile a batch payment of 500 (identical) lines in the bank reco: | | Queries | SQL | Other | |--------|---------|-------|--------| | Before | 4191 | 4.649 | 16.253 | | After | 226 | 0.989 | 3.581 | To reconcile one big batch of 1000 lines with the generic reco widget (spaghetti): | | Queries | SQL | Other | |--------|---------|-------|-------| | Before | 82 | 0.478 | 1.131 | | After | 66 | 0.634 | 0.974 | To reconcile a batch of 1000 lines *in foreign currency* | | Queries | SQL | Other | |--------|---------|-------|-------| | Before | | | | | After | 45995 | 125.0 | 146.3 | All the times are in seconds. We can see the the use case of the batch payment is improved a lot (the time is expected to be linear with regards to the number of payments), while the time for big batches is not impacted. closes odoo/odoo#152284 Related: odoo/enterprise#55633 Signed-off-by: Brice Bartoletti (bib) <bib@odoo.com>
Odoo
Odoo is a suite of web based open source business apps.
The main Odoo Apps include an Open Source CRM, Website Builder, eCommerce, Warehouse Management, Project Management, Billing & Accounting, Point of Sale, Human Resources, Marketing, Manufacturing, ...
Odoo Apps can be used as stand-alone applications, but they also integrate seamlessly so you get a full-featured Open Source ERP when you install several Apps.
Getting started with Odoo
For a standard installation please follow the Setup instructions from the documentation.
To learn the software, we recommend the Odoo eLearning, or Scale-up, the business game. Developers can start with the developer tutorials