Community PR: https://github.com/odoo/odoo/pull/91187
## Proposed feature and relevance
Add a new retrieval strategy for gathering products: "Least packages".
The goal of this strategy is to select specific product quantities from stock by using as few packages as possible, to allow for efficient logistics operations and avoid unnecessary opening/unpacking of packages.
This strategy will behave as follows:
- In case a negative qty is requested this strategy will behave as a FIFO strategy.
- Otherwise, the strategy will determine a combination using the least amount of packages that sum to the requested quantity, and adapt the provided domain to select only the quantities in these packages. This way no packages need to be opened to select the requested quantity.
- In case an exact matching of current packages to get the requested quantity is not possible, a selection of packages is made where the largest packages are selected one by one until the requested quantity is reached, or until no more quantities are available. For this approach, one package needs to be unpacked as it will not be used completely. The final selected package that will be unpacked will be the smallest one possible, to try to unpack smaller packages over bigger packages.
- In case the search for a minimal matching of packages results in an out-of-memory situation, the strategy will fall back to a FIFO strategy.
- Loose items (not tied to a package) are considered as single-item packages for the purposes of package selection.
- **NOTE:** To force the selection strategy to "clean up" certain packages that have previously been opened, it is recommended to unpack these packages into single items, as they will then be consumed to "fill up" requested quantities in later retrievals.
## Changes to allow more complex strategies
To allow for the integration of this new strategy the structure of `_gather()` needed to be adapted to allow a removal strategy to provide not only the SQL `ORDER BY` string but also adapt the domain used to retrieve the requested quantities.
This change is reflected in the method signature of `_get_removal_strategy_domain_order()`.
## How the strategy works
The strategy does one extra query of the database, to determine the different packages and their respective quantities (using the provided domain).
Loose items (that do not belong to a package) are added as single-item packages to also consider them.
For minimal exact package selection, an A* algorithm is used with 2 performance modifications:
- Using the information that the packages are sorted by size (large -> small).
- Using the information that the generated subtree for a specific package of size x is the same as all other packages of size x at a specific node.
By using the A* algorithm, as soon as a matching packaging selection is found, it is ensured to be optimal and other combinations do not need to be considered.
In case the A* algorithm runs out of memory the provided domain is just returned, which equates to a FIFO strategy.
In case the A* algorithm does not find an exact matching of packages for the requested quantity, a selection is made using the least amount of packages, unpacking the smallest possible package (or until no more quantities are available, in which case effectively all quantities in the domain are returned).
## Used heuristic
The heuristic used for the A* algorithm exploits the knowledge that our packages to select from are ordered by the highest to the lowest quantity (on the database side, SQL `ORDER BY`).
It assumes the "best case scenario" as an estimate for the amount of packages remaining, which is if we were to "match" the required quantity using fractional multiples of the next biggest package size.
***Example**:
Remaining quantity to select: 27
Remaining package sizes to select from: [10, 10, 5, 5, 5, 1, 1, 1]
Heuristic for choosing package with qty 10 as next package: 27/10 = 2.7
Heuristic for choosing package with qty 5 as next package: 27/5 = 5.4
Heuristic for choosing package with qty 1 as next package: 27/1 = 27*
This way the heuristic is always a "best case" (fractional) for the amount of packages that still need to be selected.
As required by the A* algorithm this also results in an underestimation as the fractional result is not possible, and it is not certain that there are still enough packages of this size that can actually be selected, but favours larger packages to make the search converge more quickly.
## Performance
### CPU
Two test cases were run to evaluate the performance.
***Total nr of pkg**: The total amount of different packages generated during data generation
**Qty to select**: The total quantity that needs to be selected
**Selected nr of pkg**: The amount of packages that was ultimately selected by the algorithm
**Data generation time**: The time required to generate all packages
**Assignment time**: The total time to run `_action_assign()` for the move
**A\* time**: The time to run the `_run_least_packages_removal_strategy_astar()` method*
#### First test case
This scenario simulates a case with a large quantity to select from a large amount of packages, but of relatively homogenous size.
Generate 30 different package sizes between 2 and 1000. For each package size, generate 100 to 250 packages each. Also add 100 to 250 "loose items".
The quantity to select is chosen between 1 and 50000.
| RESULTS | RUN 1 | RUN 2 | RUN 3 | RUN 4 | RUN 5 | AVG |
|--------------------------|----------|-----------|------------|-----------|----------|------------|
| Total nr of pkg | 5298 | 5772 | 5855 | 5533 | 5179 | 5527.4 |
| Qty to select | 10319 | 20762 | 44422 | 39306 | 1783 | 23318.4 |
| Selected nr of pkg | 11 | 22 | 48 | 41 | 3 | 25 |
| Data generation time (s) | 64.59124 | 92.916682 | 133.673384 | 82.711438 | 85.54542 | 91.8876328 |
| Assignment time (s) | 4.060216 | 11.212937 | 4.763776 | 8.60248 | 3.461941 | 6.42027 |
| A* time (s) | 0.28 | 6.88 | 0.21 | 3.96 | 0.03 | 2.272 |
#### Second test case
This scenario generates a large amount of small packages of different sizes.
Generate 250 different package sizes between 2 and 10. For each package size, generate 5 to 15 packages each. Also add 5 to 15 "loose items".
The quantity to select is chosen between 1 and 10000.
| RESULTS | RUN 1 | RUN 2 | RUN 3 | RUN 4 | RUN 5 | AVG |
|--------------------------|-----------|-----------|-----------|-----------|----------|------------|
| Total nr of pkg | 2485 | 2406 | 2491 | 2555 | 2542 | 2495.8 |
| Amount to assign | 27 | 4029 | 3966 | 3049 | 1833 | 2580.8 |
| Selected nr of pkg | 3 | 411 | 411 | 306 | 184 | 263 |
| Data generation time (s) | 18.502933 | 18.692048 | 19.969955 | 19.169106 | 19.16439 | 19.0996864 |
| Assignment time (s) | 1.35635 | 7.958852 | 8.470518 | 6.300406 | 3.895457 | 5.5963166 |
| A* time (s) | 0.01 | 0.07 | 0.77 | 0.56 | 0.31 | 0.344 |
### Memory
In cases with large amounts of different-sized packages (200+ different sizes and 200+ packages required to get an exact match) the memory performance of the A* strategy might become problematic.
When lots of packages have the same size (which would be considered typical in cases where there are large amounts of packages) the specific modifications to the A* algorithm ensure that the strategy still delivers good memory performance for this use case as the tree size can be significantly reduced, even with large quantities.
### IO
This strategy only requires one extra SQL query to fetch the different quantities and their respective packages.
## Limitations
No tests for this strategy were done for non-integer quantities. In these cases the performance of the A* algorithm might not be satisfactory and another approach might be preferred.
More research could be done for specific use cases where packaged quantities are non-integer values.
closesodoo/odoo#91187
Signed-off-by: William Henrotin (whe) <whe@odoo.com>
Some users were updating their access rights to allow for direct
deletion of quants. This could lead to the infamous:
"It is not possible to unreserve more products of %s than you have in stock."
error since directly deleting a quant bypasses the flows of correctly
unreserving the amounts you are deleting. Therefore we now restrict
the unlinking to:
- when in sudo mode, since the automatic unlinking of zero quants always
occurs in sudo mode.
- stock manager when unlink access right = True. Normally we would allow
any user with the unlink access to do it, but since we are using the
existing `_apply_inventory` to ensure that reserved qtys of the
quant are correctly unreserved before the unlinking, only stock
managers will correctly do this unreservation.
It is expected that some custom code may break due to this restriction,
but if custom code is directly unlinking quants without a sudo or with a
non-stock manager, then the code in these cases probably need to be
fixed anyways since this will cause inconsistent db data and lead to
the error above.
X-original-commit: d1602e7ddefd59cafa1ce0a5522a53c987135858
Part-of: odoo/odoo#115613
A lot of issue with "It is not possible to unreserve more products of ... than you have in stock".
It's the result of a desynchronisation between
`stock.move.line`.`reserved_qty` and `stock.quant`.`reserved_quantity` fields.
It should never happens in theory. However we already faced it a lot due
to bugs/custom code/server actions... It's not trivial to fix because
the issue come from data corruption. So it is hard to spot the different
main issues.
In order to avoid it, this commit try to modify the structure of stock
module. Before the operations was made in this order:
- The `stock.move` checks the quantity available on `stock.quant`
- The `stock.move` write the quantity reserved on the `stock.quant` and
save it in a variable
- The `stock.move` create a `stock.move.line` with the quanity reserved.
The main issue is that a `stock.move.line` could be easily created with
a reserved quantity while the `stock.quant` are not update nor checked.
After this commit the operations will be:
- The `stock.move` checks the quantity available
- The `stock.move` dispatch the available quantity among the `stock.move.line`
based on create or write calls.
- The `stock.move.line` reserve the quantity on `stock.quant`
- If the quantity is bigger than available, we write the max available on
`stock.move.line`
The idea is to respect the different layers
`stock.move` <-> `stock.move.line` <-> `stock.quant`.
Avoid the interactions bewteen `stock.move` and `stock.quant`
This behavior is already well managed in other use cases.
E.g. the real quantity itself, `_do_unreserve` of `stock.move`,...
opw - a lot
Part-of: odoo/odoo#115328
This commit allows the user to choose which quants to reserve from stock
and the quantity to reserve from each quant, making reservation much more
flexible from the picking form.
In addition, the `reserved_uom_qty` of stock.move.line
is now editable in views to allow for users to change the qty reserved
by existing move lines. When editing the reserved qty, if quantity is not available,
it will reserve only what is available..
closesodoo/odoo#106006
Taskid: 3090913
Signed-off-by: Arnold Moyaux (arm) <arm@odoo.com>
- Install stock
- Go to Inventory > Configuration > Settings and enable "Lots" and "Storage Locations"
- Create a Product tracked By Lots (i.e. Product X)
- Go to Inventory > Operations > Inventory Adjustments
- Create an Inventory Adjustment for Product X:
Product | Location | Lot/SN | Real Quantity
-------------------------------------------------------------
Product X | WH/Stock | LOT 01 | 20
Product X | WH/Stock | | 10
- Validate Inventory
- Go to Inventory > Operations > Transfers and create one:
* Source Location: WH/Stock
* Destination Location: WH/Stock/Shelf1
* Operation Type: Internal Transfers
* Operations:
[Product: Product X, Initial Demand: 25]
- Save Transfer, Mark As Todo and Check availability
- Click on list icon of Operation line for Product X to display Detailed Operations
- 20 units of LOT 01 and 5 units without lot have been reserved
- Set LOT 01 for the 5 reserved units without lot and confirm
- Open Detailed Operations again
- There are now 20 units of LOT 01 and 5 units of LOT 01
- Remove the row with 5 units and confirm
- Check availability and open Detailed Operation
- There is now only a row with 25 reserved units of LOT 01
- Unreserve
The following errror is raised:
"It is not possible to unreserve more products of P than you have in stock."
It happens because the system is not able to manage quants with lots and
wihtout lots at the same time. When modifying the move line to 25
reserved units. It's composed of 20 quants with lot and 5 quants without
lot. And when unreserving it will check if there is a quants with 25
units with the lot and if it's not found 25 units without lot. But never
25 units of quants with lots and without lots.
opw-2419444
Close#64497closesodoo/odoo#66029closesodoo/odoo#93141
X-original-commit: 83d55d8ed8b8b5a5232e3cdfc09abf5285f18ed7
Signed-off-by: Arnold Moyaux <amoyaux@users.noreply.github.com>
Signed-off-by: Arnold Moyaux (arm) <arm@odoo.com>
When unpacking a package, the quants are not merged
To reproduce the issue:
1. In Settings, enable "Delivery Packages"
2. Create a storable product P
3. Update its quantity: 5
4. Create a planned receipt R with 10 x P
5. Mark R as Todo
6. Put the 10 x P in pack (PK)
7. Validate R
8. Unpack PK
9. Consult the on-hand quantity of P
The use case is already fixed thanks to:
5e08aa3def
This commit only adds the associated test
OPW-2713407
closesodoo/odoo#85575
X-original-commit: 76400ba0f6c2c15d9478b762a0a393129cd20f5e
Signed-off-by: Arnold Moyaux <arm@odoo.com>
Signed-off-by: Adrien Widart <awt@odoo.com>
When a quant is creating, for instance thanks to an internal transfer,
some of these computed fields will have an incorrect value
To reproduce the issue:
1. In Settings, enable "Storage Locations"
2. Create a storable product P
3. Update P's quantity: 100 P in WH/Stock
4. Process an internal transfer
- From: WH/Stock
- To: WH/Stock/Shelf 1
- Operations: 10 x P
5. Consult the on hand quantities of P
Error: There is a new line with 10 x P in WH/Stock/Shelf 1, which is
correct. However, the Counted Quantity field is defined and equal to 0,
and so does the Difference field with a value of -10. These two fields
should not be shown.
When validating the internal transfer, a new quant is created for the
location "Shelf 1". However, there isn't any value for the field
`inventory_quantity_set`. Therefore, at the end of the transfer
validation process, when the `flush` method is called, since
`inventory_quantity_set` is a computed field, its `compute` method is
called and define the value to `True`:
https://github.com/odoo/odoo/blob/8b155b695823dfa954464ad0d5dd07445ece2471/addons/stock/models/stock_quant.py#L134-L136
This explains why, on front-end, the Counted Quantity and Difference
fields are displayed.
OPW-2733937
closesodoo/odoo#84679
X-original-commit: c519e26adec85025f40f78c59a05b2385377fb5d
Signed-off-by: Arnold Moyaux <arm@odoo.com>
Signed-off-by: Adrien Widart <awt@odoo.com>
In some cases, when making an inventory adjustment, the `in_date` of the
new quant will be incorrect
To reproduce the issue:
(Let D01 be the current date)
1. Create a storable product P
2. Set its quantity to 1
3. Process a delivery order with 1 x P
4. Set the date in the future
- Let D02 be this date
5. Make an inventory adjustment with 1 x P
Error: The `in_date` of the quant (for P in the stock location) is D01
instead of D02 (can be observed either directly in PSQL, on the form
view of the quant (via Locations > Current Stock), or by adding the
field on the tree view)
When validating the stock adjustment, at some point, the module calls
`_action_done` on a SML (1 x P from Inventory Adjustment to the Stock
Location). To do so, it decreases the quantity of the origin location
and increases the quantity of the destination location thanks to
`_update_available_quantity`:
https://github.com/odoo/odoo/blob/b4a9e5b8307ab1b730effe2de23f15260326ef6c/addons/stock/models/stock_move_line.py#L485-L493
But here is the issue: when decreasing the quantity in the virtual
location (Inventory Adjustment), it finds an old quant (the one from
step 2 in above use case). It then stores its `in_date` (D01) and since
this date is before the current one (D02), D01 is kept, used to update
the quant quantity and returned in `action_done`. As a result, when
increasing the quantity in the stock location,
`_update_available_quantity` is called with the parameter `in_date`
defined and equal to D01. Again, D01 will be the earliest date, so the
date will be used to create/update the quant in stock location.
OPW-2702198
closesodoo/odoo#83009
X-original-commit: e27ee5cd6b19351a00030310d1db3a0defe5b31a
Signed-off-by: Arnold Moyaux <arm@odoo.com>
Signed-off-by: Adrien Widart <awt@odoo.com>
As the inventory adjustments have been removed in favour of the editable
quants, this commit introduced the possibility to import pre counted
quants like non validated inventories back then.
task : 2555118
Part-of: odoo/odoo#74715
Assuming that the only thing cared about when removing items from stock
is the accessibility of the product. Then it would be better to take
products stored on the floor instead of on higher levels of my racks.
In this case current solutions in Odoo are not the most efficient.
Closest location picks items in the 'smallest' location (in alphabetical
order), so it assumes the locations are ordrered alphabetically
according to how close they are to the ground.
Assume we have the following stock :
- WH/Stock/Shelf 2
- WH/Stock
- WH/Stock/Shelf 1
Quants would be looked at in the following order :
1. WH/Stock (due to shortest name for the same stock)
2. WH/Stock/Shelf 1 (due to alphabetical order)
3. WH/Stock/Shelf 2
Task-2568735
closesodoo/odoo#75303
Signed-off-by: Arnold Moyaux <amoyaux@users.noreply.github.com>
PURPOSE
Lessen use of mail-specific calls and variables
SPECIFICATIONS
Use mail_new_test_user tool in tests, lessening use of mail-specific context
keys in tests.
LINKS
Task ID-2326281 (context keys use cleaning)
PR odoo/odoo#56631
PR odoo/enterprise#12707
X-original-commit: 910559c092dc7dfa00b91339a5423e16cd4668e1
Ensure proper domains are applied and enforced on relation fields thanks
to the `check_company` attributes.
product.template
- make responsible a property field in order to ensure proper next activities when a product
is used between multiple companies
stock.putaway.rule
- added a company_id field
stock.move.line
- company_id is not related anymore since a move line can exist without a move until its validation
stock.package_level
- added a company_id field
stock.picking.type
- company_id is now required
stock.production.lot
- added a company_id field, adapted the constraint accordingly
stock.quant
- check the consistency only in inventory mode
stock.quant.package
- company_id is now empty if the package is empty
stock.picking
- company_id is now related to the one of its picking type
Added some tests.
Moved stock_traceability in the `report` directory.
Removed useless /tests/tours/route.js.
task-1985992
This branch is the combination of several optimizations in the ORM:
* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;
* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;
* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);
* make method `modified` take advantage of inverse fields to inverse
dependencies;
* filter records by evaluating a domain on records in Python;
* a computed field with `readonly=False` behaves like a normal field
with an onchange method;
* computed fields are computed in superuser mode by default.
Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.
closesodoo/odoo#35659
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
pass all test cases without any demo data.
If a user has not installed any demo data, then also test cases must not be break.
task-2024202
closes : https://github.com/odoo/odoo/pull/34258
before on runbot:
odoo.addons.stock.tests.test_move tested in 31.79s, 63166 queries
odoo.addons.stock.tests.test_move2 tested in 33.69s, 68046 queries
odoo.addons.stock.tests.test_quant tested in 8.48s, 15351 queries
odoo.addons.stock.tests.test_stock_flow tested in 16.99s, 31506 queries
amounting to 90 seconds on a build page
after om runbot:
odoo.addons.stock.tests.test_move tested in 12.41s, 24197 queries
odoo.addons.stock.tests.test_move2 tested in 13.55s, 24254 queries
odoo.addons.stock.tests.test_stock_flow tested in 9.74s, 15795 queries
amounting to 37 seconds on a build page
Use savepoint case instead of transaction case.
We took the opportunity to rename the master data more appropriately,
which results now in a bigger diff in TestMove and TestQuant. In
TestQuant, the master data creation had to be moved in the setUpClass
too.
We removed test_shipment as test_backorder_1-4 in tet_move_2 covers the
same parts of the code.
`_action_done` could unlink a quant. This happens when moving a product from a
location to another one and makes null the quantity of this product in one of
theses locations. In this case, we unlink the quant because it is now useless
and could be confusing on reports based on quants.
The unwanted side effect is that unlinking a record will invalidate the cache. In
some pathologic cases, like making an inventory adjustment of 600 products
and reseting their quantity back to 0, the time of the operation is around 15
minutes. With this patch that tries to work carefully with the cache invalidation,
the same operations takes around 15 secondes.
This commit do not unlink quant anymore, instead it use a 'garbage
collector' on quants without quantity. The garbage collector is trigger
on the scheduler and when the user open the inventory view.
From this commit onwards, Date fields will return datetime.date objects and Datetime fields will return datetime.datetime objects, this implies a number of things that are clearly explained both in the ORM API for master.
This commit also introduces a number of helper functions for dates and datetimes that are exposed in tools.date_utils and fields.Date[time], explained in the documentation as well.
Task-ID: 47189
PURPOSE
=======
1. Unified product demo data
2. Less demo: one per use case
3. Demo data for all models
Specification
=============
1. Refactor all the brol in the demo data
2. Adapt the tests to make them green, as some products are
renamed, removed, created in python instead of as demo data
...
In a warehouse, mostly most of the products have no lot. It stays
important however to avoid having old products in stock, certainly if
we can distinguish them by pack or location.
As now in stable, in_date is not put on the quant for non-tracked products,
databases might mix quants with and without in_date of the same product.
We should see those without in_date as the oldest.
That is why we need to be able to apply a ASC NULLS FIRST (ASC is NULLS LAST by default) for FIFO
or DESC NULLS LAST for LIFO.
In order to do that, we copied the code from the _search method of the ORM to
be able to do this special order as the orm itself is restrictive as it
supposes the order itself has no spaces.
We adapted and added some tests to check it handles the combination of NULL/NOT NULL well.
Suppose you have a product tracked by serial number. In your inventory,
you have two quants: -1 for serial1, +1 for serial2.
If you create a move for 1 product and try to run `action_assign`, it
didn't work before this patch because stock.move's
`_update_reserved_quantity` is guarded by a call to
an unstricted `_get_available_quantity` (unstricted meaning considering
all quants regardless of their lot_id/package_id/owner_id). This call
resulted in an available quantity of 0, thus the stock move could not be
reserved
To fix this issue, we patched `_get_available_quantity` so that it
rightly considers products of different lot_id as different products,
i.e. it returns an available quantity of +1 in the initial scenario. Of
course, we consider all quants of the same lot_id, i.e. if you have -1
for serial1 and +1 for serial2, the available quantity amounts to 0.
In another scenario, let's say we have only -1 serial1 in stock, if we
run `_get_available_quantity`, the method should return 0: nothing is
reservable. We continue this reasonment by returning 0 everytime the
available quantity is negative, tracked or not. This is probably what a
method named `_get_available_quantity` should have done since the
beginning.
There's only a single case where we need to know the quantity even if
it's negative: it's just after we move a quant to its destination
location. If this quant was tracked and resulted in a negative quant
(i.e. we moved something we didn't have), we try to compensate with an
untracked quant. In this special case, we do not want to ignore the
negative quants when calling `_get_available_quantity`. That's why we
introduce a new argument to this method. We could have added a new method
`_get_quantity` for clarity sake.
Set the `in_date` field to now when creating a tracked quant.
Set the correct `in_date` field when moving a quant (this is done by
adding a kw argument to `_update_available_quantity` and adapting the
code that moves quant in stock.move.line).
If quants from the same lot go together in the same location, we merge
them and keep the older date (as the user will have no way to tell the
difference anyway).
We add multiple test cases:
- `in_date` are only set on tracked quants
- FIFO and LIFO are correctly applied
- keep the oldest `in_date` when merging quants
- editing a done move line or creating one in a done picking is
correctly working
Joint work with sle-odoo.
When we decrease the reserved quantity, the argument 'strict' must be
set to 'True', else we might unreserve quants of another move line.
This behavior has been lost in rev: https://github.com/odoo/odoo/commit/9bb696e13abcbf95c9552fab60b1dfb997afaa59
We introduce a test to show where an issue happened without this
patch: have tracked and untracked quants for the same product in stock,
reserve them and edit the untracked move line in order to set a lot to
it. The first step will be to unreserve the move line without lot, and
if we do not pass a Strict kw, stock.quant's `_gather` will return
quants with lot and unreserve on them. Everything is now inconsistent
and lead to issues upon validation.
When updating the quantity on hand in a location without quant but
where one of its sublocations has onee, it will increase the available
quantity in the sub location instead of the defined location.
This happens because _gather method in quant search on child_of location
in any case.
This commit change the domain by using '=' on the locations if `strict` is
passed as True in _gather, or 'child_of' if it is set as False. It also
removes the strict argument on _update_avaible_quantity since it does
not make sense to change the quantity on an undefined quant. And it was
never used in our code for good reasons.
To increase or decrease the reserved or available quantity,
`stock.quant` exposes two methods: _increase_xxx_quantity and
_decrease_xxx_quantity. This could be confusing and could lead to
unexpected result when overriding one or the other.
We remove the _decrease_xxx_quantity method and rename
_increase_xxx_quantity by _update_xxx_quantity, as it supports a signed
float anyway.
Pieter Claeys (clpi)