Before this commit, calling read_group on a model and grouping by a
selection field would return the groups in alphabetical order, meaning
that if your selection field's options were declared as:
('z', 'Z'), ('a', 'A')
read_group would return first group 'a' and then group 'z', instead of
groups 'z' and then 'a' which some might expect.
With this commit, it is now possible to set a fields.Selection
group_expand attribute to `True` when declaring it, which will use a
default group_expand implementation specific to Selection fields, this
means that when grouping by a Selection field with `group_expand=True`
it will always return the groups in the definition order of the
selection options.
We achieve this by leveraging the group_expand field attribute which was
designed for changing the groups returned by read_group.
Since this attribute was thought only to be implemented on a
Model-by-Model basis and here we need to use it as a generic function
for all Selection fields (explicit group_expand declarations have higher
precedence), the generic method has been implemented inside
fields.Selection and takes an extra records parameter which holds a
reference to the recordset/model on which read_group was called. This
extra parameter only applies to field implementations of group_expand
and in this case is what allows this specific feature to work with
dynamic Selection fields (function as options).
A side-effect of this implementation is that a read_group call that
groups by a Selection field with `group_expand=True` that uses the
default group_expand will always return all possible groups, even
empty ones.
Task-ID 2635052
closesodoo/odoo#75856
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Grouping by many2many has been added with PR odoo/odoo#68958 (commit f4f570c3d6b403512d8523c74acd34d485f46af1)
However the display names are not resolved resulting in the grouping
displaying the actual id of the records.
This also breaks some features such as `read_groups` which expect the
group by data to contain a tuple (id, x) instead of just the id, this
commit adds name resolving on many2many group bys.
Task ID: 2398734
Part-of: odoo/odoo#74985
* Objectives:
- Allow the generation of empty date/datetime groups outside the bounds
defined by the existing data during a read_group, with the fill_temporal
context key.
{[Jan - empty]->[Feb]->[Mar - empty]->[Apr]->[May - empty]}
This will allow to get a guaranteed amount of groups regardless of the
existing data.
- Request contiguous date/datetime groups between custom bounds which do not
cover the entire domain of existence (existing data)
[Jan] {[Jul]->[Aug - empty]->[Sep]} [Dec]
If some records were "forgotten" before the desired "interesting" period, we
won't have to display "many" undesired empty groups to show them.
* Implementation:
Modify the fill_temporal context key to accept a dictionary format,
with 3 keys, to be used in _read_group_fill_temporal:
1. fill_from
2. fill_to
3. min_groups
- (1., 2.) offer a way to guarantee that at least a certain date range will
be returned as groups. They are also the bounds to fill. Any group outside
those bounds will not be removed, but there will be no empty group added. If
any of those keys is omitted, existing bounds are used, if applicable. If
fill_from and fill_to are not specified, and there is no data, no group is
returned.
edge case example:
existing dates: [Jan, Sep]
fill_from: [Apr]
fill_to: [Jul]
interval: 1 month
result: [Jan, |Apr, May, Jun, Jul|, Sep]
general usage examples:
a. We have a Forecast in a Kanban view. We are in April and we want the
forecast to cover the [Apr - Jul] period.
- domain: [(>= Apr), (<= Jul)] (limit the read_group to
the forecast)
- fill_from: Apr (current month)
- fill_to: Jul
The result would be [Apr, May, Jun, Jul]
b. We want the same Forecast [Apr - Jul], but there is a chance that some
records were forgotten in Jan, and we want to be able to reschedule
them, but we don't need the fill_temporal to take effect from Jan.
- domain: [(<= Jul)] (we removed the first bound)
- fill_from: Apr (current month)
- fill_to: Jul
The result would be [Jan, |Apr, May, Jun, Jul|]. We have the forecast
with the fill_temporal, and every month preceding the forecast as long
as there is at least one record in it (no empty month before the
forecast).
- (3.) Will be used as a way to guarantee a certain amount of returned groups
for a read_group grouped on a {date, datetime} field. This will only work
if there is at least one group to be used as reference (starting point) to
create supplementary groups if necessary.
example :
existing dates: [Mar, Apr]
min_groups: 4
interval: 1 month
result: [Mar, Apr, May, Jun]
- fill_temporal: {} will be considered as True (backend & frontend)
- The new key options are independant from each other (do not require the
others to be set in order to function properly)
Task-ID: 2243913
PR odoo/odoo#69380
1. Objective
- We are in a Kanban view, and we group by a "date/datetime" field.
- In order to "quick create" a record in a group, or to "drag & drop" a
record between 2 groups, we need a default value for this date field.
- In this case, a group represents a "range" of dates. A default value could
be the last date of this range.
- Prior to this commit, the view had only access to a display string for this
date range. We want to have access to the concrete bounds (start/end dates)
- Since some date/datetime fields have backend constraints, it would be
difficult to generate a compliant global default value. As such, the use of
the default value should be disabled by default and activated on a field by
field basis.
2. Usage
- Use the last day (for dates), or second (for datetimes) of the range to set
a default value in kanban "quick create" and "drag & drop"
- Use the end of the range of the last group from a read_group to be able to
request the next group (chronological order) in subsequent read_group calls
3. What are the changes in this commit
- read_group from the "core" "models.py", with the added __range for
date(time) fields, a dictionary for each group. The keys are the field
names and the values are a dictionary with the following keys: value:
- from: starting date(time) of the range (inclusive)
- to: ending date(time) of the range (exclusive)
- XML changes:
- with a new xml attribute on <field> tag allow_group_range_value (for the
kanban view):
- we allow (or disallow) supported non-readonly fields to be:
- draggable
- quick created
- this attribute can be used only on date(time) fields, and if not set,
the default behavior is false
- the naming is subject to contention because it relates to
different features. The relation comes from the constraint:
To perform a drag&drop or a quickCreate, we must be able to get the
value from the group containing the record.
- JS changes:
- we add "group.range" after a read_group for date fields, which is a
dictionary: {field_name: {from, to}}, for each date(time) groupby field
- since date(time) fields can use the format "date_field:granularity" when
used in a groupby, we have to properly split out the granularity to ensure
compatibility with the code previously in place (drag&drop procedures)
- update mock_server and sample_server to make use of the date range.
Adding support for datetime grouping to both mock and sample servers, for
tests and previews.
- the default value for kanban drag&drop and quickCreate features is the
last day/second of the range for date/datetime fields
4. Why can't this range be computed from the original display value with
moment.js
The Babel python library that we use for date(time) formatting (2.6.0) is kept
at the same version as the package in stable Debian Linux. This version has
some issues with displaying weeks consistently around the new year in certain
locales.
We cannot reverse compute a date(time) label with moment.js to get a range since
nothing guarantees that the range will be the same one that Babel used to
output it.
5. read_group return format discussion
Prior to this commit, the format of the value for the groupBy field was
either :
- String : used as a display value for the group.
Most notably, for date and datetime, this value cannot be used as a field
value in most cases. It would be more practical to also return the date
range along with the display value, so that Drag&Drop and QuickCreate
functionalities can be properly enabled in views
example :
"March 2021" has the range ['&', (field, '>=', 2021-03-01),
(field, '<', 2021-04-01)]
This information is lost at the end of the read_group.
- Array : used to indicate a res_id in case of a many2one field
res_id = array[0]
field_diplay_value = array[1]
Since we cannot use the array format to send the date(time)s of the period range
because it could be confused with the many2one format, this commit introduces
the use of a new `__range` key which is a dictionary with specific related
keys to be used to defer usable data to the web client.
Task-ID: 2243913
PR odoo/odoo#69380
With this commit, it is now possible to group the records of a model by
a Many2many field of said model.
The result of a read_group grouped by a m2m will return as many entries
or "groups" as there are different records in the comodel that are
linked to the model through the m2m, plus a null/false group, for records
of the model that have no linked records of the comodel or for records
of the comodel for which the current user has no read access to due to
ir.rules.
For a more illustrated explanation, the tests should cover all cases in
detail.
Note that this commit only introduces this change at the ORM level, the
frontend does not yet handle grouping by m2m fields but it is planned
in the near future.
Task-2428971
closesodoo/odoo#68958
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
When we group by date with DST change within a range, we could get a
reocrd inside two date range grouping, or inside no grouping.
This is because we computed range just with [+ 1 month], so we possibly
had these ranges (in UTC):
- October 2019 : [('datetime', '>=', '2019-10-01 02:00:00')
('datetime', '<', '2019-11-01 02:00:00')]
- November 2019 : [('datetime', '>=', '2019-11-01 01:00:00')
('datetime', '<', '2019-12-01 01:00:00')]
So a record on 2019-11-01 01:30:00 would be both inside October and
November.
This happen because the DST is removed on happen on 27 October 2019 and
this was not taken into account when computing the end of the range.
With this changeset, for the given example aboth, we will have:
- October 2019 : [('datetime', '>=', '2019-10-01 02:00:00')
('datetime', '<', '2019-11-01 01:00:00')]
Added test without the change fails with "AssertionError: Lists differ"
because:
- "Q1 2019" finished on 17:00:00 instead of 16:00:00
- "Q3 2019" finished on 16:00:00 instead of 17:00:00
opw-2278829
closes#54056closesodoo/odoo#54345
Note: maxDiff added for test to work in 13.0
X-original-commit: af5d03de28fa300ebbaa37a3d226b41051ebdf0f
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Replaces the auto_join scheme which is pretty fast but doesn't quite
work (it's *completely* broken for read_group) by a subquery which is
slower but has the same semantics as the ORM version.
Ideally we'd use `EXISTS (select 1 from ...)` but that has no support
inside the SQL compiler right now, so use `id in (select inverse_field
from ...)`.
Here are some measures of a `read_group` on a simple object with lines
filtered on line values, similar to
read_group([('order_line.product_uom', '=', xxx)],
fields=['total_amount'], groupby=['user_id'])
+-------+-------+-------+--------+
| | ORM | join |subquery|
+-------+-------+-------+--------+
| large | 160 | 10 | 20 |
+-------+-------+-------+--------+
| many | 36000 | 50 | 180 |
+-------+-------+-------+--------+
* values of data fields (filtering, grouping and aggregating) randomly
picked between 10 different values
* "large" is 1000 objects with 1000~15000 lines each
* "many" is 1000000 objects with 1~40 lines each
closesodoo/odoo#48494
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This commit add an extra parameter 'fill_temporal' to read_group which allows
the orm to add missing groups for date intervals. This is useful for charts.
Suppose that we are in a use case where data are grouped by a date fields
(typically months but it could be another interval) and displayed in a Bar Chart
or a Line Char.
Let's says a request has to group records by month for August, September
and ...December. If we don't changed anything, we would get a Bar Chart
looking like this :
___
___ | |
| | | |
| | ___ | |
| || || |
|___||___||___|
Aug Sep D
December follows directly after September, it can be unintuitive for the
user, so we change that. We add some fake records for each missing months
between the earliest and the lastest date of the result
___
___ | |
| | | |
| | ___ | |
| || | | |
|___||___| ___ ___ |___|
Aug Sep Oct Nov Dec
This commit is part of task #1835644
The caller of `read_group` can now provide the aggregating function to use for
a given field:
# set aggregating operator for fields 'foo' and 'bar'
model.read_group(domain, ['foo:sum', 'bar:avg'], ...)
One can also aggregate the same field several times, by giving a specific
output name for each:
# aggregate 'foo' with both 'min' and 'max'
model.read_group(domain, ['foomin:min(foo)', 'foomax:max(foo)'], ...)
The attribute `group_expand` allows to reorder and add empty groups to the
result of `read_group`. Enable this feature for other fields than many2one
fields.
Adrian Torres