Issue:
======
When you have duplicate groupbys with `lazy=False` you will get an error.
Steps to reproduce the error:
=============================
- Install timesheet
- Go to timesheet / reporting / By Employee
- Add a groupby by month for one employee
- I will show an error.
Origin of the issue:
====================
The timesheet component will the send a request for read_group having
`['date:month', 'date:month']` so we will have duplicated groupby which
will be processed later in `_read_group_format_result` which update the
value of `row[group]` for each group so the first group will have the
original values which is ok but the second one will have the updated
values by the first iteration which will result in error.
Solution:
=========
We need to check that the value is of class BaseModel to update it , it
means that's the first time encountered.
opw-3497803
closesodoo/odoo#137000
X-original-commit: 91d6dc7ed56f67d41adab90ec0019fbb920d9042
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
How to reproduce:
- open CRM > Forecast
- group by Expected Closing > Week
Current behavior:
- some week numbers appear multiple times
Expected behavior:
- each week should only appear once
Technical explanation:
Since [1], week groups are dependent on the locale, but the
`_read_group_fill_temporal` method was not updated to also produce groups
dependent on the locale.
[1]: https://github.com/odoo/odoo/pull/93053
task-3478451
closesodoo/odoo#135952
X-original-commit: 980786e301bdf80a9a39260a95359431c5cb5e86
Signed-off-by: Damien Abeloos (abd) <abd@odoo.com>
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
Add 'id' in partner ordering, to be sure searching on duplicated partners is
deterministic.
In mail, use standard ordering when searching on users to be deterministic.
As ordering on users is based on name then login which is unique, this is a
safe ordering and can be used as it.
Task-2612945 (Mail: Defensive email formatting)
X-original-commit: odoo/odoo@1e6d50a141
Part-of: odoo/odoo#134934
Rationale
=========
Since v8, the `display_name` field is present on all models. By default,
`display_name` uses `name_get` which has pretty much the same purpose
(return record name used by the web client). Gradually, many (backend)
developers (and the ORM: https://github.com/odoo/odoo/commit/6da1c3ac4c036eac289597602976538e243cb939)
started using `display_name` (more convenient than
`record.name_get()[0][1]`) but it still had the `name_get` override.
It becomes more complex than necessary and poeple start to misunderstand
the two (and sometimes override both, leading to inconstiencies between
`display_name`/`name_get`).
To simplify the ORM and the API, we decided to keep only one of them,
the `display_name` field:
- It is much more convenient from a backend point of view
(`record.name_get()[0][1]` vs `record.display_name`)
- It is cached during the same transaction (and invalidated if
its dependencies change)
- It can be overridden like any other compute field (override
`_compute_display_name` with any extra dependencies)
- `name_get` is replaced by `read(['display_name'])`
(API perceptive), which can actually be more efficient
(if `display_name`'s depends are correct, the ORM will only fetch the
fields it needs instead of every prefetchable field)
Changes
=======
- Deprecates `name_get` for the v17 and based the method on
`display_name` (the opposite of before)
- Converts all usage of `name_get`
- Overrides of `name_get` are now overrides of `_compute_display_name`
- For `res.partner`, rename the field store `display_name` into
`complete_name` because `display_name` context-dependent and it makes
no sense to have a compute store that is context-dependent.
- Previously, it was possible to return multiple names for the same
record with `name_get`, but it was tricky and most of the usage of
this `name_get` didn't take this into account. The only example of
this is the `name_get` of `product.product`
(now use `", ".join(<names>)`).
Part-of: odoo/odoo#122085
THese are rarely intended for all users but often intended only for
employees.
account:
account.incoterms: only used within internal business models
account.journal.group: same as account.journal, add sudo in computed field
account_edi: need access to accounting objects
base_address_extended:
res.city: only employees should access address data
board: only employees uses this (old) module
crm:
crm.stage: internal users business object
hr_recruitment: employees can read
im_livechat: apply same as for the steps
l10n_ar: used on partner, not only invoices
l10n_ec: accessed only through account.move
l10n_latam: accessed on res.partner
mail:
publisher.warrenty.contract: no data, only static models
mail.channel: group_user has already his own rule
mail.group: group_user has already his own rule
mail.message.subtype: group_user has already his own rule
mail.message.all: remove, already has a portal and employee rule
partner_autocomplete: no interaction with public
project:
project.tags: only needed for project sharing
sale_management:
sale.order.option: same as sale.order
utm: employee already has write access
web_editor: test models that have nothing to do here
web_tour: only employees uses tours
website_sale:
product.ribbon: add sudo for access
base:
ir.default: only employees uses set (could probably be converted to group_system)
ir.ui.view.custom: same as ir.ui.view, add sudo when needed
report.*: portal users don't configure reports
res.users.log: create in sudo, no access needed (adapt test to use another model)
res.lang: still needed for public
closesodoo/odoo#118701
Related: odoo/enterprise#41285
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Make the public `read_group` depends of its private method `_read_group`
refactored to match the backend usage.
We try to keep the public API similar for this first part of the
rafactor, but there are still some API change:
- We cannot order by `id` anymore.
- The display_name of many2x group values are not lazy anymore.
Part-of: odoo/odoo#110737
The `_read_group` was designed to be used by the web client to
efficiently compute aggregations grouped by one or more fields.
However, more and more developers have been using it from the backend
to make computations more efficient (avoid doing the aggregation
in Python). Unfortunately, the API was designed for the web client,
which added a lot of boilerplate when used in the Python (list of
dict with misleading key name choices).
`_read_group` was created to improve the performance of read_group
for backend use (4ef0c00b4b), but didn't
change the API and based the implementation on read_group itself.
Rewrite `_read_group` from scratch with a new API to make it easier
to use from the backend (see the method documentation). Also, split
the method to make it easy to override and add custom behavior.
Part-of: odoo/odoo#110737
Steps to reproduce:
- Make sure language preference is 'en_US'
- In accounting, in the dashboard click on bills
- Filter 'due_date' by week
Issue: The start day is Monday and should be, for 'en_US', Sunday as it
is the case in the dashboard view in accounting (see appendix).
Cause: The query uses the `date_trunc('week', date)` which in Postgres
retrieves the first day of the week as Monday (ISO week).
Solution: Create an offset in the query depending on the first day of
the locale variable.
Note: the `web/tests/test_read_progress_bar.py` has been modified: since
the default language is 'en_US' there will be an offset of one day. To
make it less confusing, I used only two anglo-saxons countries so the
day offset is not the variable tested. (for this matter, pleaser refer
to `test_read_group/tests/test_read_group_process_groupby.py`)
Appendix:
Language (english-US)
VIEW (per week) | DASHBOARD
___________________________________________________________________
W23 -> 06/05 | 05/29 -> 06/04
W24 06/06 -> 06/12 | 06/05 -> 06/11
W25 06/13 -> | 06/12 -> 06/18
(Monday - Sunday) (Sunday - Saturday)
Language (french-BE)
VIEW (per week) | DASHBOARD
___________________________________________________________________
W22 -> 06/05 | 05/30 -> 06/05
W23 06/06 -> 06/12 | 06/06 -> 06/12
W24 06/13 -> | 06/13 -> 06/19
(Monday - Sunday) (Monday - Sunday)
opw-2747066
closesodoo/odoo#93053
Related: odoo/enterprise#29539
Signed-off-by: Raphael Collet <rco@odoo.com>
There was an issue with the computed `read_group` `__range` when grouping on
the same date/datetime field on multiple granularities (i.e. month, week).
Since the range was stored with the field_name as a key, the last evaluated
range would override the previous ones.
Impacted Versions:
- master
(- exists since 15.0 but it does not impact the user directly so it has been
decided to fix this only in master, since the API is modified)
Steps to reproduce:
1. Open a list view and group by a date field with at least 2 granularities
2. Open the chrome debugger (network) and check a web_read_group rpc preview
3. Find the web_read_group for groups related to one of the largest
granularities and check the `__range`
Current behavior:
- `__range = {field_name: false}`
Expected behavior:
- `__range = {field_name: {from: range_start, to: range_end}`
Explanation
Since the smaller granularities are evaluated last, and the condition to update
`__range` is related to the field_name and not the granularity, the range is
always overriden by the smaller granularities (even if their value is False)
when grouping on the same field with multiple granularities.
Furthermore, there is a conceptual problem with the current solution: it does
not allow to store multiple ranges when the read_group is not lazy and when
grouping on the same field with multiple granularities.
Therefore, the proposed solution is to use the full groupby keys in the
`__range` to allow storing multiple ranges depending on granularity. The keys
in `__range` would thus match the group value keys and allow more flexibility
if a domain must be forged from the group(s) range(s).
Task-2894519
closesodoo/odoo#95193
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Before this commit, calling read_group on a model and grouping by a
selection field would return the groups in alphabetical order, meaning
that if your selection field's options were declared as:
('z', 'Z'), ('a', 'A')
read_group would return first group 'a' and then group 'z', instead of
groups 'z' and then 'a' which some might expect.
With this commit, it is now possible to set a fields.Selection
group_expand attribute to `True` when declaring it, which will use a
default group_expand implementation specific to Selection fields, this
means that when grouping by a Selection field with `group_expand=True`
it will always return the groups in the definition order of the
selection options.
We achieve this by leveraging the group_expand field attribute which was
designed for changing the groups returned by read_group.
Since this attribute was thought only to be implemented on a
Model-by-Model basis and here we need to use it as a generic function
for all Selection fields (explicit group_expand declarations have higher
precedence), the generic method has been implemented inside
fields.Selection and takes an extra records parameter which holds a
reference to the recordset/model on which read_group was called. This
extra parameter only applies to field implementations of group_expand
and in this case is what allows this specific feature to work with
dynamic Selection fields (function as options).
A side-effect of this implementation is that a read_group call that
groups by a Selection field with `group_expand=True` that uses the
default group_expand will always return all possible groups, even
empty ones.
Task-ID 2635052
closesodoo/odoo#75856
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Grouping by many2many has been added with PR odoo/odoo#68958 (commit f4f570c3d6b403512d8523c74acd34d485f46af1)
However the display names are not resolved resulting in the grouping
displaying the actual id of the records.
This also breaks some features such as `read_groups` which expect the
group by data to contain a tuple (id, x) instead of just the id, this
commit adds name resolving on many2many group bys.
Task ID: 2398734
Part-of: odoo/odoo#74985
The license is missing in most enterprise manifest so
the decision was taken to make it explicit in all cases.
When not defined, a warning will be triggered starting from
14.0 when falling back on the default LGPL-3.
closesodoo/odoo#74245
Related: odoo/design-themes#48
Related: odoo/enterprise#19862
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
* Objectives:
- Allow the generation of empty date/datetime groups outside the bounds
defined by the existing data during a read_group, with the fill_temporal
context key.
{[Jan - empty]->[Feb]->[Mar - empty]->[Apr]->[May - empty]}
This will allow to get a guaranteed amount of groups regardless of the
existing data.
- Request contiguous date/datetime groups between custom bounds which do not
cover the entire domain of existence (existing data)
[Jan] {[Jul]->[Aug - empty]->[Sep]} [Dec]
If some records were "forgotten" before the desired "interesting" period, we
won't have to display "many" undesired empty groups to show them.
* Implementation:
Modify the fill_temporal context key to accept a dictionary format,
with 3 keys, to be used in _read_group_fill_temporal:
1. fill_from
2. fill_to
3. min_groups
- (1., 2.) offer a way to guarantee that at least a certain date range will
be returned as groups. They are also the bounds to fill. Any group outside
those bounds will not be removed, but there will be no empty group added. If
any of those keys is omitted, existing bounds are used, if applicable. If
fill_from and fill_to are not specified, and there is no data, no group is
returned.
edge case example:
existing dates: [Jan, Sep]
fill_from: [Apr]
fill_to: [Jul]
interval: 1 month
result: [Jan, |Apr, May, Jun, Jul|, Sep]
general usage examples:
a. We have a Forecast in a Kanban view. We are in April and we want the
forecast to cover the [Apr - Jul] period.
- domain: [(>= Apr), (<= Jul)] (limit the read_group to
the forecast)
- fill_from: Apr (current month)
- fill_to: Jul
The result would be [Apr, May, Jun, Jul]
b. We want the same Forecast [Apr - Jul], but there is a chance that some
records were forgotten in Jan, and we want to be able to reschedule
them, but we don't need the fill_temporal to take effect from Jan.
- domain: [(<= Jul)] (we removed the first bound)
- fill_from: Apr (current month)
- fill_to: Jul
The result would be [Jan, |Apr, May, Jun, Jul|]. We have the forecast
with the fill_temporal, and every month preceding the forecast as long
as there is at least one record in it (no empty month before the
forecast).
- (3.) Will be used as a way to guarantee a certain amount of returned groups
for a read_group grouped on a {date, datetime} field. This will only work
if there is at least one group to be used as reference (starting point) to
create supplementary groups if necessary.
example :
existing dates: [Mar, Apr]
min_groups: 4
interval: 1 month
result: [Mar, Apr, May, Jun]
- fill_temporal: {} will be considered as True (backend & frontend)
- The new key options are independant from each other (do not require the
others to be set in order to function properly)
Task-ID: 2243913
PR odoo/odoo#69380
1. Objective
- We are in a Kanban view, and we group by a "date/datetime" field.
- In order to "quick create" a record in a group, or to "drag & drop" a
record between 2 groups, we need a default value for this date field.
- In this case, a group represents a "range" of dates. A default value could
be the last date of this range.
- Prior to this commit, the view had only access to a display string for this
date range. We want to have access to the concrete bounds (start/end dates)
- Since some date/datetime fields have backend constraints, it would be
difficult to generate a compliant global default value. As such, the use of
the default value should be disabled by default and activated on a field by
field basis.
2. Usage
- Use the last day (for dates), or second (for datetimes) of the range to set
a default value in kanban "quick create" and "drag & drop"
- Use the end of the range of the last group from a read_group to be able to
request the next group (chronological order) in subsequent read_group calls
3. What are the changes in this commit
- read_group from the "core" "models.py", with the added __range for
date(time) fields, a dictionary for each group. The keys are the field
names and the values are a dictionary with the following keys: value:
- from: starting date(time) of the range (inclusive)
- to: ending date(time) of the range (exclusive)
- XML changes:
- with a new xml attribute on <field> tag allow_group_range_value (for the
kanban view):
- we allow (or disallow) supported non-readonly fields to be:
- draggable
- quick created
- this attribute can be used only on date(time) fields, and if not set,
the default behavior is false
- the naming is subject to contention because it relates to
different features. The relation comes from the constraint:
To perform a drag&drop or a quickCreate, we must be able to get the
value from the group containing the record.
- JS changes:
- we add "group.range" after a read_group for date fields, which is a
dictionary: {field_name: {from, to}}, for each date(time) groupby field
- since date(time) fields can use the format "date_field:granularity" when
used in a groupby, we have to properly split out the granularity to ensure
compatibility with the code previously in place (drag&drop procedures)
- update mock_server and sample_server to make use of the date range.
Adding support for datetime grouping to both mock and sample servers, for
tests and previews.
- the default value for kanban drag&drop and quickCreate features is the
last day/second of the range for date/datetime fields
4. Why can't this range be computed from the original display value with
moment.js
The Babel python library that we use for date(time) formatting (2.6.0) is kept
at the same version as the package in stable Debian Linux. This version has
some issues with displaying weeks consistently around the new year in certain
locales.
We cannot reverse compute a date(time) label with moment.js to get a range since
nothing guarantees that the range will be the same one that Babel used to
output it.
5. read_group return format discussion
Prior to this commit, the format of the value for the groupBy field was
either :
- String : used as a display value for the group.
Most notably, for date and datetime, this value cannot be used as a field
value in most cases. It would be more practical to also return the date
range along with the display value, so that Drag&Drop and QuickCreate
functionalities can be properly enabled in views
example :
"March 2021" has the range ['&', (field, '>=', 2021-03-01),
(field, '<', 2021-04-01)]
This information is lost at the end of the read_group.
- Array : used to indicate a res_id in case of a many2one field
res_id = array[0]
field_diplay_value = array[1]
Since we cannot use the array format to send the date(time)s of the period range
because it could be confused with the many2one format, this commit introduces
the use of a new `__range` key which is a dictionary with specific related
keys to be used to defer usable data to the web client.
Task-ID: 2243913
PR odoo/odoo#69380
With this commit, it is now possible to group the records of a model by
a Many2many field of said model.
The result of a read_group grouped by a m2m will return as many entries
or "groups" as there are different records in the comodel that are
linked to the model through the m2m, plus a null/false group, for records
of the model that have no linked records of the comodel or for records
of the comodel for which the current user has no read access to due to
ir.rules.
For a more illustrated explanation, the tests should cover all cases in
detail.
Note that this commit only introduces this change at the ORM level, the
frontend does not yet handle grouping by m2m fields but it is planned
in the near future.
Task-2428971
closesodoo/odoo#68958
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
When we group by date with DST change within a range, we could get a
reocrd inside two date range grouping, or inside no grouping.
This is because we computed range just with [+ 1 month], so we possibly
had these ranges (in UTC):
- October 2019 : [('datetime', '>=', '2019-10-01 02:00:00')
('datetime', '<', '2019-11-01 02:00:00')]
- November 2019 : [('datetime', '>=', '2019-11-01 01:00:00')
('datetime', '<', '2019-12-01 01:00:00')]
So a record on 2019-11-01 01:30:00 would be both inside October and
November.
This happen because the DST is removed on happen on 27 October 2019 and
this was not taken into account when computing the end of the range.
With this changeset, for the given example aboth, we will have:
- October 2019 : [('datetime', '>=', '2019-10-01 02:00:00')
('datetime', '<', '2019-11-01 01:00:00')]
Added test without the change fails with "AssertionError: Lists differ"
because:
- "Q1 2019" finished on 17:00:00 instead of 16:00:00
- "Q3 2019" finished on 16:00:00 instead of 17:00:00
opw-2278829
closes#54056closesodoo/odoo#54345
Note: maxDiff added for test to work in 13.0
X-original-commit: af5d03de28fa300ebbaa37a3d226b41051ebdf0f
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Replaces the auto_join scheme which is pretty fast but doesn't quite
work (it's *completely* broken for read_group) by a subquery which is
slower but has the same semantics as the ORM version.
Ideally we'd use `EXISTS (select 1 from ...)` but that has no support
inside the SQL compiler right now, so use `id in (select inverse_field
from ...)`.
Here are some measures of a `read_group` on a simple object with lines
filtered on line values, similar to
read_group([('order_line.product_uom', '=', xxx)],
fields=['total_amount'], groupby=['user_id'])
+-------+-------+-------+--------+
| | ORM | join |subquery|
+-------+-------+-------+--------+
| large | 160 | 10 | 20 |
+-------+-------+-------+--------+
| many | 36000 | 50 | 180 |
+-------+-------+-------+--------+
* values of data fields (filtering, grouping and aggregating) randomly
picked between 10 different values
* "large" is 1000 objects with 1000~15000 lines each
* "many" is 1000000 objects with 1~40 lines each
closesodoo/odoo#48494
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Purpose
=======
The current kanban view is messy. It is difficult to identify which
apps are installed or not. The user can completely miss a module
that might have interested him. A search panel would make things way
more readable.
closesodoo/odoo#44401
Taskid: 2181557
Related: odoo/enterprise#8144
Related: odoo/upgrade#879
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
Purpose of this commit is to give description more "business oriented"
because those descriptions appears in Odoo Studio which is supposed to be used by end users, not only by developers.
Related Task ID : 37311
This commit add an extra parameter 'fill_temporal' to read_group which allows
the orm to add missing groups for date intervals. This is useful for charts.
Suppose that we are in a use case where data are grouped by a date fields
(typically months but it could be another interval) and displayed in a Bar Chart
or a Line Char.
Let's says a request has to group records by month for August, September
and ...December. If we don't changed anything, we would get a Bar Chart
looking like this :
___
___ | |
| | | |
| | ___ | |
| || || |
|___||___||___|
Aug Sep D
December follows directly after September, it can be unintuitive for the
user, so we change that. We add some fake records for each missing months
between the earliest and the lastest date of the result
___
___ | |
| | | |
| | ___ | |
| || | | |
|___||___| ___ ___ |___|
Aug Sep Oct Nov Dec
This commit is part of task #1835644
The caller of `read_group` can now provide the aggregating function to use for
a given field:
# set aggregating operator for fields 'foo' and 'bar'
model.read_group(domain, ['foo:sum', 'bar:avg'], ...)
One can also aggregate the same field several times, by giving a specific
output name for each:
# aggregate 'foo' with both 'min' and 'max'
model.read_group(domain, ['foomin:min(foo)', 'foomax:max(foo)'], ...)
The attribute `group_expand` allows to reorder and add empty groups to the
result of `read_group`. Enable this feature for other fields than many2one
fields.
Mahdi Cheikh Rouhou (macr)