The value returned by the search_read in read_progress_bar when passing
a m2m field is a list of ids, which then read_progress_bar tries to use
in a dictionary, list is not a hashable type thus it crashes.
With this commit we convert the group_by_value from list to tuple,
which is hashable, if we're dealing with a many2many field.
task-2508608
Part-of: odoo/odoo#74985
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes
closesodoo/odoo#68299
Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
Fetching all the groups from the DB causes MemoryError on some DBs
Example Accounting > Accounting > Journals > Miscellaneous menu:
```
Traceback (most recent call last):
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 176, in crawl_menu
self.mock_action(action_vals)
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 267, in mock_action
mock_method(model, view, fields_list, domain, group_by)
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 380, in mock_view_tree
self.mock_web_read_group(model, view, domain, group_by, fields_list, limit_group=5)
File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 425, in mock_web_read_group
data = model.web_read_group(domain, fields_list, group_by, limit=limit)["groups"]
File "/home/odoo/src/odoo/14.0/addons/web/models/models.py", line 96, in web_read_group
all_groups = self.read_group(domain, ['display_name'], groupby, lazy=True)
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2248, in read_group
result = self._read_group_raw(domain, fields, groupby, offset=offset, limit=limit, orderby=orderby, lazy=lazy)
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in _read_group_raw
result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in <listcomp>
result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
MemoryError
```
This issue was observed during the upgrade requests upg-18830, target
14.0; and upg-61934 (legacy), target 13.0
The issue can be reproduced on a clean DB with just account_accountant
installed and ~2 millions account moves on a Misc journal.
closesodoo/odoo#73855
X-original-commit: a9b3d9cf9bfd2bf4c9b0904059f1606ad6e04ee4
Signed-off-by: Christophe Simonis <chs@odoo.com>
The method is used to get progress per column in kanban view
(green-yellow-red-red lines in Project, CRM etc). There are two main
usages:
1. get statistics for ``kanban_state`` (red/green circles)
2. get statistics for ``activity_state`` (colored clock icon for overdue/today/planned)
Before this commit all cases were handled by calling search_read and then
counting records per group in a python script. This is very inefficient,
especially for ``activity_state``.
This new implementation relies on ``read_group`` when possible, i.e.,
when both grouping fields (kanban column and progressbar field) are
stored (case 1). It then falls back on a naive implementation inside
``_read_progress_bar``. Cases like 2 above can be addressed by
overriding ``_read_progress_bar``.
We also added some minimal test to ensure that we don't break anything.
1. Performance test on 60 K project.task records (kanban_state):
With a filter for 6 records:
```
| measurement | before | after |
|--------------------+--------+-------|
| number of queries | 8 | 5 |
| query time, ms | 11 | 7 |
| remaining time, ms | 21 | 9 |
```
All records:
```
| measurement | before | after |
|--------------------+--------+-------|
| number of queries | 67 | 5 |
| query time, ms | 300 | 55 |
| remaining time, ms | 1780 | 12 |
```
---
opw-2346901
task-1915411
X-original-commit: 153621bdbab94a2a94a5bbfcabb4111cbc5970d8
Co-authored-by: Raphael Collet <rco@odoo.com>
* IMPACTED VERSIONS
12.0+
* HOW TO REPRODUCE
locale : Locale is en_US (or other SUNDAY based)
view: CRM - My Pipeline - Kanban view
groupBy: date_deadline:week (Expected closing)
records: one record with a planned activity, on date_deadline = 2021-05-02 (SUNDAY)
one record with no planned activity, on date_deadline = 2021-05-09 (SUNDAY)
remark: don't keep any other record in MAY for better visibility
* PROBLEM
The progressbar of the week containing 2021-05-09 displays information about the record
from the week containing 2021-05-02
* CAUSE
1. PostgreSQL `date_trunc` function follows ISO8601 which essentially means that
the start of a WEEK is always MONDAY. There is no argument to change this.
2. _read_group_format_result
https://github.com/odoo/odoo/blob/27da86a138089c1838e4b94f8a6976995b9c1fff/odoo/models.py#L2210-L2219
- Computes a label for a group of records.
- Follows the locale for the label of the week, based on a date which is
always a MONDAY because of `date_trunc`.
3. read_progress_bar
https://github.com/odoo/odoo/blob/88957afca09662af7eaa19df1e40b3699e45e79e/addons/web/models/models.py#L167-L175
- Associates a group label to a record.
- Follows the locale for the label of the week, based on the date of a record
which can be any day of the week. If the record is related to a SUNDAY and
SUNDAY is the first day of the week, it would have been in a group with a
different label in (2.) than in (3.) prior to this change.
* FIX
In 3., before associating a label to a record, we truncate the date to the
ISO start of the period, so that the label is determined for a record in the
same conditions than in 2. The locale is still used to get language-dependent
outputs with babel, but the grouping will always follows ISO8601 (date_trunc).
* TEST
Added a test for this problem case
TASK-ID : 2517848
closesodoo/odoo#70498
X-original-commit: 4560925b26fa79740b9618fd9241d3517b64f43f
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Done for res.partner.bank, res.company and board models. Some override may
have been ignored because heavily linked to business code (like company
in stock).
See merge commit for more details.
Task ID-2330149
COM PR odoo/odoo#61246
ENT PR odoo/enterprise#14561
This commit fixes all issues detected by the new pylint
gettext-variable test.
It converts some calls to the new syntax
_("Foo %s", bar)
to progressively migrate the code to the new syntax.
A few calls were not technically incorrect but still detected by the
linter.
_("Foo" +
"Bar")
has been converted to
_("Foo"
"Bar")
as it has the same effect and make sure the argument is of type
asteroid.Const instead of BinOp).
closesodoo/odoo#53683
Related: odoo/enterprise#11467
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
When company layout is changed, we do not update the company styles for
report: thus we still have the old style for other layout that will not
change anything (besides font).
opw-2269849
closes#53706closesodoo/odoo#53720
X-original-commit: 8897701b1754b40d8a83c5681907c38fb205657f
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
The counters for categories were not updated when filter values were
selected. This commit fixes that situation.
This might impact the global performances of the search panel but
there is still room for improvement. Actually, it could be
possible to reload categories and filters less often by carefully
track the internal changes and the categories/filters attributes
(enable counters, expand,...).
closesodoo/odoo#49307
Related: odoo/enterprise#9795
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
In the search panel, the domain used to compute the field image values
(when the attribute expand is false) was the same as the domain used
to compute counters (if enabled). The idea was that a value bringing an
empty domain is totally useless and should not displayed.
That being true, it turns out that if one does not allow them and
several fields are used, selecting a value in the search panel will very
often totally transform the search panel. From a UI perspective,
this turns out to be bad: the search panel 'moves'.
Let us give an example:
Let us start from a search panel that looks like to:
first_field
A 1
B 3
second_field
C 1
D 2 <--- mouse above D
E 1
with first_field and second_field both with expand="0" and
enable_counters="1". Let us also assume that no record in the global
domain has both first_field=A and second_field=D.
Click on D would make the search panel look like to something like
first_field
B 1
second_field
C 1
D 2
E 1 <--- mouse here
(the selection of D does not impact the values for the second_field but
does for the other first_field values).
This has led us to use basically only the domain comming from
outside of the search panel to compute field image values.
This means that we might now have value with zero count even if expand
is false. In the above situation, a click on D would give us
first_field
A
B 1
second_field
C 1
D 2 <--- mouse still above D
E 1
Task ID: 2154749
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
This commit introduces a new attribute 'limit' for search panel fields
that allows to avoid performance issues. That integer attribute (with
default 200) allows to fix a maximal number of values to display for the
fields. When the number of field values to display reaches the limit,
no values will be displayed. Instead, a warning message will be shown
in the corresponding search panel section.
Note it is possible to have no limit using limit="0" on a field.
This commit reintroduces in a better way the principle brought by the
fix 8d57153b34c04952a85f6642951cb2697016da84.
Task ID: 2154749
The commit introduces two new attributes for search panel fields:
- hierarchize: boolean attribute (default True) available for
many2one fields with select="one". It allows to choose whether
to hierarchize the field values using the _parent_name (if set)
on the field comodel.
Note that a sanitization of the parent hierarchy takes place.
Basically, it ensures that parent chains are
completely in the domain (on comodel) accessible by the user.
See _search_panel_sanitized_parent_hierarchy documentation for
more information.
- expand: boolean attribute (default False) available for many2one
and many2many fields. If set to true, all field values are fetched
and displayed in the search panel. If set to false, only the
values that have at least one corresponding value in the field
model (and in some domain) are fetched.
An exception in the case of an hierarchized field
(hierarchize=True and _parent_name set): more/less values can be
displayed in order to have a good representation of the parent
hierarchy. That means we complete and sanitize the set of initial
field image values.
Note that the fix 8d57153b34c04952a85f6642951cb2697016da84 bringing the
notion of limit in search panel has been reverted in the present commit.
An upcomming commit will reintroduce the limit principle in a better way.
Task ID: 2154749
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
render, render_template, load, activity_schedule_with_view,
get_website_pages should all be private:
It should not be possible to render an aribtrary template only with
its name or id
Still need to render some qweb views from js so the method
render_template is kept public.
This explains why the website editor still need read access on
ir.ui.view as we want to allow any snippet to be rendered.
This commit introduces several changes in the search panel with
respect to record counts:
- the record counts are now also available for
the fields with select="one" attribute
(if not disabled explicitely).
- the record counts are better computed using the idea that
selected values within a group should not impact the counts
for the group values but only the counts for the other
group values.
On the way we have changed two keys in the values returned
by the server:
- 'count' becomes '__count'.
It has been done to avoid a possible clash in case a model would
have a field named 'count' and that the field values would be
wanted for some reason.
- 'name' (multi case) becomes 'display_name'.
It has been done in order to make the select one and multi cases
more similar and factorize some code.
TASK-ID: 2166814
Co-authored-by: Raphaël Collet <rco@openerp.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Alexis Lacroix <laa@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Before this commit, if a many2X field with a big comodel was added
in a search panel, the view using it would crash. For instance that
problem occured in the kanban view for hr.job, where res.users
appears as the comodel for the field user_id.
Now, we fix an arbitrary limit of 200 to the numbers of values to fetch
for each many2X fields in the search panel. This avoid the problem
mentionned above.
Furthermore, in case the limit is attained for a field used
as select="one", the values are displayed without being hierarchized.
Indeed the limit can leads to gaps in the knowledge of the hierarchy
and consequently to a bad representation of it.
Task ID: 2154668
closesodoo/odoo#50334
X-original-commit: 8d57153b34c04952a85f6642951cb2697016da84
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Following https://github.com/odoo/odoo/pull/44393
Delete the qweb template `web.styles_company_report`
Recompute company specific style by going into
Settings > Configure Document Layout > change stuff and save
Print a report, in HTML to get an human readable error
(PDF rendering would just ignore the error)
Before this commit, there was an error "could not get asset content"
This was because the css asset created in db had a value of type string
whereas it should have been the same type as b64encode (which is byte-like)
After this commit, there is no error at rendering time
closes#49456closesodoo/odoo#49529
X-original-commit: 2a7e06663c1281f0cf75f72fc491bc2cc39ef81c
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
With this commit, Selection fields with `required=True` which are
extended via `selection_add` are given proper ondelete policies to
ensure the cleanup of records containing these extended options during
uninstall of the extending module.
This commit also cleans up leftover uninstall hooks that were being used
to handle the same set of problems prior to the ondelete mechanism being
implemented for Selection fields.
closesodoo/odoo#46325
Related: odoo/enterprise#9117
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit, the company specific colors and font were
implemented by systematically overwriting a "virtual" report SCSS
asset file before rendering each report.
This caused many issues, forced frequent asset bundles recomputations
(performance problem + cache invalidation causing random bugs).
And it could simply not work in a multi-company setup where multiple
styles are involved, as the asset management could be made not
thread-safe.
PR #44225 was a first attempt to mitigate the numerous problems by
making the asset bundle invalidation less frequent. But the problems
were still present and a more complete solution was necessary for
multi-company setups.
Besides, the design of the bundle forbade making company specific assets.
This commit uses a different approach: instead of having a
company-specific asset that needs to be constantly updated, a global
"multi-company" asset is maintained and included in the report assets.
It only needs to be generated when a company style changes, not for
every rendering operation.
Unfortunately this change cannot be fully performed without updating the
template declarations, so it will require an update of the `web` (or
`base`) module to be operational.
As this represents a rather invasive change in a stable branch, extensive
testing was conducted to minimize the effects and ensure proper
degradation of features for production deployments where the new code
would be deployed without forcing an update of the `web` module:
- The report SCSS files were left untouched, to prevent any bundle
invalidation, ensuring that old cached assets would remain valid. This
means that single-company setups should not see any visible difference
after pulling the code (with or without updating `web`).
SCSS cleanup will be done later.
- Existing Python methods were kept but emptied, to make sure that
old templates and code would not crash.
- For multi-companies, the last used colors will be applied for all
reports until the `web` module is updated. The old behavior was not
working correctly anyways, so the degradation is actually limited.
- For all setups, changing the colors after deploying this patch will
have no effect unless the `web` module is updated.
/UPDATED FOR master on top of #44393/:
- removed the `res_company.update_scss()` method entirely
- fixed the report SCSS styles, as the variable names referred to the old
behavior, e.g. "$o-company-primary-color" is nonsense.
Renamed to "$o-default-report-primary-color" etc.
--
Improves #44225
Forward of #44393
opw-2168623
opw-2171040
closesodoo/odoo#46647
X-original-commit: a5b1421aecf27b1de408434d3bb6d7ac81f57dc3
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Issue
- Install CRM for example
- Add 41 leads
- Create an activity on each of them
Everything ok, load more shows up
- Add another activity on one of them
Load more doesn't shows up
Cause
The uniquify method:
https://github.com/odoo/odoo/blob/saas-12.3/odoo/models.py#L4187:#L4191
Consider the second activity as a duplicate and removes it.
So, in `web_search_read`:
`len(records) <= limit` is `True` and we ignore all
the others records
Solution
Add `force_search_count` in the context when using this action
to avoid uniquify to falsify the records length.
I added the tree view for this action too. It improves UX.
OPW-2165455
closesodoo/odoo#43149
X-original-commit: 13ec3503fbfb5d3d2b1e824059737bd866a3a9f4
Signed-off-by: Jason Van Malder <jvm-odoo@users.noreply.github.com>
en_US may not be activated as it is possible to create a database in
another language using the database manager.
When trying to install a chart of account, the tax return entry tried
to format a date at the installation of the module, with no lang in
the context. The fallback was made on en_US but an error is raised if
that language is not activated.
As it is a very common scenario to retrieve a language from the
context, add a generic tool method to do it.
Replace and closesodoo/odoo#37629closesodoo/odoo#37568
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
This rev. adds params to the 'web_read_group' method, allowing to
search_read records inside each groups in a single RPC, instead of
doing a 'web_read_group' RPC followed by n 'search_read' RPCs,
where n is the number of opened groups.
Part of task 1915702
With this rev., when there are a lot of groups in a grouped list
view, groups are displayed under several pages, whereas they were
all displayed in the same page before.
This is especially interesting with the new 'expand' attribute, to
ensure that we don't read records for a large number of groups.
By default, the groups limit is set to 80 (like records), and to 10
is the 'expand' attribute is set to true. This limit can be
overriden with the 'groups_limit' attribute.
Part of task 1915702
The departments being already rendered according to the hierarchy,
only the name of the department should be displayed (e.g. 'Direct
Sales' instead of 'Sales / Direct Sales'.
closesodoo/odoo#31353
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This rev. defines a searchPanel widget used in Kanban views to
refine search according to specific dimensions. This wigdet is
displayed as a sidebar to the left of the kanban view.
Part of task 1892462
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Fine-tuning of commit 92b4e2e866,
itself a performance fine-tuning of commit b7e2d47596
which added progress bar support for selection fields.
However it assumed that every record had a value in the selection,
whereas False is (nearly) always a possible value.
opw 1916472
closesodoo/odoo#30322
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
The read_progress_bar method is incredibly not efficient. It is also
wrong in other ways (for example, if 2 groups have same labels, there is
a collision). But let us talk about efficiency: it iterates on each
records on the domain, then, it computes some information. It is really
worse for selection fields: it performs a fields_get for each iteration.
This commit improves the situation by doing the fields_get only once.
Note that this is not the end of the story: this only has a small effect on
the performance of this method. It looks like there is a deeper issue in
some cases (kanban view for subscription, field 'activity_state')
Since rev. 960360af, Date (resp. Datetime) fields return
datetime.date (resp. datetime.datetime) objects. This caused an
issue in the read_progress_bar method which returns a dict, whose
keys are values of the group_by. Indeed, when grouped on a Date or
Datetime field, the key was no longer a string but a datetime.date
or datetime.datetime object, and it crashed when that dict was
turned into a JSON string.
This method isn't tested at all for now, but tests will come
shortly.
Task 1878254.
Purpose
=======
The progress bar on a kanban column doesn't work with selection fields
Specification
=============
In kanban view, subgroupCounts of progressbar should work if columns are grouped with a selection field
Currently, progressbar's subgroupCounts only worked if column is grouped by a 'Many2one' field. If column
is grouped by a 'selection' field then its not working.
For example,
If column is grouped by a 'Many2one' field then we've tuple,
{'state': (2, 'Signatures in Progress')}
So, here for preparing progressBar values for subgroupCounts we'll get
{Signatures in Progress: {overdue: 0, planned: 0, today: 1}}
But for a selection field, we'll have a string 'sent' i.e a technical value,
{'state': 'sent'}
So here 'group_by_value' as a 'key' will never be matched as we have 'sent' instead
of 'Signatures in Progress'. We've a technical value instead of its UI value and
dict of subgroupCounts would be based on UI values that's why it'll never find values.
To fix this issue we need to set proper 'group_by_value'.
That's why added one condition for 'selection' field to gets its value of specific
state inside 'read_progress_bar' method.
Related to Task: 1862371
Before this commit, when grouping a kanban by date with a modifier
e.g. crm.lead grouped by create_date:month
A traceback was thrown on the python side bacause create_date:month was literally not a field
After this commit, the progress bar in that situation displays well,
at the price of copying bits of the read_group infrastructure
OPW 785021
* crm, project
Add a new feature which allows to put a progressbar in the kanban
columns. The progressbar shows with the same color the amount of
records whose value of a given field are the same in the column.
It also indicate the sum of another given field or simply the total
number of records. It also allows to subgroup the column content.
To define a progressbar, add this as a direct child of the kanban
arch:
<progressbar field="<name of the field to use for subgroups>"
colors="{<one possible value for the above field>: <success, warning or danger>, ...}"
sum="<name of the field to sum or nothing to use total number of records>"/>
Also:
- Properly update record model data's parentID when moving a record
- ...
Adrian Torres