Commit Graph
66 Commits
Author SHA1 Message Date
std-odoo 15353f06d7 [IMP] base, web: allow to group by properties
Purpose
=======
Allow to group records by their property values,
like we can do with normal fields.

Technical
=========
Because there's no foreign key, when we group by a relational property
we need to check the existence of the ids in the query (and same for
selection and tags, because we might have value of a deleted
option / tag in database).

Task-3032464

Part-of: odoo/odoo#103510
2023-09-08 09:46:56 +00:00
Jorge Pinna Puissant f609a616d1 [IMP] web: allow to save and read next record in one rpc call
In [1], a new python function was introduced that allows to save and
read in one rpc. This commit, adds the possiblitity to read another
record in the same rpc call. This is particularly useful when you want
to save and read the next record (for instance on a pager).

1: ebd538a194

Task-id : 3453184

closes odoo/odoo#134125

Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2023-09-05 12:22:19 +00:00
Jorge Pinna Puissant ebd538a194 [IMP] web, *: optimize save record from webclient
This commit, adds a new python method (`web_save`) to save a record, and
optionally read-it again in one rpc call. This optimizes the current
behavior that is to save a record in one rpc, and read-it in a second
rpc.

web_save, will receive the list of IDs of the records to save (if this
list is empty it will create the records, if not, it will write on the
existing records), the list of changed fields, and the unity
specification as optional argument to read the created/modified records
(if the specification is not set, the function will return a list of IDs
of the created/modified records).

closes odoo/odoo#133021

Task-id: 3453184
Related: odoo/enterprise#46559
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2023-09-04 09:19:06 +00:00
Aaron Bohy 2a121a32a2 [REF] *: rename unity_web_search_read into web_search_read
Part-of: odoo/odoo#133617
2023-08-31 09:04:58 +00:00
Aaron Bohy 1d71e35cde [REF] web: remove old web_search_read
as it will be replaced by unity_web_search_read

Part-of: odoo/odoo#133617
2023-08-31 09:04:58 +00:00
Raphael Collet 132e9f72ed [REF] *: rename onchange2 to onchange
closes odoo/odoo#133049

Related: odoo/enterprise#46240
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-08-31 05:11:44 +00:00
Aaron Bohy ffcd0bd156 [REF] web: remove expand parameters from web_read_group
The expand, expand_limit and expand_groupby allowed to ask
read_group to populate groups with search_read results directly.
It was only used in the list view, when `expand="1"` was set on
its root node in the arch. Since [1], the list view no longer uses
these arguments, as we uniformized the logic between list and
kanban (where groups are always opened by default).

[1] https://github.com/odoo/odoo/pull/114024

Part of task~3179751

closes odoo/odoo#129881

Signed-off-by: Mathieu Duckerts-Antoine (dam) <dam@odoo.com>
2023-07-27 16:14:21 +02:00
Raphael Collet 7bfbd75d57 [FIX] core: web_read() on new records
This completes 96a98f4f4c in the case
where a many2one field has a new record as value, even when that
many2one field is not a delegate field.

Part-of: odoo/odoo#114024
2023-07-24 20:17:49 +02:00
Raphael Collet 96a98f4f4c [FIX] web: web_read() on new records with _inherits
This commit partly reverts 59afbf6686 and
provides an alternative solution.  In this solution, records.web_read()
returns a list of dicts where the key 'id' corresponds to each record.id
in records, which enables to reliably relate each record to its values.

This also fixes the implementation of web_read() on new records where
the fields contain a reference field with some context.

closes odoo/odoo#128878

Signed-off-by: Raphael Collet <rco@odoo.com>
2023-07-18 19:28:13 +02:00
Raphael Collet 59afbf6686 [FIX] web: web_read() on new records with _inherits
Part-of: odoo/odoo#127718
2023-07-10 18:16:19 +02:00
Raphael Collet b3e8560c9c [FIX] tests: make first call to onchange2() return complete x2many value
The use-case is a first call to onchange2() where:
 - a one2many field has a default value with a new line
 - some onchange method discards that line

The diff should not return a "delete" command for the discarded line,
since the client does not know about it.  Instead, for that first call,
it should behave like if the initial value of the field was empty.

Part-of: odoo/odoo#127718
2023-07-10 18:16:18 +02:00
Raphael Collet 92e5c7b160 [FIX] web: make onchange2() use new() with initial values
This enables an override of new() to do what is needed to make onchanges
on account.payment work as expected.

Part-of: odoo/odoo#127400
2023-07-05 18:46:50 +02:00
Pierre Masereel a746294b46 [FIX] model: correctly count groups
When you are sorting agregates of groups and you have more groups than
the limit, you get a traceback.

It is because you give the orderby to the private _read_group that is
not managed the same way when you pass it to the private _web_read_group
so it doesn't correctly build the query.

As we are calling _read_group to count the groups, we don't need to
order it. So we don't.

closes odoo/odoo#126862

X-original-commit: 6fdf86823088e0e413bef72dc0f20f926aa8f10a
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
Signed-off-by: Pierre Masereel (pim) <pim@odoo.com>
2023-06-30 01:33:00 +02:00
Raphael Collet d30b5c0392 [FIX] web: onchange2() format of values in LINK commands
The LINK command must include the corresponding record data in the
web_read() format instead of the diff() format.  The real difference
shows up in x2many fields: web_read() returns them as lists of dicts (or
list of ids), while diff() returns them as lists of commands.

Part-of: odoo/odoo#124612
2023-06-13 12:49:22 +02:00
Raphael Collet 67a402edb5 [FIX] web: onchange2() copy of sub-x2many fields in cache
onchange2() reads the origin record from database, and copies its field
values in cache on the corresponding new records.  The copied x2many
fields must use NewId instead of their real id.

Part-of: odoo/odoo#124612
2023-06-13 12:49:21 +02:00
Raphael Collet 243a2e8d03 [FIX] web: make onchange2() apply x2many commands on existing record values
When doing an onchange on some existing record, the x2many commands must
be applied on the current x2many value of the record.  When doing an
onchange on a new record, the x2many commands must be applied on empty
x2many values.

Part-of: odoo/odoo#124612
2023-06-13 12:49:20 +02:00
Raphael Collet a980d58e48 [FIX] web: onchange2() did not return default value for x2many field
Make the parameter 'force' in method RecordSnapshot.diff() effective in
the case of x2many fields.

Part-of: odoo/odoo#124612
2023-06-13 12:49:19 +02:00
Raphael Collet b4702cc208 [FIX] web: RecordSnapshot uses variable with incorrect value
Part-of: odoo/odoo#120457
2023-05-05 18:08:09 +02:00
Vincent Schippefilt 7d276aa941 [IMP] web: add reference fields to web_read
add support for fields of type `reference` and `many2one_reference` to `web_read` and `unity_web_search_read`

for both you can add a field_spec requesting fields of the "co-model":

request:
```python
{
    #reference
    'field_reference':
        {
            'fields': {'write_date': {}},
        }

    #many2one_reference
    'm2o_reference_id':
        {
            'fields': {'display_name': {}, 'write_date': {}},
        },
    'm2o_reference_model': {}
}
```
response:
```python
{
    'id': ...,
    #reference
    'field_reference': {
        'id': {'id': 3, 'model': 'comodel_name'},
        'write_date': '2004-11-23 11:30'
    }

    #many2one_reference
    'm2o_reference_id': {
        'id': 3,
        'display_name': "special first day",
        'write_date': '2004-11-23 11:30'
    },
    'm2o_reference_model': 'comodel_name',
}
```

closes odoo/odoo#119995

Task-id: 3284222
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-05-04 00:46:53 +02:00
f5e6494da3 [IMP] web: introduce onchange2
The purpose of onchange2() is to adress two shortcomings of onchange():
 - reduce the payload of the RPC call by minimizing the diff
 - use the "unity" format for returning the data

Because of the dependency of onchange2() on web_read(), the new method
has been introduced in module web.

closes odoo/odoo#119510

Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Julien Castiaux <juc@odoo.com>
Co-authored-by: VincentSchippefilt <VincentSchippefilt@users.noreply.github.com>
2023-04-28 14:44:00 +02:00
Rémy Voet (ryv) 70fd18ef67 [FIX] web: fix bad groupby as str instead of list
A mistake introduced in  234db70d86, the
`groupby` of  `_read_group` should be list/tuple of  `str`, not a `str`.

closes odoo/odoo#119400

Signed-off-by: Raphael Collet <rco@odoo.com>
2023-04-22 06:53:49 +02:00
Vincent Schippefilt 7d2baaa0c7 [IMP] web: project unity_read
Introduce an optimised way of reading a graph of data from the webclient.

Before this commit:
When reading data from the webclient it could at most read multiple ids of the same model in one RPC.
This mean that when reading x2many or specific information on many2one, that could only be done after the initial read (when the client knows the ids of the comodels) and model by model.

After this commit:
Introduce methods web_read and web_search_read_unity. Both method receive a specificiation for the fields instead of a list of fields. The specification can request fields from the model, as well as follow relations and request fields for each relations, recursively.

Example of web_read specification for account_move
```python
{'name': {}},
{'date': {}},
{'journal_id': {'fields': {'display_name':{}}}
},
...
{'invoice_line_ids' :
    {
        'fields': {
            'journal_id' : {'fields': {'display_name:{}}},
            'move_name' : {},
            ...
            'tax_ids' : {
                fields: {
                    'display_name':{},
                    ...
                }
            }
        }
    }
}
```

Result for this example with 2 invoice lines
```python
{
    'id': 1234,
    'name' : 'invoice name ABC',
    'journal_id: {
        'id': 999,
        'display_name': 'Customer Invoices'
    },
    ...
    'invoice_line_ids': [
        {
            'id': 666,
            'journal_id': {
                'id': 999,
                'display_name': 'Customer Invoices'
            },
            'move_name': 'a move name',
            'tax_ids': [
                {
                    'id': 333,
                    'display_name: "15% tax",
                    ...
                }
            ]
        },
        {
            'id': 667,
            'journal_id': {
                'id': 999,
                'display_name': 'Customer Invoices'
            },
            'move_name': 'another move name',
            'tax_ids': [
                {
                    'id': 334,
                    'display_name: "21% customer tax",
                    ...
                }
            ]
        }
    ]

}
```

closes odoo/odoo#119034

Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
2023-04-21 08:02:17 +02:00
Rémy Voet (ryv) 6d930c7b9d [REF] mail: replace override of read_progress_bar _read_group_groupby
closes odoo/odoo#110737

Related: odoo/documentation#4064
Related: odoo/enterprise#38639
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
2023-04-19 21:58:28 +02:00
Rémy Voet (ryv) 234db70d86 [IMP] *: Use the new API of _read_group for backend use
Part-of: odoo/odoo#110737
2023-04-19 21:58:27 +02:00
Yolann Sabaux 3a177c448d [FIX] models: use localized first day of the week when grouping by week
Steps to reproduce:
- Make sure language preference is 'en_US'
- In accounting, in the dashboard click on bills
- Filter 'due_date' by week

Issue: The start day is Monday and should be, for 'en_US', Sunday as it
is the case in the dashboard view in accounting (see appendix).

Cause: The query uses the `date_trunc('week', date)` which in Postgres
retrieves the first day of the week as Monday (ISO week).

Solution: Create an offset in the query depending on the first day of
the locale variable.

Note: the `web/tests/test_read_progress_bar.py` has been modified: since
the default language is 'en_US' there will be an offset of one day.  To
make it less confusing, I used only two anglo-saxons countries so the
day offset is not the variable tested.  (for this matter, pleaser refer
to `test_read_group/tests/test_read_group_process_groupby.py`)

Appendix:
Language (english-US)

		VIEW (per week)			|		DASHBOARD
	___________________________________________________________________
	W23		->	06/05		|	05/29	->	06/04
	W24	06/06	->	06/12		|	06/05 	->	06/11
	W25	06/13	->			|	06/12	->	06/18

		(Monday - Sunday)			(Sunday - Saturday)

Language (french-BE)

		VIEW (per week)			|		DASHBOARD
	___________________________________________________________________
	W22		->	06/05		|	05/30	->	06/05
	W23	06/06	->	06/12		|	06/06 	->	06/12
	W24	06/13	->			|	06/13	->	06/19

		(Monday - Sunday)			(Monday - Sunday)

opw-2747066

closes odoo/odoo#93053

Related: odoo/enterprise#29539
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-02-24 10:05:58 +01:00
Laurent Smet d724f69a1d [FIX] web: Fix useless search_count with count_limit
When count_limit is set and lower than the current number of fetched records, making an extra search_count is useless.

closes odoo/odoo#111895

X-original-commit: 6f90d6924e24e700694111732ee85465f802b22f
Signed-off-by: Rémy Voet <ryv@odoo.com>
2023-02-03 17:41:15 +01:00
fdardenne 4edbe54456 [REM] web: qweb views
QWeb views allowed to render QWeb templates rendered by the server in
a view.

Today, QWeb views are only used in website to see the hierarchy of
their views. The use case of website has now more sense in a client
action rather than an extension of a QWeb view.

This commit removes this view because it is not used anymore and will
probably not find any useful use case in the future.

closes odoo/odoo#103477

Related: odoo/upgrade#4016
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2022-11-15 18:25:12 +01:00
6acfad2651 [IMP] web: speed up pager total counter for large DB
Doing a `search_count` on millions of records could be very slow. On
large databases, the pager counter "1-80 / 8000000" could take several
seconds just to get the total number of records.

This commit limits the counter in list and kanban views to 10k records,
and shows "1-80 / 10000+" if the limit is reached. If you click on
10000+, it updates with the real count (or if you go up to 10000 with
pager or input value).

The search_count on res.partner of odoo.com takes 4s. With this patch,
it will take ~20ms.

Task 2761165

closes odoo/odoo#95642

Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Damien Bouvy <dbo@odoo.com>
2022-07-19 11:50:24 +02:00
Denis Ledoux b03c227e88 [REF] models: refactor fields_view_get, load_views
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.

e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
  in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`

The goal of this revision is to change that so it sends the list of all fields
only once.

In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
  - `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
  - `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`

The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
  As it no longer contains the fields,
  the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
  which is a dict with as key the model name and as values
  the model fields description. It contains the fields description
  for all models implied in the view:
  the model of the main view and the model of all one2many and many2many fields.

With this change, the fields description will only be sent once by model
implied in the view.

In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.

- one2many and many2many fields views are passed directly in the main view
  architecture rather than being put in the `views` key
  of the field description.
  This is actually easier to treat by the web client,
  and this will allow in a future work to cache an entire view in one block
  of text rather than having to combine multiple cached blocks of text
  to return one view.
- one2many and many2many fields which do not have directly embedded views
  have their views directly injected in the architecture,
  so the web client doesn't have to do RPC calls to `load_views`
  for each one2many and many2many fields not having embedded views.
  For instance, this allow to reduce the number of RPC calls to `load_views`
  from 8 to 1 when loading the form of `product.product`.
  Currently, this behavior is limited to 1 level deep but we consider making it
  go all the way down in future works. We did not do it for the moment because
  in certain cases it rises the processing time and the size (bytes) too much.
  e.g. the sale.order view can be 5 levels deep,
  meaning you can reach 4 dialogs on top the main view.
  ```
  sale.order form > order_line > sale.order.line form > invoice_lines >
  account.move.line form > asset_ids > account.asset form >
  depreciation_move_ids > account.move form.
  ```
  This will also benefit in future works to cache an entire view in one block
  of text rather to having to combine multiple cached block of text
  to get one view.
- `fields_view_get` becomes `get_view`.
  As it no longer returns the fields description,
  keeping the `fields` in the name `fields_view_get` no longer makes sense.
  Hence removing `fields` from the method name, it becomes `view_get`.
  As it gets renamed anyway, we take the opportunity to rename it `get_view`,
  which is more in line with the general getter/setter guidelines
  in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
  This is not mandatory, there is no technical reason to rename `load_views` as
  it practically sends the same info as before,
  the view architectures and their fields description. Just in another way.
  We just take the opportunity of this pull request to suggest a cleaner API:
  `_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
  `_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
  in `_get_view` and `get_view`.
  The rationale is that submenu was already no longer used (deprecated)
  and the mobile options is introduced.
  The mobile options is necessary to tell the server to send the mobile views
  for x2many fields (kanban instead of tree).
  Instead of adding a new argument each time we add a new option to
  `fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
  to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
  view information. Now, `get_view` returns a tuple with the view architecture
  as an `etree` node, and the view as a browse record. The rationale is that all
  overrides of `_fields_view_get` were about modifying the arch only
  (e.g. changing the address format/re-organizing the address related field
  nodes of the partner according to the company country).
  To do so, all these overrides were doing `etree.fromstring` to parse the arch
  which was sent in text to convert it to an `etree`,
  then operations were done on the `etree`,
  and then `etree.tostring` was called to convert back the arch to string.
  With this change of signature to send the arch as an `etree`,
  all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
  allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
  has been performed in `get_view`:
  - `fields` is removed, as explained above,
  - `view_id` is renamed `id`,
  - `name` is removed, it was unused by the web client,
  - `type` is removed, it was unused by the web client,
  - `field_parent` is removed, it was unused by the web client,
  - `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
  (now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
  as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
  `fields_view_get`, `_fields_view_get` and `load_views` are provided,
  with deprecation warnings in them.

- The web client could cache the model fields description
  (as it already caches the views),
  so it doesn't need to fetch them again if it asks for another view of a model
  for which he already has the fields description.
  If we do so, `get_views` could return only the list of models used by
  the views, without the fields description as of now,
  and the web client would then call `fields_get` independently only for
  the models for which it doesn't have yet the fields description.
  This would avoid the server to return the fields description
  and to call `fields_get`, which is costly, for each `get_views`,
  therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
  unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
  This is already done for qweb views, it's not done for back-end views.
  Therefore the postprocessing of the views is performed for each `get_views`,
  which is costly, while the view architecture doesn't change for users
  belonging to the same groups, according to the groups implied by the view.

This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.

Part-of: odoo/odoo#87522
2022-04-29 09:57:44 +02:00
Gorash 880954ebfc [IMP] *: remove _render from ir.ui.view and simplify report
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.

The report rendering and call `ir.qweb` instead of `ir.ui.view`.

Part-of: odoo/odoo#85110
2022-03-29 10:56:15 +02:00
Adrian Torres e2f3ac24d0 [IMP] web: allow read_progress_bar to group by m2m fields
The value returned by the search_read in read_progress_bar when passing
a m2m field is a list of ids, which then read_progress_bar tries to use
in a dictionary, list is not a hashable type thus it crashes.

With this commit we convert the group_by_value from list to tuple,
which is hashable, if we're dealing with a many2many field.

task-2508608

Part-of: odoo/odoo#74985
2021-08-26 16:24:59 +00:00
Gorash 7df343dd1b [IMP] base: QWeb _render return Markup unicode instead of utf8 bytes
In order to limit encoding decoding, the _render method returns a
unicode string in the markup safe object instead of a MarkupSafeBytes

closes odoo/odoo#68299

Related: odoo/upgrade#2454
Related: odoo/enterprise#17270
Signed-off-by: Antony Lesuisse (al) <al@openerp.com>
2021-08-03 16:20:22 +00:00
Alvaro Fuentes aa4fad64ce [FIX] web: fix web_read_group total groups count
Fetching all the groups from the DB causes MemoryError on some DBs
Example Accounting > Accounting > Journals > Miscellaneous menu:
```
 Traceback (most recent call last):
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 176, in crawl_menu
    self.mock_action(action_vals)
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 267, in mock_action
    mock_method(model, view, fields_list, domain, group_by)
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 380, in mock_view_tree
    self.mock_web_read_group(model, view, domain, group_by, fields_list, limit_group=5)
   File "/tmp/tmpbnah9jtp/migrations/base/tests/test_mock_crawl.py", line 425, in mock_web_read_group
    data = model.web_read_group(domain, fields_list, group_by, limit=limit)["groups"]
   File "/home/odoo/src/odoo/14.0/addons/web/models/models.py", line 96, in web_read_group
    all_groups = self.read_group(domain, ['display_name'], groupby, lazy=True)
   File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2248, in read_group
    result = self._read_group_raw(domain, fields, groupby, offset=offset, limit=limit, orderby=orderby, lazy=lazy)
   File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in _read_group_raw
    result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
   File "/home/odoo/src/odoo/14.0/odoo/models.py", line 2387, in <listcomp>
    result = [self._read_group_format_result(d, annotated_groupbys, groupby, domain) for d in data]
 MemoryError
```

This issue was observed during the upgrade requests upg-18830, target
14.0; and upg-61934 (legacy), target 13.0

The issue can be reproduced on a clean DB with just account_accountant
installed and ~2 millions account moves on a Misc journal.

closes odoo/odoo#73855

X-original-commit: a9b3d9cf9bfd2bf4c9b0904059f1606ad6e04ee4
Signed-off-by: Christophe Simonis <chs@odoo.com>
2021-07-16 11:29:29 +00:00
Ivan YelizarievandRaphael Collet feecd15956 [FIX] web: speed up read_progress_bar
The method is used to get progress per column in kanban view
(green-yellow-red-red lines in Project, CRM etc).  There are two main
usages:

1. get statistics for ``kanban_state`` (red/green circles)
2. get statistics for ``activity_state`` (colored clock icon for overdue/today/planned)

Before this commit all cases were handled by calling search_read and then
counting records per group in a python script.  This is very inefficient,
especially for ``activity_state``.

This new implementation relies on ``read_group`` when possible, i.e.,
when both grouping fields (kanban column and progressbar field) are
stored (case 1).  It then falls back on a naive implementation inside
``_read_progress_bar``.  Cases like 2 above can be addressed by
overriding ``_read_progress_bar``.

We also added some minimal test to ensure that we don't break anything.

1. Performance test on 60 K project.task records (kanban_state):

With a filter for 6 records:

```
| measurement        | before | after |
|--------------------+--------+-------|
| number of queries  |      8 |     5 |
| query time, ms     |     11 |     7 |
| remaining time, ms |     21 |     9 |
```

All records:
```
| measurement        | before | after |
|--------------------+--------+-------|
| number of queries  |     67 |     5 |
| query time, ms     |    300 |    55 |
| remaining time, ms |   1780 |    12 |
```

---

opw-2346901
task-1915411

X-original-commit: 153621bdbab94a2a94a5bbfcabb4111cbc5970d8
Co-authored-by: Raphael Collet <rco@odoo.com>
2021-07-16 11:16:34 +00:00
abd-msyukyu-odoo a03c882a56 [FIX] web: fix kanban view progressbars related to records in another group (groupby:week)
* IMPACTED VERSIONS

  12.0+

* HOW TO REPRODUCE

locale :  Locale is en_US (or other SUNDAY based)
view:     CRM - My Pipeline - Kanban view
groupBy:  date_deadline:week (Expected closing)
records:  one record with a planned activity, on date_deadline = 2021-05-02 (SUNDAY)
          one record with no planned activity, on date_deadline = 2021-05-09 (SUNDAY)
remark:   don't keep any other record in MAY for better visibility

* PROBLEM

The progressbar of the week containing 2021-05-09 displays information about the record
from the week containing 2021-05-02

* CAUSE

1. PostgreSQL `date_trunc` function follows ISO8601 which essentially means that
  the start of a WEEK is always MONDAY. There is no argument to change this.

2. _read_group_format_result
  https://github.com/odoo/odoo/blob/27da86a138089c1838e4b94f8a6976995b9c1fff/odoo/models.py#L2210-L2219

  - Computes a label for a group of records.
  - Follows the locale for the label of the week, based on a date which is
    always a MONDAY because of `date_trunc`.

3. read_progress_bar
  https://github.com/odoo/odoo/blob/88957afca09662af7eaa19df1e40b3699e45e79e/addons/web/models/models.py#L167-L175

  - Associates a group label to a record.
  - Follows the locale for the label of the week, based on the date of a record
    which can be any day of the week. If the record is related to a SUNDAY and
    SUNDAY is the first day of the week, it would have been in a group with a
    different label in (2.) than in (3.) prior to this change.

* FIX

In 3., before associating a label to a record, we truncate the date to the
ISO start of the period, so that the label is determined for a record in the
same conditions than in 2. The locale is still used to get language-dependent
outputs with babel, but the grouping will always follows ISO8601 (date_trunc).

* TEST

Added a test for this problem case

TASK-ID : 2517848

closes odoo/odoo#70498

X-original-commit: 4560925b26fa79740b9618fd9241d3517b64f43f
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-05-06 17:56:20 +00:00
Victor Feyens caeda18bec [IMP] base, various: support batch creation
Done for res.partner.bank, res.company and board models. Some override may
have been ignored because heavily linked to business code (like company
in stock).

See merge commit for more details.

Task ID-2330149
COM PR odoo/odoo#61246
ENT PR odoo/enterprise#14561
2020-12-03 10:18:36 +00:00
Martin Trigaux 400cc4f14e [FIX] *: correct all or improve code translation lookup
This commit fixes all issues detected by the new pylint
gettext-variable test.
It converts some calls to the new syntax
  _("Foo %s", bar)

to progressively migrate the code to the new syntax.

A few calls were not technically incorrect but still detected by the
linter.

  _("Foo" +
    "Bar")

has been converted to

  _("Foo"
    "Bar")

as it has the same effect and make sure the argument is of type
asteroid.Const instead of BinOp).

closes odoo/odoo#53683

Related: odoo/enterprise#11467
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-06-30 10:19:59 +00:00
Nicolas Lempereur 03a0b2d22c [FIX] web: company report style on change layout
When company layout is changed, we do not update the company styles for
report: thus we still have the old style for other layout that will not
change anything (besides font).

opw-2269849
closes #53706

closes odoo/odoo#53720

X-original-commit: 8897701b1754b40d8a83c5681907c38fb205657f
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
2020-06-26 09:15:44 +00:00
Mathieu Duckerts-Antoine 7002bdb239 [FIX] web: always use filter_domain for search panel counters
The counters for categories were not updated when filter values were
selected. This commit fixes that situation.
This might impact the global performances of the search panel but
there is still room for improvement. Actually, it could be
possible to reload categories and filters less often by carefully
track the internal changes and the categories/filters attributes
(enable counters, expand,...).

closes odoo/odoo#49307

Related: odoo/enterprise#9795
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2020-06-12 13:34:04 +00:00
Mathieu Duckerts-AntoineandJulien Mougenot 46306b95d8 [IMP] web: expand domain != count domain
In the search panel, the domain used to compute the field image values
(when the attribute expand is false) was the same as the domain used
to compute counters (if enabled). The idea was that a value bringing an
empty domain is totally useless and should not displayed.
That being true, it turns out that if one does not allow them and
several fields are used, selecting a value in the search panel will very
often totally transform the search panel. From a UI perspective,
this turns out to be bad: the search panel 'moves'.
Let us give an example:
Let us start from a search panel that looks like to:

first_field
    A  1
    B  3
second_field
    C  1
    D  2 <--- mouse above D
    E  1

with first_field and second_field both with expand="0" and
enable_counters="1". Let us also assume that no record in the global
domain has both first_field=A and second_field=D.
Click on D would make the search panel look like to something like

first_field
    B  1
second_field
    C  1
    D  2
    E  1 <--- mouse here

(the selection of D does not impact the values for the second_field but
does for the other first_field values).

This has led us to use basically only the domain comming from
outside of the search panel to compute field image values.

This means that we might now have value with zero count even if expand
is false. In the above situation, a click on D would give us

first_field
    A
    B  1
second_field
    C  1
    D  2 <--- mouse still above D
    E  1

Task ID: 2154749

Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
2020-06-12 13:34:04 +00:00
Mathieu Duckerts-Antoine 3306a0f414 [IMP] *: limit in search panel
This commit introduces a new attribute 'limit' for search panel fields
that allows to avoid performance issues. That integer attribute (with
default 200) allows to fix a maximal number of values to display for the
fields. When the number of field values to display reaches the limit,
no values will be displayed. Instead, a warning message will be shown
in the corresponding search panel section.
Note it is possible to have no limit using limit="0" on a field.
This commit reintroduces in a better way the principle brought by the
fix 8d57153b34c04952a85f6642951cb2697016da84.

Task ID: 2154749
2020-06-12 13:34:04 +00:00
Mathieu Duckerts-AntoineandJulien Mougenot e5585c078e [IMP] *: expand and hierarchize in search panel
The commit introduces two new attributes for search panel fields:

    - hierarchize: boolean attribute (default True) available for
      many2one fields with select="one". It allows to choose whether
      to hierarchize the field values using the _parent_name (if set)
      on the field comodel.
      Note that a sanitization of the parent hierarchy takes place.
      Basically, it ensures that parent chains are
      completely in the domain (on comodel) accessible by the user.
      See _search_panel_sanitized_parent_hierarchy documentation for
      more information.

    - expand: boolean attribute (default False) available for many2one
      and many2many fields. If set to true, all field values are fetched
      and displayed in the search panel. If set to false, only the
      values that have at least one corresponding value in the field
      model (and in some domain) are fetched.
      An exception in the case of an hierarchized field
      (hierarchize=True and _parent_name set): more/less values can be
      displayed in order to have a good representation of the parent
      hierarchy. That means we complete and sanitize the set of initial
      field image values.

Note that the fix 8d57153b34c04952a85f6642951cb2697016da84 bringing the
notion of limit in search panel has been reverted in the present commit.
An upcomming commit will reintroduce the limit principle in a better way.

Task ID: 2154749

Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
2020-06-12 13:34:04 +00:00
Julien Mougenot f5f0ca843d [IMP] *: enable_counters with false as default
In the search panel, the attribute disable_counters with default False
has been changed to enable_counters with default False.
2020-06-12 13:34:04 +00:00
Martin Trigaux d9287caf94 [IMP] *: convert to private methods
render, render_template, load, activity_schedule_with_view,
get_website_pages should all be private:
It should not be possible to render an aribtrary template only with
its name or id

Still need to render some qweb views from js so the method
render_template is kept public.
This explains why the website editor still need read access on
ir.ui.view as we want to allow any snippet to be rendered.
2020-05-14 13:59:10 +02:00
2c102c292f [IMP] web: search panel counter changes
This commit introduces several changes in the search panel with
respect to record counts:

    - the record counts are now also available for
      the fields with select="one" attribute
      (if not disabled explicitely).
    - the record counts are better computed using the idea that
      selected values within a group should not impact the counts
      for the group values but only the counts for the other
      group values.

On the way we have changed two keys in the values returned
by the server:
    - 'count' becomes '__count'.
      It has been done to avoid a possible clash in case a model would
      have a field named 'count' and that the field values would be
      wanted for some reason.
    - 'name' (multi case) becomes 'display_name'.
      It has been done in order to make the select one and multi cases
      more similar and factorize some code.

TASK-ID: 2166814

Co-authored-by: Raphaël Collet <rco@openerp.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Alexis Lacroix <laa@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
2020-05-05 19:23:20 +00:00
Mathieu Duckerts-Antoine 7b402c5e59 [FIX] web: limit on search panel values
Before this commit, if a many2X field with a big comodel was added
in a search panel, the view using it would crash. For instance that
problem occured in the kanban view for hr.job, where res.users
appears as the comodel for the field user_id.

Now, we fix an arbitrary limit of 200 to the numbers of values to fetch
for each many2X fields in the search panel. This avoid the problem
mentionned above.
Furthermore, in case the limit is attained for a field used
as select="one", the values are displayed without being hierarchized.
Indeed the limit can leads to gaps in the knowledge of the hierarchy
and consequently to a bad representation of it.

Task ID: 2154668

closes odoo/odoo#50334

X-original-commit: 8d57153b34c04952a85f6642951cb2697016da84
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2020-04-28 14:57:58 +00:00
Lucas Perais (lpe) 2ff76674d2 [FIX] web: style attachment company specific always in bytes
Following https://github.com/odoo/odoo/pull/44393

Delete the qweb template `web.styles_company_report`
Recompute company specific style by going into
Settings > Configure Document Layout > change stuff and save
Print a report, in HTML to get an human readable error
(PDF rendering would just ignore the error)

Before this commit, there was an error "could not get asset content"
This was because the css asset created in db had a value of type string
whereas it should have been the same type as b64encode (which is byte-like)

After this commit, there is no error at rendering time

closes #49456

closes odoo/odoo#49529

X-original-commit: 2a7e06663c1281f0cf75f72fc491bc2cc39ef81c
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
2020-04-14 12:17:00 +00:00
Adrian Torres 1daf8eb127 [FIX] *: set ondelete policy of required Selection fields
With this commit, Selection fields with `required=True` which are
extended via `selection_add` are given proper ondelete policies to
ensure the cleanup of records containing these extended options during
uninstall of the extending module.

This commit also cleans up leftover uninstall hooks that were being used
to handle the same set of problems prior to the ondelete mechanism being
implemented for Selection fields.

closes odoo/odoo#46325

Related: odoo/enterprise#9117
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-03-30 13:42:04 +00:00
Lucas Perais (lpe) 9ce6136be7 [FIX] web, base: make company-specific report assets static
Before this commit, the company specific colors and font were
implemented by systematically overwriting a "virtual" report SCSS
asset file before rendering each report.
This caused many issues, forced frequent asset bundles recomputations
(performance problem + cache invalidation causing random bugs).
And it could simply not work in a multi-company setup where multiple
styles are involved, as the asset management could be made not
thread-safe.

PR #44225 was a first attempt to mitigate the numerous problems by
making the asset bundle invalidation less frequent. But the problems
were still present and a more complete solution was necessary for
multi-company setups.

Besides, the design of the bundle forbade making company specific assets.

This commit uses a different approach: instead of having a
company-specific asset that needs to be constantly updated, a global
"multi-company" asset is maintained and included in the report assets.
It only needs to be generated when a company style changes, not for
every rendering operation.

Unfortunately this change cannot be fully performed without updating the
template declarations, so it will require an update of the `web` (or
`base`) module to be operational.

As this represents a rather invasive change in a stable branch, extensive
testing was conducted to minimize the effects and ensure proper
degradation of features for production deployments where the new code
would be deployed without forcing an update of the `web` module:
- The report SCSS files were left untouched, to prevent any bundle
  invalidation, ensuring that old cached assets would remain valid. This
  means that single-company setups should not see any visible difference
  after pulling the code (with or without updating `web`).
  SCSS cleanup will be done later.
- Existing Python methods were kept but emptied, to make sure that
  old templates and code would not crash.
- For multi-companies, the last used colors will be applied for all
  reports until the `web` module is updated. The old behavior was not
  working correctly anyways, so the degradation is actually limited.
- For all setups, changing the colors after deploying this patch will
  have no effect unless the `web` module is updated.

/UPDATED FOR master on top of #44393/:
- removed the `res_company.update_scss()` method entirely
- fixed the report SCSS styles, as the variable names referred to the old
  behavior, e.g. "$o-company-primary-color" is nonsense.
  Renamed to "$o-default-report-primary-color" etc.

--
Improves #44225
Forward of #44393

opw-2168623
opw-2171040

closes odoo/odoo#46647

X-original-commit: a5b1421aecf27b1de408434d3bb6d7ac81f57dc3
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2020-03-02 13:22:09 +00:00
Lucas Lefèvre 3519b9872e [IMP] web: Add support of js_class attribute for qweb views
Task 2119567
2020-01-21 13:16:58 +01:00