- The batch version of get_metadata won't work correctly for the xmlid
returns due to the limit=1 (cause by
718b0f8b63 ). Fix the
previous issue by reversing the order and without limit.
- The read should be read with the correct user.
closesodoo/odoo#79091
X-original-commit: 29a53f19a7729f486cbc6ff69893b8d150c3512d
Signed-off-by: Rémy Voet <ryv@odoo.com>
With a lot of module installed setup_models can be quite slow.
One of the reason for that are the call to resolve_mro.
This operation is fast, but called for a lot of models/fields this
become an important part of get_depends and setup_base.
- This is call once for each models in setup_base
- This is call more than once for each computed_field in get_depends.
The use of a cache shows significative performance improvements.
During the install of a database with all modules, the install times
is reduced by ~3 minutes over ~14
Part-of: odoo/odoo#78898
Co-authored-by: Raphael Collet <rco@odoo.com>
The exists method (of BaseModel) doesn't work with model
without SQL table but with a table query (see
f2ceef0e2f)
This method is call when we try to read a forbidden
record (`forbidden = missing.exists()` in `_read`)
- Fix it by using a Query object (which able the case).
- Also use a partition method instead of duplicate it in the method.
- Fix the CRM Lead Report which can return a Falsy id
task-2633558
closesodoo/odoo#77644
X-original-commit: 70a68615c722e7145b2ba8bb7ca86a8373b5aa4e
Signed-off-by: Rémy Voet <ryv-odoo@users.noreply.github.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The query count increases by one because the old code (with
hand-rolled savepoints) never released the `model_load` savepoint.
Part-of: odoo/odoo#76243
Because the field is stored it doesn't make much sense to have a
context dependency and that can cause significant issues. However
`name_get` looks pretty innocent and it's not necessarily insane to
have a context dependency *there* (or even in the average
`display_name` as they're normally non-stored computed fields).
`_compute_display_name` previously blanked (blacklisted) just a few
context values, but doing the opposite is probably the better idea.
See also: odoo/odoo#68946
OPW-2453956
Task 2500978
closesodoo/odoo#76469
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
For searches, Odoo uses `unaccent` if it's available. On some
technical fields this is completely unnecessary and precludes the use
of indexes.
This PR provides:
* an opt-out (`unaccent = False`) on `String` and `Text` fields
* a warning if `unaccent` is enabled on *parent_path* fields as their
performance can be rather critical and not using the index is quite
an issue (note: the check that `parent_path` fields have been moved
outside of the check for their existence as we want to check that
the field is declared and correctly configured in all cases,
probably)
Task 2627454
Part-of: odoo/odoo#76436
Assume that model A has a computed field G that depends on both fields
F1 and F2. Assume that model B inherits from A (_inherits). When G is
computed during an onchange, it must use the form values for all its
dependencies F1 and F2. Before this commit, only the field triggering
the onchange was assigned on the parent record (see (*) below.)
record parent
--- ------------+-------------+------------
initial state | F1=0, F2=0 | F1=0, F2=0
assign F1=1 | F1=1, F2=0 | F1=1, F2=0
assign F2=2 | F1=1, F2=1 | F1=0, F2=1 (*)
The commit also includes another patch: when assigning the parent
record, the ORM determines the dependent records (in this case, the main
record in the onchange). If the delegate field (many2one from B to A)
has an inverse one2many field, the ORM uses that field to determine
dependent records. However, in the case of a new record, that field is
not in cache, and its value is determined to be... empty! The patch
consists in "fixing" the value of that field when we determine the
parent record (when the delegate field is accessed.)
Part-of: odoo/odoo#76042
Co-authored-by: William André <wan@odoo.com>
Before this commit, calling read_group on a model and grouping by a
selection field would return the groups in alphabetical order, meaning
that if your selection field's options were declared as:
('z', 'Z'), ('a', 'A')
read_group would return first group 'a' and then group 'z', instead of
groups 'z' and then 'a' which some might expect.
With this commit, it is now possible to set a fields.Selection
group_expand attribute to `True` when declaring it, which will use a
default group_expand implementation specific to Selection fields, this
means that when grouping by a Selection field with `group_expand=True`
it will always return the groups in the definition order of the
selection options.
We achieve this by leveraging the group_expand field attribute which was
designed for changing the groups returned by read_group.
Since this attribute was thought only to be implemented on a
Model-by-Model basis and here we need to use it as a generic function
for all Selection fields (explicit group_expand declarations have higher
precedence), the generic method has been implemented inside
fields.Selection and takes an extra records parameter which holds a
reference to the recordset/model on which read_group was called. This
extra parameter only applies to field implementations of group_expand
and in this case is what allows this specific feature to work with
dynamic Selection fields (function as options).
A side-effect of this implementation is that a read_group call that
groups by a Selection field with `group_expand=True` that uses the
default group_expand will always return all possible groups, even
empty ones.
Task-ID 2635052
closesodoo/odoo#75856
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
PURPOSE
Improve import wizard as followup of recent improvements.
SPECIFICATIONS
Preview lines
- make the preview lines more visible
- set font-weight: 500 on o_import_header_name
- remove the text-muted class from the data preview line
- allow the user to preview the first 5 non-empty values
- when hovering on the data preview span, display a tooltip with the first
5 non-empty values of the file column (with 'Preview' as the tooltip title)
- In the 'when a value cannot be matched' dropdown:
add a new 'skip record' option for the following field types: boolean,
many2one, many2many, selection
add a new 'set empty' for the following field types: many2one, many2many,
selection hide this option if the field is required
When a value cannot be matched:
the 'skip record' option will make sure that lines with an unmatched value
will not be imported (i.e. skipped at import)
the 'set empty' option will set the field value to False when the value
cannot be matched
- Restore colored background on alert boxes by removing some css rules
Task-2504343
closesodoo/odoo#73713
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Co-authored-by: Mohammed Shekha <msh@odoo.com>
* = crm_livechat, hr, hr_holidays, im_livechat, mail_bot, purchase, sms,
snailmail, survey, test_discuss_full, test_mail, web_editor, website,
website_livechat
- Create new model `mail.guest` for guests.
- Rewrite some RPCs to target routes rather than model methods so that
guests are able to use them.
- Patch JS and python models to support guests.
- Create a stand-alone page and boot the channel in it.
task-2494829
closesodoo/odoo#75496
Related: odoo/enterprise#20417
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
Grouping by many2many has been added with PR odoo/odoo#68958 (commit f4f570c3d6b403512d8523c74acd34d485f46af1)
However the display names are not resolved resulting in the grouping
displaying the actual id of the records.
This also breaks some features such as `read_groups` which expect the
group by data to contain a tuple (id, x) instead of just the id, this
commit adds name resolving on many2many group bys.
Task ID: 2398734
Part-of: odoo/odoo#74985
Since now, there was no possibilities to pay a Sale Order in a point of
sale without re-encoding every lines of the sale order in a pos order.
So we've now added the possibility to choose a sale order from the point
of sale, and apply a down payment or settle the selected order. You
still get the possibility to invoice it, and the stock is correctly
managed.
The SO is then updated accordingly to what have been done throught the
POS.
closesodoo/odoo#74096
Task-id: 2500902
Signed-off-by: pimodoo <pimodoo@users.noreply.github.com>
The fields dict maps field names to their description. Before this
commit, the field name wasn't specified in the description, meaning
that we could never manipulate the field description without its
key, which would often be convenient.
closesodoo/odoo#75121
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The method is used to get progress per column in kanban view
(green-yellow-red-red lines in Project, CRM etc). There are two main
usages:
1. get statistics for ``kanban_state`` (red/green circles)
2. get statistics for ``activity_state`` (colored clock icon for overdue/today/planned)
Before this commit all cases were handled by calling search_read and then
counting records per group in a python script. This is very inefficient,
especially for ``activity_state``.
This new implementation relies on ``read_group`` when possible, i.e.,
when both grouping fields (kanban column and progressbar field) are
stored (case 1). It then falls back on a naive implementation inside
``_read_progress_bar``. Cases like 2 above can be addressed by
overriding ``_read_progress_bar``.
We also added some minimal test to ensure that we don't break anything.
1. Performance test on 60 K project.task records (kanban_state):
With a filter for 6 records:
```
| measurement | before | after |
|--------------------+--------+-------|
| number of queries | 8 | 5 |
| query time, ms | 11 | 7 |
| remaining time, ms | 21 | 9 |
```
All records:
```
| measurement | before | after |
|--------------------+--------+-------|
| number of queries | 67 | 5 |
| query time, ms | 300 | 55 |
| remaining time, ms | 1780 | 12 |
```
---
opw-2346901
task-1915411
X-original-commit: 153621bdbab94a2a94a5bbfcabb4111cbc5970d8
Co-authored-by: Raphael Collet <rco@odoo.com>
Re-introduce after discussion with AL the function that allow to specify
a custom placeholder for a specific model.
It has been removed because no more used since we use avatar mixin for
res.users and res.company. But it doesn't means that each model should add
his own mixin and controller and ... Keep it simple!
Use it for product and product template to have a default placeholder more
representative of the model.
Courtesy of xlu-odoo for this design
opw-2513801
closesodoo/odoo#73576
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
The use-case is when a "parent" field is put on the form view, as a
readonly field, and we create a new record. Before the fix, the fields
that depend on the parent record are always False, until the record is
actually created.
closesodoo/odoo#73446
X-original-commit: ff2af932c3b73dc59620a4ce17da3a13cab12cbe
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This allows onchange to recompute fields that depend on the inherited
field that has changed. The use-case we have is a model that inherits
from 'account.move'. On its form view, setting the (inherited) journal
field does not recompute the (inherited) company field.
When the field 'journal_id' is modified, the onchange should:
- assign the new value to move_id.journal_id (*)
- recompute move_id.company_id (related='journal_id.company_id')
- recompute company_id
This commit adds the missing part (*).
X-original-commit: 2e05d0a361aa06a34eecf64613821f2e92295298
Due to odoo/odoo#51075 it's an error to assign the `_date_name` field on
an empty recordset.
```
Traceback (most recent call last):
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 1582, in _fields_view_get
arch_etree = getattr(self, '_get_default_%s_view' % view_type)()
File "/home/odoo/src/odoo/14.0/odoo/models.py", line 1495, in _get_default_calendar_view
self._date_name = dt
AttributeError: 'project.phase' object attribute '_date_name' is read-only
```
Issue observed on upgrade requests 2615 and 3121.
closesodoo/odoo#73401
X-original-commit: 1bbe88796a5b03ab1ab2faec1f0fe4f735fcb485
Signed-off-by: Christophe Simonis <chs@odoo.com>
Co-authored-by: Christophe Simonis <chs@odoo.com>
* Objectives:
- Allow the generation of empty date/datetime groups outside the bounds
defined by the existing data during a read_group, with the fill_temporal
context key.
{[Jan - empty]->[Feb]->[Mar - empty]->[Apr]->[May - empty]}
This will allow to get a guaranteed amount of groups regardless of the
existing data.
- Request contiguous date/datetime groups between custom bounds which do not
cover the entire domain of existence (existing data)
[Jan] {[Jul]->[Aug - empty]->[Sep]} [Dec]
If some records were "forgotten" before the desired "interesting" period, we
won't have to display "many" undesired empty groups to show them.
* Implementation:
Modify the fill_temporal context key to accept a dictionary format,
with 3 keys, to be used in _read_group_fill_temporal:
1. fill_from
2. fill_to
3. min_groups
- (1., 2.) offer a way to guarantee that at least a certain date range will
be returned as groups. They are also the bounds to fill. Any group outside
those bounds will not be removed, but there will be no empty group added. If
any of those keys is omitted, existing bounds are used, if applicable. If
fill_from and fill_to are not specified, and there is no data, no group is
returned.
edge case example:
existing dates: [Jan, Sep]
fill_from: [Apr]
fill_to: [Jul]
interval: 1 month
result: [Jan, |Apr, May, Jun, Jul|, Sep]
general usage examples:
a. We have a Forecast in a Kanban view. We are in April and we want the
forecast to cover the [Apr - Jul] period.
- domain: [(>= Apr), (<= Jul)] (limit the read_group to
the forecast)
- fill_from: Apr (current month)
- fill_to: Jul
The result would be [Apr, May, Jun, Jul]
b. We want the same Forecast [Apr - Jul], but there is a chance that some
records were forgotten in Jan, and we want to be able to reschedule
them, but we don't need the fill_temporal to take effect from Jan.
- domain: [(<= Jul)] (we removed the first bound)
- fill_from: Apr (current month)
- fill_to: Jul
The result would be [Jan, |Apr, May, Jun, Jul|]. We have the forecast
with the fill_temporal, and every month preceding the forecast as long
as there is at least one record in it (no empty month before the
forecast).
- (3.) Will be used as a way to guarantee a certain amount of returned groups
for a read_group grouped on a {date, datetime} field. This will only work
if there is at least one group to be used as reference (starting point) to
create supplementary groups if necessary.
example :
existing dates: [Mar, Apr]
min_groups: 4
interval: 1 month
result: [Mar, Apr, May, Jun]
- fill_temporal: {} will be considered as True (backend & frontend)
- The new key options are independant from each other (do not require the
others to be set in order to function properly)
Task-ID: 2243913
PR odoo/odoo#69380
1. Objective
- We are in a Kanban view, and we group by a "date/datetime" field.
- In order to "quick create" a record in a group, or to "drag & drop" a
record between 2 groups, we need a default value for this date field.
- In this case, a group represents a "range" of dates. A default value could
be the last date of this range.
- Prior to this commit, the view had only access to a display string for this
date range. We want to have access to the concrete bounds (start/end dates)
- Since some date/datetime fields have backend constraints, it would be
difficult to generate a compliant global default value. As such, the use of
the default value should be disabled by default and activated on a field by
field basis.
2. Usage
- Use the last day (for dates), or second (for datetimes) of the range to set
a default value in kanban "quick create" and "drag & drop"
- Use the end of the range of the last group from a read_group to be able to
request the next group (chronological order) in subsequent read_group calls
3. What are the changes in this commit
- read_group from the "core" "models.py", with the added __range for
date(time) fields, a dictionary for each group. The keys are the field
names and the values are a dictionary with the following keys: value:
- from: starting date(time) of the range (inclusive)
- to: ending date(time) of the range (exclusive)
- XML changes:
- with a new xml attribute on <field> tag allow_group_range_value (for the
kanban view):
- we allow (or disallow) supported non-readonly fields to be:
- draggable
- quick created
- this attribute can be used only on date(time) fields, and if not set,
the default behavior is false
- the naming is subject to contention because it relates to
different features. The relation comes from the constraint:
To perform a drag&drop or a quickCreate, we must be able to get the
value from the group containing the record.
- JS changes:
- we add "group.range" after a read_group for date fields, which is a
dictionary: {field_name: {from, to}}, for each date(time) groupby field
- since date(time) fields can use the format "date_field:granularity" when
used in a groupby, we have to properly split out the granularity to ensure
compatibility with the code previously in place (drag&drop procedures)
- update mock_server and sample_server to make use of the date range.
Adding support for datetime grouping to both mock and sample servers, for
tests and previews.
- the default value for kanban drag&drop and quickCreate features is the
last day/second of the range for date/datetime fields
4. Why can't this range be computed from the original display value with
moment.js
The Babel python library that we use for date(time) formatting (2.6.0) is kept
at the same version as the package in stable Debian Linux. This version has
some issues with displaying weeks consistently around the new year in certain
locales.
We cannot reverse compute a date(time) label with moment.js to get a range since
nothing guarantees that the range will be the same one that Babel used to
output it.
5. read_group return format discussion
Prior to this commit, the format of the value for the groupBy field was
either :
- String : used as a display value for the group.
Most notably, for date and datetime, this value cannot be used as a field
value in most cases. It would be more practical to also return the date
range along with the display value, so that Drag&Drop and QuickCreate
functionalities can be properly enabled in views
example :
"March 2021" has the range ['&', (field, '>=', 2021-03-01),
(field, '<', 2021-04-01)]
This information is lost at the end of the read_group.
- Array : used to indicate a res_id in case of a many2one field
res_id = array[0]
field_diplay_value = array[1]
Since we cannot use the array format to send the date(time)s of the period range
because it could be confused with the many2one format, this commit introduces
the use of a new `__range` key which is a dictionary with specific related
keys to be used to defer usable data to the web client.
Task-ID: 2243913
PR odoo/odoo#69380
The import logging (ish) assumes that if an exception has at least 2
args the second arg is metadata added by the callee.
As it turns out, `UnicodeEncodeError` has *five* arguments, none of
which is added by us. So if encoding something fails during the
process (e.g. because the file contains a lone surrogate, which leads
to the database insert failing when psycopg2 tries to encode the query
to UTF8), then the `_log` function itself will fail, yielding a very
unhelpful error of:
dictionary update sequence element #0 has length 1; 2 is required
(because we tried to update a dict using a string).
This issue occurs only during *field conversion* and most fields have
no need to interact with the database (so don't need to encode the
value, which is what fails), however it is a problem when the invalid
string is used as a record name to look for (e.g. an m2o).
Further improve the experience by converting the UnicodeEncodeError to
a ValueError using the stringified UEE: `_log` assumes the first
argument to the exception is an error message of some sort, but for
UnicodeError subclasses it's just the encoding involved in the
error (here `utf-8`), which doesn't really serve as an error message.
Stringifying the exception generates a complete error message which is
quite a bit more helpful.
Specific update notes:
* avoid modifying the exception in-place, doesn't seem useful
* not sure why `field_name` was added as part of the augmentation
rather than up-front when `record` is created
Issue 2480064
closesodoo/odoo#72517
X-original-commit: 6c3c500929cd463cd3a1749f4ede8f3a8afd5748
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The method read_combined() has been deprecated in favor of explicit
calls to read() and get_combined_arch().
There are places where we call _get_combined_arch() instead, in order to
avoid parsing the XML that has just been serialized. This saves useless
serialization-deserialization.
Task: 2541577
Meta task: 2463632
Co-authored-by: Fabien Pinckaers <fp@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Before this commit, `get_base_url()` could not be called on an empty recordset.
It might be called on an empty recordset for instance when getting the paypal
payment endpoint URLs.
This commit allows that and returns the ICP value in that case.
The goal is to always use the util method `get_base_url()` instead of directly
accessing the ICP.
closesodoo/odoo#68201
Related: odoo/enterprise#17538
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Description of the issue/feature this PR addresses:
It is currently quite difficult to differentiate users. Most of the time, people
don't take the time to upload an actual avatar so everybody looks the same. This
PR generates a custom avatar with the users initials and random color to
differentiate them. For res.users, res.partner and hr.employee, image fields now
hold the binary image and avatar are used to show the image or svg.
Current behavior before PR:
Avatar had only random colors and was being saved in database, being inefficient
Desired behavior after PR is merged:
A new mixin defines image fields and in case no image is set, it generates an
SVG image with the user's initials and random color.
closesodoo/odoo#69819
Task: 2404630
Related: odoo/enterprise#18199
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
The method filtered_domain() is broken for domains with hierarchical
terms ('child_of'/'parent_of').
To see *one* of the ways the implementation is broken, let `A` be a
model with `parent_id` pointing to `A`, and `a1` a record of model `A`
without parent (`a1.parent_id` is `False`), then this fails:
assert a1 in a1.filtered_domain([("parent_id", "child_of", a1.id)])
The reason it fails is that on
https://github.com/odoo/odoo/blob/f5519586d214a9b34ad24683a7f97c47802a3bad/odoo/models.py#L5377-L5380
`data` is empty since `a1` has no parent, thus
https://github.com/odoo/odoo/blob/f5519586d214a9b34ad24683a7f97c47802a3bad/odoo/models.py#L5403-L5404
fails, therefore the result of `filtered_domain` is empty.
Note: the implementation of the hierarchical operators is full of quirks
that are hard to emulate otherwise than by reusing the original code.
As a consequence, the current implementation may be broken in more than
one way.
Let's see another way the implementation is broken: let `B` be a model
without a `parent_id` field and with a `friend_id` field pointing
to `B`, and let `b1` be a record of model `B`. Then
b1.filtered_domain([("friend_id", "child_of", b1.id)])
throws an exception of the form shown below:
ValueError: Invalid field 'parent_id' in leaf "<osv.ExtendedLeaf: ('parent_id', 'child_of', 1) ...
Meanwhile the following code is still valid and returs b1:
B.search([("friend_id", "child_of", b1.id)])
closesodoo/odoo#71237
X-original-commit: e7a5ba95d8b7df5bbf545ef8afe0a1f5d0f70272
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Use `frozendict` for dicts that often remain empty on models, like
_inherits and _depends, and also make `frozendict` more compact in
memory.
On a registry with 296 modules, this saves 635 kilobytes of memory,
which is about 6% of the registry's memory footprint.
closesodoo/odoo#70402
Related: odoo/enterprise#18142
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This makes the union of many dicts into a single dict.
On a registry with 296 modules, this saves 300 kilobytes of memory,
which is about 3% of the registry's memory footprint.
This mainly keeps field.related's type consistent.
On a registry with 296 modules, this saves 325 kilobytes of memory,
which is about 3% of the registry's memory footprint.
This avoids a call to __init__() which no longer has access to the
model's class' parent classes, and was not even correct...
closesodoo/odoo#70400
Related: odoo/enterprise#18141
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The base classes of models in the registry are modified when models are
extended by inheritance. But modifying cls.__bases__ is a costly
operation, so we manage to do it once per model when loading a registry.
Our benchmark consists in loading a registry with 296 modules. The net
time to load the registry is 25% smaller. In other words, the registry
loads 25% faster. Note that the time does not include the time to load
modules themselves, which does not change.
Have in a purchase order a first row with unit price 0
Export the PO adding unit price in the list of fields to export
The unit price is not reported
This occur because the first line of the order lines is meld with the
purchase order line but in the process the 0.0 values is discarded
opw-2510917
closesodoo/odoo#70368
X-original-commit: 242bf5024ce6ae9c32176cd1a65a4f57a242dd45
Signed-off-by: agr-odoo <agr-odoo@users.noreply.github.com>
Also speed up the basic setup of fields that are always duplicated
top-level (on the model's registry class). This saves time and memory,
as we also discard field.args and field._base_fields on toplevel fields
(those values are no longer useful after setup).
The attributes field.depends and field.depends_context are problematic
for sharing fields across registries, because they depend on the model's
registry class, which may vary from one registry to another. In order
to make computed fields shareable, we have to move those values away
from fields.
For the same reason, field.recursive should not be inferred, because its
value may depend on the registry, although it is generally not the case.
Moreover, the flag recursive=True is set on a field when field triggers
are determined (on the registry). A compute method may be called before
the flag is set (if no update has been done yet), and that can lead to
incorrect computations.
This happened in test TestUsers2.test_reified_groups in module 'base'.
The user groups view was apparently determined without the flag being
set, and the view depends on the recursive field 'trans_implied_ids',
which was not correctly computed.
We thus force developers to be explicit about recursive computed fields.
The code now logs a warning when the flag is not set up properly.
The basic setup of fields is now made in Field.__set_name__(), which
automatically makes this information shared across registries, because
it is done when importing Odoo modules.
Fields are no longer retrieved with inspect.getmembers(), which is
time-consuming. Instead, the classes defining models automatically
collect their own fields (via Field.__set_name__()), and the field
retrieval is done by using those collections. Also, a field no longer
needs to introspect its model class to find its field definitions.
A class that defines a model automatically determines its '_name',
'_inherit' and '_module' upon creation, and the fields determine their
'name', 'model_name' and '_module' via Field.__set_name__().
Loading a registry with 290 modules from scratch is now 25% faster.
Magic and inherited fields are not really useful on abstract models.
The _inherits specification is used anyway by models that inherit from
those abstract models.
The main goal of this change is to prepare a refactoring of models where
fields are no longer duplicated on the registry classes, but fields
defined on classes are used directly. But this new design cannot be
applied to all fields: a field being overridden simply cannot be used
directly. This branch improves the situation by avoiding unnecessary
field overridings.
closesodoo/odoo#69372
Related: odoo/upgrade#2409
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
With this commit, it is now possible to group the records of a model by
a Many2many field of said model.
The result of a read_group grouped by a m2m will return as many entries
or "groups" as there are different records in the comodel that are
linked to the model through the m2m, plus a null/false group, for records
of the model that have no linked records of the comodel or for records
of the comodel for which the current user has no read access to due to
ir.rules.
For a more illustrated explanation, the tests should cover all cases in
detail.
Note that this commit only introduces this change at the ORM level, the
frontend does not yet handle grouping by m2m fields but it is planned
in the near future.
Task-2428971
closesodoo/odoo#68958
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
CRM install query count can vary from one execution to another, leading
to difficulties when analysing performances evolution.
The main reason for this is that some compute methods were called in
different order. Even if compute order shouldn't have any effect on the
final result, making it well defined will help finding other causes of
non-determinism.
The initial observation was that sorting Environment.fields_to_compute
leads to a fixed number of query when installing crm.
The main cause of non-determinisim is the usage of `set` impacting
Field.compute_value and BaseModel._modified_triggers.
Transforming all these `set` to `OrderedSet` solves the problem.
The query count is now deterministic when installing a database from
scratch, but not when updating a database with -i crm.
OrderedSet is also slightly optimised by using a dict instead of an
closesodoo/odoo#68692
Ordereddict: dict order is deterministic since python3.6
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Purpose
=======
Propose an import tool that is more intuitive and allows more import options to
ease the import.
Specifications
==============
Update upload import file screen
--------------------------------
- Relabel import welcome screen
- relabel the 'load file' button to 'upload file'
- relabel the first (bold) line of the helper to 'Upload an Excel or CSV
file to import'
- Move all import options in a left panel
- Invert rows and columns in the mapping view
- One row for each column (header) to map in the imported file
- Columns: File Column / Odoo Field / If a match cannot be found / Comments
Left panel details
------------------
The sidepanel to the left of the mapping view is composed as followed:
- Imported File
- Import file name
- Sheet: Dropdown to select the sheet in the file if there are more than 1.
- 'use first row as header' checkbox: like the existing option, defines
whether the first row of the file should be considered as the header.
True by default. If False, display the first value only under 'File Column'
(see above)
- Formatting: only available if the file is a .csv
- list every option that already existed in previous import implementation.
- Batch Import
- is only available in debug moide and if the file exceeds the batch limit
- Batch limit : 2000
- Allow to define the threshold and the batch size.
- Help
- Download Template link
- Go to FAQ link
- Advanced
- Track history during import. (same as existing functionallity)
- Show Fields of relation fields. (same as existing functionallity)
Columns details
---------------
- File Column:
- Contains the column headers of the import file and the first non-empty
value of the column as a 'subtitle' (in grey italic)
- If 'use first row as header' is False, only display the first column value
(without the grey italic)
- Odoo Field:
- Contains dropdowns to the fields of the current model
- Required fields are diplayed in bold in the dropdown
- Icon: In the front of the field label, the field type (char, many2one,..)
is indicated by an icon.
- Tooltip: when hovering a field, display a tooltip with the following
information: field label, technical name, type, related model (if any)
- Placeholder: If no field is selected, display the placeholder
'To import, select a field...' in text-warning bold (i.e. orange)
- Allow clear: there is a close (fa-times) icon at the end of the dropdown
body to remove the Odoo field (and prevent import of this column)
- Multi Mapping: when setting a field already matched to another column,
unset it from the 'old' column (NB: exception made for char/text fields)
- Comments:
- Contains feedback from the system to the user, either import
errors/warnings or any additional details (multi mapping comments,...)
- For many2many field, display in related comment cell the following:
"To import multiple values, separate them by a comma".
- If there is an import error for a specific field, the error will be
displayed in the comment cell of the related field.
- If there is a mapping error, mapping options are displayed under the
error div to let the user choose the best option to fix that error.
Import errors management
------------------------
- Errors/warnings that can be matched to a specific field are displayed as
alert-danger/warning in the corresponding 'Comments' cell of the field.
- For 'no match found' errors, if X values couldn't be found,
display an unique error box will all the errors.
- Values beyond the first three are folded under a 'More' button.
- No changes on the 'See possible values' button.
- If an unmatched value is present in one row only, display :
'<value> at row X (<name of the row if the name field is matched>)'
- If an unmatched value is multiple rows, display:
'<value> at multiple rows'
- Errors/warnings/infos that cannot be matched to specific field are displayed
above the mapping listview. Global errors are not regrouped by error types to
avoid too much code complexy just to handle the rare times where multiple errors
of same types cannot be linked to a specific mapped field.
BaseImportError class have been introduced to ease the formatting of the various
exceptions that can occur during the import.
After testing the import, display the following above the mapping table:
- No warning/error:
'Everything seems valid' (alert-info)
- At least one error:
'The file contains blocking errors (see below)' (alert-error)
- At least one warning but no error:
'The file contains non-blocking warning (see below)' (alert-warning)
Mapping options
---------------
Mapping options are only displayed after testing or importing the file,
if there are import errors. The goal is to have a clean interface and to guide
the user step by step. The import process can therefore be a bit longer as it
needs to import -> choose solution for errors -> re-import but that is easier
for users to learn and understand this reworked import tool.
Possible values when a value cannot be matched:
- For many2one / many2many fields:
- Prevent import: (selected by default) not finding a match is blocking the
import
- Skip unknown values: values that cannot be matched will be skipped.
(hidden if the field is required)
- Create new values: Create records for values that cannot be matched
- For selection fields:
- Prevent import (selected by default)
- Skip unknown values: (hidden if the field is required)
- Set to <first value>: if cannot be matched, set it to <first value>
- Set to <second value>
- Set to <third value>
- etc.
- For boolean fields:
- Prevent Import (selected by default)
- Set to True
- Set to False
Note: With this rework, boolean warnings where the system assumes the
replacement value in case of matching error is removed and is replaced by a
blocking error. The user now has to choose the value to set.
"Prevent import" is the default behaviour when testing or importing.
Automatic mapping proposal
---------------------------
When loading a file, an automated mapping is directly proposed to the user,
based on word distance (see below), and on mapping created on previous
imports.
- Priority is given for mapping created on previous imports, skip fuzzy mapping.
- a distance of -1 is used to ensure priority during duplicates removal .
- In case multiple headers are mapped on the same field, if the mapped field
is already taken by another header, use fuzzy mapping instead.
- If no previous mapping for that header on that model, try an exact match on
every field id and field name of the model. (distance = 0)
- If no match is found, fuzzy mapping is applied (word distance).
The fuzzy mapping is executed only on the most likely fields (see below).
- Remove duplicates: keep the header-field couple that has the smallest
distance. In case of equality, keep the first.
Automatic mapping is therefore optimised for previous mapping or exact match,
as fuzzy mapping requires heaviest treatment. This is intended to prioritise the
import of files based on import templates.
Most likely fields
------------------
When parsing the import file, each header is analysed to guess what type of data
the column contains. For example, ff the column contains float, we suppose that
that header will most likely be matched on float or monetary fields.
The most likely fields are a subset of the model's fields that match the
header types.
Most likely fields are used for the fuzzy mapping, to propose the user a field
mapping based on the header types, if an exact match could not be found.
Most likely fields are also listed under "Suggested fields" in the mapping
dropdown in "Odoo Field" column of the import tool.
For now on, every fields (no mather their type) can be matched to any header,
but we prioritise the most likely fields. This way, we don't constrain the
mapping possibility based on what we suppose the user would do, but we instead
guide suggest the user the most likely mapping solutions.
Word distance mapping
----------------------------
In order to improve the mapping configuration, if an exact match cannot be found
between the file column and one of the odoo field:
- Use Word distance:
Word distance return a indice between 0 and 1.
- 0: exact match
- 1: completely different
- A: First try on field['name']
- B: Then on field['string']
- Keep the minimal distance between A and B for each odoo_field
- C: Keep the field that has the minimal distance.
- Match the column to the field by default if C['distance'] < 0.3.
Note: 0.3 has been chosen to ensure proximity but still having a little
error margin.
Multi mapping
-------------
When multiple file columns are matched to the same char/text/many2many field,
display the following alert-info box in the "Comments" column of the related
fields:
"Those columns will be concatenated in field <field label>"
Multi mapping rule :
- If it is a char field, separate the concatenated values by a space
- If it is a text field, separate the concatenated values by a line break
- If it is a many2many field, separate the concatenated values by a comma
Various improvements
--------------------
- During Import (or Test), the first waiting message displayed has been modified
to 'Importing...' or 'Testing...'. The progress (x record imported/tested) is
only displayed after the first batch of record have been imported/tested.
- Ease matching for Selection field: use case insensitive comparaison instead
of exact match.
- Add filename to import wizard.
- Add placeholder to search input of field mapping dropdown.
Tests have been adapted accordingly.
Links
=====
Task ID: 2352241
closesodoo/odoo#61948
Related: odoo/enterprise#17246
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Commit 6a0028f91944b1d9e4eac026e86ca249ef5bc7ee introduced a change that
allowed field_triggers to be computed lazily per registry, this change
introduced a behavioral change that is not easy to notice, in order to
explain the behavioral change I will use the following example which was
the original bug reported:
- Install studio
- Create an app (create a new custom model M)
- Uninstall studio
-> Uninstall fails because field display_name of the custom model M
depends on field M.x_name which has been removed due to the
uninstall process.
To understand why it didn't happen before the aforementioned commit, we
must first understand what happens with said commit applied:
- We trigger the uninstall of studio, this triggers the uninstall of
any modules that depend on it, namely studio_customizations which is
the module in which all customizations done with studio live in.
- We gather all data belonging to the studio_customization and we
start deleting in the following order: ...,
ir.model.fields.selection, ir.model.fields, ..., ir.model
- During the unlink process, we first remove the actual fields from
the model instances before deleting their database reflections, this
is done in the ir.model.fields._drop_column() method, it is this
method that will delete the x_name field but **not** the
display_name field, since it is a base field and not a custom one.
- After the deletion of the fields in memory, we call modified() to
mark fields that might've depended on the fields we just modified so
that they can be recomputed later.
- The call to modified will in turn access field_triggers, but since
we're in a new registry and field_triggers is a lazy property, it
will be computed right at this moment, this means that it will call
resolve_depends on the display_name field which still exists, and
this field has a dependency on the x_name field that we just
deleted! This is what will trigger the crash.
With that context, we can now understand how it didn't crash before
commit 6a0028f91944b1d9e4eac026e86ca249ef5bc7ee:
- Before the aforementioned commit, the field_triggers attribute was
computed during the registry's setup_models(), in the case of an
uninstall this call to setup_models was done way before the
uninstall step of the registry (Step 3 is the last to call
setup_models before Step 5).
- This means that the old behavior was technically a bug, because
right after removing x_name from the model, the field_triggers still
contained a dependency from display_name to x_name, the former being
no-longer present in memory.
With this commit, we simply reset the _rec_name and the dependencies of
the display_name if the x_name field is being removed, this ensures that
the computation of field_triggers won't crash and burn.
opw-2452498
opw-2478589
closesodoo/odoo#67823
X-original-commit: e6d22a43ff9f40e5fc7b8c84cd4dfe43f926d872
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
The "TypeError: Mixing apples and oranges" error message is raised when
users are using recordsets from different models together. The error
message was confusing some users thus have been reworded to be more
explicit.
The initial proposal was to replace "apples" and "oranges" by "torchons"
and "serviettes" but it was rejected because the French are not capable
not to mixup the two. They suggested to instead replace "apples" and
"oranges" by "pain au chocolat" and "chocolatine", this was rejected
because none of us understood the difference between the two, they both
are couques.
closesodoo/odoo#62266
Task: 2366612
Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
When a record is written on a computed field with an inverse method and
an x2many field wit a delete command, the computed field is not properly
handled by the inverse method, because of a whole cache invalidation
that happens during the unlink.
With this commit, we write in cache on the fields that have been
invalidated before invoking the inverse method.
opw #2439861closesodoo/odoo#67525
X-original-commit: d39855bad5153654d443d1b7ede3907f8bae3e66
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Writing on x2many field should be done last, because deleting x2many
lines causes pending computations and updates to be flushed. Writing on
column fields after that inevitably adds extra update queries.
We introduce an attribute `write_sequence` on fields to order fields for
write. The prescribed order is: all fields except monetary and x2many,
monetary fields, x2many fields.
closesodoo/odoo#65959
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
The goal of this change is to simplify the code managing `create_date`
and `write_date` in methods `create()` and `write()`, and also to remove
weird behaviors caused by the way those fields were updated.
Assume we update a simple field on a record. This adds pending updates
for the field and `write_date`. However, the value of `write_date` is
not known yet: it will be updated as `NOW() AT TIME ZONE 'UTC'` in SQL.
So `write_date` is actually given a dummy value in pending updates, and
it is invalidated from cache, until its value is flushed to the database
and fetched again.
Now assume we access another field on the record, and that field is not
in cache. The prefetching mechanism will read all column fields,
including `write_date`, and flush them first.
# this adds pending updates foo: 42, write_uid: 1, write_date: False
record.foo = 42
# assume 'bar' is not in cache; this prefetches all column fields,
# which flushes the pending updates above before reading them back
result = record.bar
We can avoid flushing pending updates if the values read from database
do not overwrite existing values in cache. If you assume that the value
of a pending update is in cache (in the example, `foo: 42`), you don't
need to flush the corresponding field. Indeed, the value of `foo` will
remain 42 in cache, whatever its value in the database. This assumption
(pending updates are in cache) is true for all fields *except* for
`write_date`: it is invalidated from cache, and given a dummy value in
pending updates. This branch actually makes this assumption true for
all fields. The avoidance of flushing pending updates will be done in
another commit.
In order to directly assign `write_date` its value, we use a cache for
the value `NOW() AT TIME ZONE 'UTC'` from the database. This costs at
most one query per transaction, and potentially saves a few queries.
Co-authored-by: Victor Feyens <vfe@odoo.com>
Rémy Voet (ryv)