Commit Graph
973 Commits
Author SHA1 Message Date
ef00294e71 [IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation.  Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table.  The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages).  Empty text is allowed in translation values, but not NULL.

Here are examples for a field with translate=True:

    NULL
    {"en_US": "Foo"}
    {"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
    {"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}

Like before, writing False to the field makes it NULL, i.e., False in all
languages.  However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.

Here are examples for a field with translate=xml_translate:

    NULL
    {"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}

Change for callable(translate) fields: one can now write any value in any
language on such a field.  The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values.  Basically the
structure of the value must remain the same in all languages, like before.

Reading a translated field is now both simpler and faster than the former
implementation.  We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:

    SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...

The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages).  For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.

Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language.  Before this change, only the source value of the
field ('en_US') could be indexed.

Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages.  We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.

Code translations are no longer stored into the database.  They become static,
and are extracted from the PO files when needed.  The worker simply uses a cache
with extracted code translations for performance.  This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker.  Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).

Performance summary:
 (+) reading 'model' translated fields is faster
 (+) reading 'model_terms' translated fields is much faster (no need to inject
     translations into the source value)
 (+) searching translated fields with operator 'ilike' is much faster when the
     field is indexed with 'trigram'
 (+) updating translated fields requires less ORM flushing
 (-) importing translations from PO files is 2x slower

Some extra fixes:
 - make field 'name' of ir.actions.actions translated; because of the PG
   inheritance, this is necessary to make the column definition consistent in
   all models that inherit from ir.actions.actions.
 - add some backend API for the web/website client for editing translations
 - move methods get_field_string() to model ir.model.fields
 - move _load_module_terms to model ir.module.module
 - adapt tests in test_impex, test_new_api
 - because env.lang is injected into SQL queries, its returned value is
   now guaranteed to correspond to a valid active language or None
 - remove wizard to insert missing translations (no longer makes sense)

task-id: 2081307

Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-09-15 22:37:50 +02:00
Gorash 39ea7a1fab [IMP] web/all: XML templates are now declared into the python manifest.
Adapt all manifest, split some XML file and update JavaScript files.

Part-of: odoo/odoo#95500
2022-09-14 20:25:01 +02:00
Aaron Bohy 37f9c31231 [FIX] base_import,web: do not allow import if create="0"
This commit is three-fold:
 1) we now honnor attributes `create` and `import` which can be
    set to a falsy value in archs (and in these cases, we disable
    the import feature)
 2) we converted the base_import test suite to use the owl views
 3) we whitelisted the base_import addon for the lint/prettier
    script

closes odoo/odoo#100026

Signed-off-by: Géry Debongnie <ged@odoo.com>
2022-09-14 01:52:47 +02:00
Stanislas Sobieski d76e28b43b [FIX] base_import: fix test
Test was skipped on runbot since odf was missing

closes odoo/odoo#99282

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2022-09-01 19:17:11 +02:00
aliya 514ff773a2 [IMP] account, base_import: improve accounting import
- Remove FEC from res.config.settings - everything is now linked in the import guide (in enterprise)
- Compute debit and credit from opening_balance
- Update import templates for account.account, account.move, res.partner
- Small UI changes in base_import: add an action title and improve the template button design

task-2888243

closes odoo/odoo#96291

Related: odoo/enterprise#29627
Related: odoo/upgrade#3794
Signed-off-by: Cedric Snauwaert <csn@odoo.com>
2022-08-26 17:16:52 +02:00
Romeo Fragomeli 1fcd098af5 [REF] *: BS5: migration
Automated change made by a lot of RegEx to change all think that is
possible to automate.

https://getbootstrap.com/docs/5.1/migration

Task ID: 2766483

Part-of: odoo/odoo#95450
2022-07-07 13:30:24 +02:00
Romeo Fragomeli 9b5c9484fe [REF] *: BS5: adapt changes in media query mixin
> Media query mixins parameters have changed for a more logical approach

> media-breakpoint-down() uses the breakpoint itself instead of the next
> breakpoint (e.g., media-breakpoint-down(lg) instead of
> media-breakpoint-down(md) targets viewports smaller than lg).
> Similarly, the second parameter in media-breakpoint-between() also
> uses the breakpoint itself instead of the next breakpoint (e.g.,
> media-between(sm, lg) instead of media-breakpoint-between(sm, md)
> targets viewports between sm and lg).

https://getbootstrap.com/docs/5.1/migration/#sass

Task ID: 2766483

Part-of: odoo/odoo#95450
2022-07-07 13:30:24 +02:00
Lucas Perais (lpe) 2cafc7be88 [REF] web, base, website_sale_loyalty: remove unnecessary flags on action
action act_window can have a "flags" field which contains tweaking parameter for the views.
A little inventory:
- ation_buttons: if true displays on List and Kanban the buttons that trigger action
  in the control-panel bottom left area. It is true by default in JS.
- withControlPanel: whether to display the ControlPanel as a whole. True by default.
- search_view: not used or dealt with.
- mode ("readonly"|"edit") whether to initiate a form view in that mode.
  (used and practical -- but will be outdated when the edit mode by default is implemented)

This commit removes the occurences of those flags keys that are useless

closes odoo/odoo#94078

Related: odoo/enterprise#28620
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2022-06-22 14:28:43 +02:00
Géry Debongnie 87ed548bc9 [FIX] base_import: properly display controlpanel buttons
Problem: open any (multi record) view, open the "favorites" menu and
select the import action. The import client action now opens, but no
button is visible in the control panel.

Since a recent owl update, the import client action did not display its
control panel buttons (upload file, cancel, ...). This is caused by an
unfortunate situation: the control panel buttons are rendered, but after
the control panel was rendered, so owl does not add them in the DOM.  It
worked by chance before: the props were completely reactive, and the
code actually modifies in place some object received in props. So, a
render was forced on the controlpanel, which means that it properly
picked up the buttons.

Now, this render was accidental, so the new owl code is actually
correct. This commit fixes the issue by simply rendering the buttons
before the controlpanel is even rendered, so it now works as expected.

closes odoo/odoo#92810

X-original-commit: f185f543452abf65b9948e907aeb45e918cf3bed
Signed-off-by: Lucas Perais (lpe) <lpe@odoo.com>
Signed-off-by: Géry Debongnie <ged@odoo.com>
2022-06-02 21:45:27 +02:00
Adrien Minet 8b6eecc8f1 [FIX] base_import: read error message clearer for import
How to reproduce the bug ?

- install the inventory app
- open the app and try to import a file where at least one cell has a
wrong value

What is the bug ?

When you try to import a file in the inventory app where at least one
cell has a wrong value, the file won't be imported. In addition to that,
you will get an error saying that one python module is missing while
this might not be the case.

opw-2715625

closes odoo/odoo#91482

X-original-commit: cafd8699ff8ec8eb93aa283a4cac57317ce40708
Signed-off-by: Adrien Minet <admi@odoo.com>
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
Signed-off-by: Simon Goffin <sig@odoo.com>
2022-05-17 13:39:44 +02:00
John Laterre (jol) ff0275be6b [IMP] account: credit limit by partners
The basic idea of this functionality is for users
to be able to put limits on Partners that would
trigger non-blocking warnings when trying to confirm new:

- Sales Orders
- Invoices

when that partner has too many open invoices.

task-2722165

Part-of: odoo/odoo#83205
2022-04-01 18:49:46 +02:00
Jorge Pinna Puissant bc20eb4790 [IMP] test_lint, *: enable no-unused-vars rule
Part-of: odoo/odoo#85569
2022-03-02 13:05:42 +00:00
Michael (mcm) b81861233d [REF] *: use LegacyComponent
This commit replaces Component by LegacyComponent when the component
uses removed features from owl 1 like getting its `el` or `trigger` an event.
It also adds `useService("rpc")` when component needs it.

closes odoo/odoo#85389

Related: odoo/enterprise#24745
Signed-off-by: Géry Debongnie <ged@odoo.com>
2022-03-01 09:57:39 +00:00
Antoine Vandevenne (anv) adf70bf9dc [FIX] *: retarget documentation links to master
closes odoo/odoo#84990

X-original-commit: 39bdf46
Related: odoo/enterprise#24582
Signed-off-by: Antoine Vandevenne (anv) <anv@odoo.com>
2022-02-21 17:01:02 +00:00
+2 318cdcc0b8 [REF] *: adapt code to owl 2
Owl 2 changelog: https://github.com/odoo/owl/blob/a9f29c4caad4f32d06be1ec4780572825781cd9b/CHANGELOG.md

Part-of: odoo/odoo#80156
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Géry Debongnie <ged@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Simon Genin (ges) <ges@odoo.com>
Co-authored-by: Francois (fge) <fge@odoo.com>
Co-authored-by: Michael Mattiello (mcm) <mcm@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: luvi <luvi@odoo.com>
Co-authored-by: Lucas Perais (lpe) <lpe@odoo.com>
Co-authored-by: Jorge Pinna Puissant <jpp@odoo.com>
2022-02-10 07:46:13 +00:00
Hubert Van de Walle (huvw) d72754bbae [FIX] base_import: allow importing a new file after one has been imported
Steps to follow:

  - Go to the Accounting Dashboard
  - Click on Bank > Import statements
  - Follow the flow to import a file
  - Once redirected to the reconciliation, go back to the import from
    the breadcrumbs
  - Load a new file

Cause of the issue

  The state machine used to handle events didn't allow going from
  imported to filed_loaded

opw-2706505

closes odoo/odoo#83219

X-original-commit: a19a0bdca369c9038a7a34363a1d9b7ee06f899d
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Hubert Van De Walle <huvw@odoo.com>
2022-01-22 14:13:41 +00:00
Martin Trigaux a8e50921af [FIX] *: correct typos and English errors
closes odoo/odoo#80181

X-original-commit: efd178daee689192d4e930a075475587038b3e0d
Related: odoo/enterprise#22439
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-11-22 14:48:04 +00:00
awa-odoo 7c96f286db [IMP] base_import: add a test for the multi-mapping feature
This commit simply adds a small test for the import multi-mapping feature that
allows the end-user to map multiple columns of his import file to the same
field.

The values in the files are merged together, meaning concatenated for Char/Text
fields and accumulated for Many2many fields.

This commit is done as a preliminary step before fixing an issue related
to this feature in v15.

Task-2687407

closes odoo/odoo#79913

X-original-commit: ac832586e68e0873d69c0b10a3f8cc8b08ea506e
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-11-17 10:44:32 +00:00
std-odoo 8125165b00 [FIX] base_import: improve the import speed
Purpose
=======
Since v15, the time to import records in Odoo is much slower than in
v14. With a profiling, we know that the cause is `fields_get`. Indeed,
this method is heavy and call many times (#rows * #columns * ~#batches).

So the import time (in seconds) for ~1000 leads in CRM is;

Batch size | v14 | v15 | v15 fix
--------------------------------
2000       |   7 |  20 |      14
 200       |  11 |  48 |      15
  20       |  17 | 423 |      23

We can see that just by calling `._fields.get` instead of `fields_get`
we gain a lot of time. The reason for that is `fields_get` just read in
a python dictionary (which is really really fast), while `fields_get`
checks the access right, get the description of each fields (which
might make SQL queries to retrieve translation, etc).

Task-2687407

X-original-commit: 96e72a8b1657df00d5ed7ca6db7bd99e352d7096
Part-of: odoo/odoo#79913
2021-11-17 10:44:32 +00:00
Victor Feyens ab022ec12d [FIX] *: target v15.0 documentation with doc links
X-original-commit: acc95ec204baa1dddbe292c379a1768fe1deccbf
Part-of: odoo/odoo#77923
2021-10-07 17:59:52 +00:00
Julien Mougenot 53f9b5f235 [REF] *: adapt modules to action and view contexts
This commit handles the adaptation of any component using the search
model's 'action' and/or 'view' key, now replaced by the environment
keys 'actionContext' and 'viewContext'.

Affected modules are:
- base_import
- board
- crm
- google_spreadsheet
- project

X-original-commit: 40f1ae87e1436192a70a8a7432eb72d379cfa6dc
Part-of: odoo/odoo#77463
2021-09-30 10:38:30 +00:00
36988d0edc [IMP] *: use new dropdown in legacy too
This commit replaces the control panel dropdowns with the new <Dropdown/> component, in order to get consistent through the new/legacy views (because the current Odoo version is in a state where some views uses the new infrastructure and some others are still not converted - see odoo/odoo#73311).

The diff seems massive, but it is mostly due to tests adaptations.

closes odoo/odoo#77001

X-original-commit: d679cd0d8ba9a2420e81a42a698763e9be2327e1
Related: odoo/enterprise#21077
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
2021-09-23 07:17:43 +00:00
Martin Trigaux ef8ad324b0 [I18N] *: export 15.0 source terms
closes odoo/odoo#76542

X-original-commit: 63e6807437295519a0f4705fb88644d6d557ca3a
Related: odoo/enterprise#20882
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-09-16 07:17:40 +00:00
aeb8e49972 [IMP] web,*: view infrastructure
We refactor the action service and allow the legacy and new views to share
a global state within a same action.

We introduce two new components View and WithSearch along with
the search infrastructure: ControlPanel, FilterMenu,...

Part-of: odoo/odoo#73311
Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Géry Debongnie <ged@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Michaël Mattiello <mcm@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Jorge Pinna Puissant <jpp@odoo.com>
Co-authored-by: Stefano Rigano <sri@odoo.com>
2021-09-07 10:04:35 +00:00
Julien Mougenot add86e4abd [REF] base_import: Move legacy files
This commit moves the content of the static base_import files to a
dedicated `legacy` folder to mark the transition with the new Owl
environment.

Part-of: odoo/odoo#73311
2021-09-07 10:04:35 +00:00
Nikunj LadavaandMohammed Shekha 662ecf4e20 [IMP] base_import: improve import wizard preview
PURPOSE

Improve import wizard as followup of recent improvements.

SPECIFICATIONS

Preview lines

  - make the preview lines more visible
    - set font-weight: 500 on o_import_header_name
    - remove the text-muted class from the data preview line

  - allow the user to preview the first 5 non-empty values
    - when hovering on the data preview span, display a tooltip with the first
    5 non-empty values of the file column (with 'Preview' as the tooltip title)

  - In the 'when a value cannot be matched' dropdown:
    add a new 'skip record' option for the following field types: boolean,
    many2one, many2many, selection
    add a new 'set empty' for the following field types: many2one, many2many,
    selection hide this option if the field is required

When a value cannot be matched:
  the 'skip record' option will make sure that lines with an unmatched value
  will not be imported (i.e. skipped at import)
  the 'set empty' option will set the field value to False when the value
  cannot be matched

  - Restore colored background on alert boxes by removing some css rules

Task-2504343

closes odoo/odoo#73713

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Co-authored-by: Mohammed Shekha <msh@odoo.com>
2021-09-02 12:57:13 +00:00
David BeguinandAurélien Warnon 409ccc833d [IMP] base_import: add import progress bar for batch mode
This commit adds a progress bar that is displayed during the import (or import
test). It is only displayed when importing in batch mode.

The user can decide to stop the import. If it's the case, the current
batch is finalized and the import is interrupted afterwards.

The user can easily resume the import later starting from the line where the
batch was interrupted
e.g: interrupted after 300 / 500 lines, when you resume, it will start from
line 301.
(The above behavior is not applied in test mode, we never alter the starting
line when testing).

Side note: This commit also removes the alert-info box from batch import
disclaimer, for design beauty purpose.

Task-2567591

closes odoo/odoo#73243

Related: odoo/enterprise#19467
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Co-authored-by: Aurélien Warnon <awa@odoo.com>
2021-08-23 09:02:08 +00:00
Xavier-Do 288595f558 [FIX] *: add explicit license to all manifest
The license is missing in most enterprise manifest so
the decision was taken to make it explicit in all cases.
When not defined, a warning will be triggered starting from
14.0 when falling back on the default LGPL-3.

closes odoo/odoo#74245

Related: odoo/design-themes#48
Related: odoo/enterprise#19862
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-07-26 13:09:57 +00:00
Martin Trigaux 6758868731 [I18N] *: export saas-14.4 source terms
Without demo data

closes odoo/odoo#73560

X-original-commit: 802e46541117573e028b711ea33dad9df9075a39
Related: odoo/enterprise#19602
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-07-12 10:57:37 +00:00
+1 14bffd983e [REF] *: adapt code to new owl webclient
This commit adapts the community codebase to the rewriting of the
/web application in owl.

Co-authored-by: Aaron Bohy <aab@odoo.com>
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Géry Debongnie <ged@odoo.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Simon Genin (ges) <ges@odoo.com>
Co-authored-by: Francois (fge) <fge@odoo.com>
Co-authored-by: Michael Mattiello (mcm) <mcm@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais (lpe) <lpe@odoo.com>
Co-authored-by: Jorge Pinna Puissant <jpp@odoo.com>
2021-06-18 21:31:27 +02:00
Sébastien Theys c6716847aa [IMP] web, *: clean up notification API
task-2476867

closes odoo/odoo#67009

Related: odoo/enterprise#16760
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2021-05-27 13:47:23 +00:00
Victor Feyens 0348b95aee [FIX] *: update documentation links
Following the recent reorganisation of the documentation in 12.0+,
the majority of the documents have been moved and their old links are no longer valid.
Some redirection rules will soon be deployed, but those rules might be dropped in some years
and we want the links to still work, which is why we still replace the links to the new ones.

FW-Port of odoo/odoo#70675 (13.0)

closes odoo/odoo#70920

X-original-commit: bc9c1eef538ba6095e74c19d5d9ed9e01625ec7c
Related: odoo/enterprise#18361
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2021-05-17 19:26:27 +00:00
Martin Trigaux 41d8b8cf68 [I18N] *: export saas-14.3 source terms
closes odoo/odoo#70673

X-original-commit: bcb9ff784e44462384b0a43a0a23eed7a1111bc5
Related: odoo/enterprise#18269
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-05-11 12:28:28 +00:00
Jorge Pinna PuissantandSamuel Degueldre 9a76719893 [FIX] *: undeclared variables and restricted globals
closes odoo/odoo#69988

Related: odoo/enterprise#17991
Signed-off-by: Géry Debongnie (ged) <ged@openerp.com>
Co-authored-by: Samuel Degueldre <sad@odoo.com>
2021-05-04 13:32:49 +00:00
David Beguin 49dc69cfea [IMP] base_import: fix field dropdown list return values
When generating field completion, if there are no o2m or suggested fields,
the method was returning the list of values without adding the regulars fields.

This commit returns correctly the complete set of values.

Task ID: 2508902

closes odoo/odoo#69423

X-original-commit: 134b0f28a43042b22bed1c5f657344544e0f8d22
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-04-16 16:54:52 +00:00
David Beguin c483f59b37 [REF] base_import: revamp import module
Purpose
=======

Propose an import tool that is more intuitive and allows more import options to
ease the import.

Specifications
==============

Update upload import file screen
--------------------------------

- Relabel import welcome screen
	- relabel the 'load file' button to 'upload file'
	- relabel the first (bold) line of the helper to 'Upload an Excel or CSV
	file to import'

- Move all import options in a left panel

- Invert rows and columns in the mapping view
	- One row for each column (header) to map in the imported file
	- Columns: File Column / Odoo Field / If a match cannot be found / Comments

Left panel details
------------------

The sidepanel to the left of the mapping view is composed as followed:
- Imported File
	- Import file name
	- Sheet: Dropdown to select the sheet in the file if there are more than 1.
	- 'use first row as header' checkbox: like the existing option, defines
	whether the first row of the file should be considered as the header.
	True by default. If False, display the first value only under 'File Column'
	(see above)
- Formatting: only available if the file is a .csv
	- list every option that already existed in previous import implementation.
- Batch Import
	- is only available in debug moide and if the file exceeds the batch limit
	- Batch limit : 2000
	- Allow to define the threshold and the batch size.
- Help
	- Download Template link
	- Go to FAQ link
- Advanced
	- Track history during import. (same as existing functionallity)
	- Show Fields of relation fields. (same as existing functionallity)

Columns details
---------------

- File Column:
	- Contains the column headers of the import file and the first non-empty
	value of the column as a 'subtitle' (in grey italic)
	- If 'use first row as header' is False, only display the first column value
	(without the grey italic)

- Odoo Field:
	- Contains dropdowns to the fields of the current model
	- Required fields are diplayed in bold in the dropdown
	- Icon: In the front of the field label, the field type (char, many2one,..)
	is indicated by an icon.
	- Tooltip: when hovering a field, display a tooltip with the following
	information: field label, technical name, type, related model (if any)
	- Placeholder: If no field is selected, display the placeholder
	'To import, select a field...' in text-warning bold (i.e. orange)
	- Allow clear: there is a close (fa-times) icon at the end of the dropdown
	body to remove the Odoo field (and prevent import of this column)
	- Multi Mapping: when setting a field already matched to another column,
	unset it from the 'old' column (NB: exception made for char/text fields)

- Comments:
	- Contains feedback from the system to the user, either import
	errors/warnings or any additional details (multi mapping comments,...)
	- For many2many field, display in related comment cell the following:
	"To import multiple values, separate them by a comma".
	- If there is an import error for a specific field, the error will be
	displayed in the comment cell of the related field.
	- If there is a mapping error, mapping options are displayed under the
	error div to let the user choose the best option to fix that error.

Import errors management
------------------------

- Errors/warnings that can be matched to a specific field are displayed as
alert-danger/warning in the corresponding 'Comments' cell of the field.
- For 'no match found' errors, if X values couldn't be found,
display an unique error box will all the errors.
	- Values beyond the first three are folded under a 'More' button.
	- No changes on the 'See possible values' button.
- If an unmatched value is present in one row only, display :
	'<value> at row X (<name of the row if the name field is matched>)'
- If an unmatched value is multiple rows, display:
	'<value> at multiple rows'
- Errors/warnings/infos that cannot be matched to specific field are displayed
above the mapping listview. Global errors are not regrouped by error types to
avoid too much code complexy just to handle the rare times where multiple errors
of same types cannot be linked to a specific mapped field.

BaseImportError class have been introduced to ease the formatting of the various
exceptions that can occur during the import.

After testing the import, display the following above the mapping table:
- No warning/error:
	'Everything seems valid' (alert-info)
- At least one error:
	'The file contains blocking errors (see below)' (alert-error)
- At least one warning but no error:
	'The file contains non-blocking warning (see below)' (alert-warning)

Mapping options
---------------

Mapping options are only displayed after testing or importing the file,
if there are import errors. The goal is to have a clean interface and to guide
the user step by step. The import process can therefore be a bit longer as it
needs to import -> choose solution for errors -> re-import but that is easier
for users to learn and understand this reworked import tool.

Possible values when a value cannot be matched:
- For many2one / many2many fields:
	- Prevent import: (selected by default) not finding a match is blocking the
	import
	- Skip unknown values: values that cannot be matched will be skipped.
	(hidden if the field is required)
	- Create new values: Create records for values that cannot be matched
- For selection fields:
	- Prevent import (selected by default)
	- Skip unknown values: (hidden if the field is required)
	- Set to <first value>: if cannot be matched, set it to <first value>
	- Set to <second value>
	- Set to <third value>
	- etc.
- For boolean fields:
	- Prevent Import (selected by default)
	- Set to True
	- Set to False
	Note: With this rework, boolean warnings where the system assumes the
	replacement value in case of matching error is removed and is replaced by a
	blocking error. The user now has to choose the value to set.

"Prevent import" is the default behaviour when testing or importing.

Automatic mapping proposal
---------------------------

When loading a file, an automated mapping is directly proposed to the user,
based on word distance (see below), and on mapping created on previous
imports.
- Priority is given for mapping created on previous imports, skip fuzzy mapping.
	- a distance of -1 is used to ensure priority during duplicates removal .
	- In case multiple headers are mapped on the same field, if the mapped field
	is already taken by another header, use fuzzy mapping instead.
- If no previous mapping for that header on that model, try an exact match on
every field id and field name of the model. (distance = 0)
- If no match is found, fuzzy mapping is applied (word distance).
The fuzzy mapping is executed only on the most likely fields (see below).
- Remove duplicates: keep the header-field couple that has the smallest
distance. In case of equality, keep the first.

Automatic mapping is therefore optimised for previous mapping or exact match,
as fuzzy mapping requires heaviest treatment. This is intended to prioritise the
import of files based on import templates.

Most likely fields
------------------

When parsing the import file, each header is analysed to guess what type of data
the column contains. For example, ff the column contains float, we suppose that
that header will most likely be matched on float or monetary fields.
The most likely fields are a subset of the model's fields that match the
header types.

Most likely fields are used for the fuzzy mapping, to propose the user a field
mapping based on the header types, if an exact match could not be found.

Most likely fields are also listed under "Suggested fields" in the mapping
dropdown in "Odoo Field" column of the import tool.
For now on, every fields (no mather their type) can be matched to any header,
but we prioritise the most likely fields. This way, we don't constrain the
mapping possibility based on what we suppose the user would do, but we instead
guide suggest the user the most likely mapping solutions.

Word distance mapping
----------------------------

In order to improve the mapping configuration, if an exact match cannot be found
between the file column and one of the odoo field:
- Use Word distance:
	Word distance return a indice between 0 and 1.
		- 0: exact match
		- 1: completely different
	- A: First try on field['name']
	- B: Then on field['string']
	- Keep the minimal distance between A and B for each odoo_field
	- C: Keep the field that has the minimal distance.
	- Match the column to the field by default if C['distance'] < 0.3.
	Note: 0.3 has been chosen to ensure proximity but still having a little
	error margin.

Multi mapping
-------------

When multiple file columns are matched to the same char/text/many2many field,
display the following alert-info box in the "Comments" column of the related
fields:

	"Those columns will be concatenated in field <field label>"

Multi mapping rule :
	- If it is a char field, separate the concatenated values by a space
	- If it is a text field, separate the concatenated values by a line break
	- If it is a many2many field, separate the concatenated values by a comma

Various improvements
--------------------

- During Import (or Test), the first waiting message displayed has been modified
to 'Importing...' or 'Testing...'. The progress (x record imported/tested) is
only displayed after the first batch of record have been imported/tested.
- Ease matching for Selection field: use case insensitive comparaison instead
of exact match.
- Add filename to import wizard.
- Add placeholder to search input of field mapping dropdown.

Tests have been adapted accordingly.

Links
=====

Task ID: 2352241

closes odoo/odoo#61948

Related: odoo/enterprise#17246
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2021-04-02 11:17:31 +00:00
Julien MougenotandSimon Genin 03641610c2 [REF] *: convert all modules to new asset system
Conversion of all modules to the new manifest assets declaration.

Part of task: 2352566

Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
2021-03-31 13:57:18 +02:00
Francois (fge)andSimon Genin d29666c04c [IMP] web, *: adapt some JS files to new module system
The previous work on adding support for native JS modules needs to adapt
some existing files, which have an incompatible name (with a '/').

Also, we convert a few JS file in /web to the native JS module system,
to show that it can work.

Part of PR 63177

Co-authored-by: Simon Genin (ges) <ges@odoo.com>
2021-02-15 10:18:11 +01:00
Adrien Dieudonne 89beb798ff [REF] *: o_button_import doesn't exist anymore
closes odoo/odoo#42082

Related: odoo/enterprise#15492
Signed-off-by: Adrien Dieudonné (adr) <adr@odoo.com>
2020-12-29 13:41:25 +00:00
Mohammed Shekha f7ddf3398c [FIX] base_import,web: remove Import from m2x dialogs
Before this commit, the "Import" button was available in the
favorite menus of dialogs opened from many2one and many2many field
widgets. It makes no sense from those dialogs.

Task 2376279

closes odoo/odoo#62079

Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2020-12-21 08:45:57 +00:00
alt-odoo 57ad950178 [FIX] base_import: allow import of dates in German format
The German date format is the following: DD.MM.YYYY but when importing
such value, it is wrongly considered as a float value due to the '.'
being confused with the thousands separator in the regexp of method
_remove_currency_symbol.

We should check both date/time and float/monetary formats, so it will
correctly identify the German date format as well as float values.

closes odoo/odoo#62205

X-original-commit: 1bdd0cc247d36c869ffdee7d501cb65d36be47a6
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Signed-off-by: Alex Tuyls <alt-odoo@users.noreply.github.com>
2020-11-23 16:46:24 +00:00
Xavier Morel 3cadee4170 [CHG] base_import: increase recursion limit to 3
As part of QoL PR #26267 the way the recursion limit was
managed (where it is checked) was changed, effectively decreasing the
recursion limit by 1 (to the originally expected limit of 2).

According to #29877 this is an inconvenient change / regression in
e.g. accounting context: if an object has journal entries, updating a
field of a journal item requires 3 levels of indirection (entries ->
items -> field). The same would occur if an object is e.g. linked to
multiple projects (projects -> tasks -> field) or SO (-> lines ->
field).

Therefore bump the recursion limit up to 3.

Closes #29877

closes odoo/odoo#61997

X-original-commit: 64ff41ce805c72eb700b45d55adfc7379de56862
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-11-19 10:38:05 +00:00
Swapnesh Shah b07f2d31d2 [FIX] *: update document links
Before this commit, links to the documentation were referenced the
previous version, 13.0, instead of the current one, 14.0.

Eventhough there is a redirection done by NGINX of a "versionless" URL
to the latest one (e.g. /documentation/user/general/auth/google.html
-> /documentation/user/14.0/general/auth/google.html as of today), the
goal is to keep links owrking for users that will still be using the
14.0 in three years (and should not endup on the 17.0 doc).

closes odoo/odoo#60228

X-original-commit: 7ac08486d91d0ff0151abeeda057ffa6beda72e8
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2020-10-19 07:03:01 +00:00
Martin Trigaux e79531c136 [I18N] *: export 14.0 source terms
Including demo data this time

closes odoo/odoo#58862

X-original-commit: 575abde110acb3d12b25f177a863374becef0894
Related: odoo/enterprise#13705
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2020-09-29 17:52:02 +00:00
Moisés López 9857b34fcc [FIX] base_import: mitigate concurrent update errors
Column mappings are updated in-place, if multiple users are importing records
of the same model at the same time, this will trigger concurrency errors.

This is made worse by the error only being reported on commit (after having
processed the entire import) and being retried automatically, so it slows down
the user and the entire system, the more concurrent imports the slower.

Log except:

    INFO dbname odoo.addons.base_import.models.base_import: done
    ERROR dbname odoo.sql_db: bad query: UPDATE "base_import_mapping" SET "field_name"='name',"write_uid"=%s,"write_date"=(now() at time zone 'UTC') WHERE id IN (%s)
    ERROR: could not serialize access due to concurrent update
    INFO dbname odoo.service.model: SERIALIZATION_FAILURE, retry 1/5 in 0.8720 sec...

closes odoo/odoo#57655

X-original-commit: 3f5e9ca625b53021c100245f5786bda4069974ad
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2020-09-14 13:40:19 +00:00
Hardik Prajapati dbe6b1b30d [FIX] base_import: remove uppercase under Favorites
this commit removes unnecessary uppercase used under the favorites menu

task - 2325684

closes odoo/odoo#57319

X-original-commit: 1d54564d280d23b506e3764294318a359c0ad681
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2020-09-09 07:30:47 +00:00
Martin Trigaux 90d85eb9c5 [I18N] export saas-13.5 source terms
Without demo data

closes odoo/odoo#56869

X-original-commit: 33f251b6489455cd7221f2c62dee0400a69784b8
Related: odoo/enterprise#12836
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2020-09-01 11:18:00 +00:00
Julien MougenotandMathieu Duckerts-Antoine bb3bc7c9e4 [IMP] *: adapt modules to Owl search panel
Adaptation of the rest of the environment to the changes regarding the
refactored search panel and model.

This commit includes the following changes:

- Since the control panel model is now part of the search model, the
view widget is responsible of its instantiation with the appropriate
API; creating an ActionModel holding the required model extensions.

- The controller is now importing/exporting the state of the entire
search model as well as the current state of the search panel (given
throught its props). Changes have been made to adapt to this behaviour.

- The multiple components linked to the control panel model have been
adapted to reflect the changes brought to it (they will now listen to
the "searchModel" instead of the "controlPanelModel").

- The `test_utils_create` methods have also been adapted to properly
instantiate a searchModel

Part of task 2228968

Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
2020-07-23 08:21:10 +00:00
jvm-odoo 5a60dbccd4 [FIX] base_import: fix re-importing same file
Issue

	Chrome

	- Sales > Product
	- Select a file
	- Change something in the file
	- Re-select it

	Nothing changed

Cause

	Chrome doesn't trigger change on
	input if the file is the same

Solution

	Clear the input value after the
	parse_preview so we can load a new
	file even if the preview failed.

OPW-2288191

closes odoo/odoo#54175

X-original-commit: f7d2bdf65901e3ee78e1055cab7cf8d60f798621
Signed-off-by: Jason Van Malder (jvm) <jvm@odoo.com>
2020-07-07 09:40:11 +00:00
Martin Trigaux 400cc4f14e [FIX] *: correct all or improve code translation lookup
This commit fixes all issues detected by the new pylint
gettext-variable test.
It converts some calls to the new syntax
  _("Foo %s", bar)

to progressively migrate the code to the new syntax.

A few calls were not technically incorrect but still detected by the
linter.

  _("Foo" +
    "Bar")

has been converted to

  _("Foo"
    "Bar")

as it has the same effect and make sure the argument is of type
asteroid.Const instead of BinOp).

closes odoo/odoo#53683

Related: odoo/enterprise#11467
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-06-30 10:19:59 +00:00