Commit Graph
889 Commits
Author SHA1 Message Date
Ayman Adil 6a3444e5bd [FIX] convert.py: skip the creation of a forcecreate=0 record when a reference is missing
When a record is `forcecreate=0` and a reference of one of its fields is missing, the update of the corresponding module fails. In this case, we can skip the creation of this record.

An example of this issue is this [record](https://github.com/odoo/enterprise/blob/6411ae071ace980834befeb8040c94b1a05c8034/documents_account/data/data.xml#L17) in `documents_account` module (enterprise) when the reference of this [field](https://github.com/odoo/enterprise/blob/6411ae071ace980834befeb8040c94b1a05c8034/documents_account/data/data.xml#L20) is missing.

This can be reproduced as follows:
1. Create a DB with `documents_account` module
2. Uninstall `documents_account` module
3. Remove `documents.documents_finance_status`
4. Reinstall `documents_account`

```
Traceback (most recent call last):
  File "/home/ayman/src/odoo/15.0/odoo/tools/convert.py", line 680, in _tag_root
    f(rec)
  File "/home/ayman/src/odoo/15.0/odoo/tools/convert.py", line 567, in _tag_record
    f_val = self.id_get(f_ref)
  File "/home/ayman/src/odoo/15.0/odoo/tools/convert.py", line 663, in id_get
    res = self.model_id_get(id_str, raise_if_not_found)
  File "/home/ayman/src/odoo/15.0/odoo/tools/convert.py", line 669, in model_id_get
    return self.env['ir.model.data']._xmlid_to_res_model_res_id(id_str, raise_if_not_found=raise_if_not_found)
  File "/home/ayman/src/odoo/15.0/odoo/addons/base/models/ir_model.py", line 1943, in _xmlid_to_res_model_res_id
    return self._xmlid_lookup(xmlid)[1:3]
  File "<decorator-gen-35>", line 2, in _xmlid_lookup
  File "/home/ayman/src/odoo/15.0/odoo/tools/cache.py", line 90, in lookup
    value = d[key] = self.method(*args, **kwargs)
  File "/home/ayman/src/odoo/15.0/odoo/addons/base/models/ir_model.py", line 1936, in _xmlid_lookup
    raise ValueError('External ID not found in the system: %s' % xmlid)
ValueError: External ID not found in the system: documents.documents_finance_status
```

closes odoo/odoo#107766

X-original-commit: 2e4f0667397d2d6670df50b9088fa9c23f1f9f5a
Signed-off-by: Christophe Simonis <chs@odoo.com>
2022-12-12 18:16:15 +01:00
Denis Ledoux 15948702e5 [IMP] tools: locate_node, remove useless code
`if isinstance(node, SKIPPED_ELEMENT_TYPES):`
---------------------------------------------

It has been added in
https://github.com/odoo/odoo/commit/31b9bee00676e179c124f3d42539103305cf7423

It's hard to tell why it has been added. Though it seems not useful:
- Has we iter the nodes using the tag of the specification,
  `arch.iter(spec.tag)`,
  this is normally impossible to iter on a node type from that
  SKIPPED_ELEMENT_TYPES node type list
  (`etree._Comment, etree._ProcessingInstruction, etree.CommentBase, etree.PIBase, etree._Entity`),
  as those node types do not have a tag (`spec.tag`).
  In addition, in the place calling `locate_node`,
  we already do this same check on the specification node:
  https://github.com/odoo/odoo/blob/48109c15d319ae69a8fcdd4a6c3b19172a2de6d9/odoo/tools/template_inheritance.py#L144-L150

Version spec should match parent's root element's version
---------------------------------------------------------

It has also been added in the same revision:
https://github.com/odoo/odoo/commit/31b9bee00676e179c124f3d42539103305cf7423

This has been added in 7.0 to handle `version="7.0"` in xml views.

However, these `version="7.0"` have been removed in 8.0, in revision
odoo/odoo@faa09da325
After the above revision, there were still some leftovers,
but which have all been removed in 11.0 in revision
odoo/odoo@70942e4cfb
during an improvement of the RNG validator.

This is therefore enough evidence to say this code
checking for the `version` attribute in `locate_node`
is no longer useful since a quite long time now.
It can therefore be removed.

closes odoo/odoo#107340

Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2022-12-12 11:22:32 +01:00
Chong Wang (cwg) eab341aec3 [FIX] core: drop mismatched and illegal model terms
The model terms translations assume the number of terms in each language of a
model_term translated field should be the same.
However, sometimes the assumption cannot be promised for sake of bad
translations

This commit
1. drops illegal model term translations while translating
2. drops mismatched terms at run-time in case the database has been contaminated

closes odoo/odoo#107373

X-original-commit: f9ab5ca3e99b2883ada18a8f6deb12d45608b43f
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-12-07 18:29:18 +01:00
Chong Wang (cwg) a0bd7d0d72 [IMP] core: improve translation import speed
this commit improves the performance of importing translations by
1. groupup udpating data for different languages and different modules
2. read model_terms translations data directly with xmlid to get rid of fetching
   data from ir_model_data

for modules:sale_management, point_of_sale, industry_fsm, helpdesk, mrp, mrp_plm
(62 installed modules)
time for test_language_install is 45.14% less

for all modules:
time for test_language_install is 68.03% less

closes odoo/odoo#107332

X-original-commit: 727ef9e0a2ca93089b472eb1313117fee8dc876d
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-12-06 17:12:21 +01:00
jard@odoo.com d65f3e64b2 [FIX] tools.image: Allow conversion of P to JPEG
While upgrade of a customer database, A P-type image
was being converted to JPEG image. It didn't allow to save
as direct conversion from P-type(palette) to JPEG. So, fix
covers conversion of such corner cases to first convert it
to RGB and then RGB can convert it to desired output format
opw-3043418

closes odoo/odoo#107300

X-original-commit: d0db7a14dec7ee6a983ede3bf7b968675f83979d
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2022-12-06 17:12:02 +01:00
william-andre 88879f9f27 [IMP] core: allow to have virtual SQL constraints
Sometimes a constraint is too complex to be expressed in the general
`_sql_contraint`.
This allows to hook on a constraint if it matches the name in order to
display a user friendly error message.

X-original-commit: e1f06479a526c703ccabc441b1e194646206b966
Part-of: odoo/odoo#106325
2022-11-23 16:14:44 +01:00
abd-msyukyu-odoo 51b5ee8d5f [FIX] web_editor, knowledge: prevent modification of data-oe-protected
Change class="oe-blackbox" to data-oe-protected="true" (same purpose), because
the name was decided in a rush before 16.0

When the editor history management and sanitization (sanitize.js) handle nodes
of an embedded view, OWL can become corrupted, which will lead to a traceback.

This commit uses the attribute data-oe-protected="true" to prevent
historyRollbacks or sanitization, and ensure that protected elements children
are removed during the cleanForSave operation.

The collaborative DOMPurify sanitization is untouched, for security reasons. To
properly use `data-oe-protected` in collaborative mode, a more complex
mechanism should be put in place to ensure that the protected element children
are properly reconstructed for each client, and that it is not part of the
collaboration. (i.e.: embedded view in knowledge do not support the
collaborative mode currently).

This commit also blocks the selection handling in tables which would be in a
data-oe-protected="true" node.

Task-3012552

closes odoo/odoo#106255

X-original-commit: 394e4fe567bb2eecbd0865de82914aa7396fde63
Related: odoo/enterprise#34245
Signed-off-by: David Beguin (dbe) <dbe@odoo.com>
Signed-off-by: Abeloos Damien (abd) <abd@odoo.com>
2022-11-23 11:26:12 +01:00
FrancoisGe b1f87e1088 [IMP] tools: transpiler adds a QUnit.module to js test files
The purpose of this improvement is to easily identify the module of a
test and to be able to run only the tests of a module.

closes odoo/odoo#105530

Task-id: 3062058
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2022-11-21 11:50:05 +01:00
Vincent Schippefilt 2fb1ed9175 [FIX] profiler: fix typos
closes odoo/odoo#105928

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2022-11-17 13:57:35 +01:00
Chong Wang (cwg) dc15f81d08 [IMP] core: new comments for translations
before this commit:
If a translation is used for web client, the translation has comment
'openerp-web'.

after this commit:
If a translation is used for web client, the translation has comment
'odoo-javascript'
If a translation is used for python, the translation has comment
'odoo-python'

PO/POT files with old comments still works. The only overhead is: translation
only for web client will also be stored as code translation for python.

closes odoo/odoo#105764

X-original-commit: 13a5fa56ac7a102ad38f7ecc11874786bdb2fe85
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2022-11-15 16:16:54 +01:00
Chong Wang (cwg) 424774e081 [FIX] core: fix export web client translations
before the commit: web client code translations cannot be exported in the po.

If a web client only translation (not used for python translation and has
comment openerp-web) has its translation in the po file of /i18n, when it is
exported in a po file from Settings -> Translations -> Export Translation,
its corresponding translation is missing.

after this commit: web client code translations can be exported in the po.

X-original-commit: c2bf7665363b988d7c7e8bff43e71b7b1097ad84
Part-of: odoo/odoo#105764
2022-11-15 16:16:54 +01:00
Rémy Voet (ryv) 29e9d99364 [FIX] core: field indexed by trigram cannot become translated
Field indexed by trigram cannot become translated because the
`convert_column_translatable` is based on the old index naming convention
(changed in https://github.com/odoo/odoo/pull/100736).
Then it generates a PostgreSQL error:
`psycopg2.errors.DatatypeMismatch: operator class "gin_trgm_ops" does not accept data type jsonb`

closes odoo/odoo#105295

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-11-08 13:36:05 +01:00
Chong Wang (cwg) d57d8534bb [FIX] core: fix search with accent
For a trigram indexed translated field field_x,
domain leaf ('field_x', '=', value) and ('field_x', 'ilike', pattern)
will miss records if the pattern contains accent characters

This commit set ensure_ascii=False for json.dumps to avoid escaping special
characters

opw-3048182

closes odoo/odoo#105240

X-original-commit: 26b5c8c2baeaee9fcd5384aafa5f9b39543d6903
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2022-11-07 21:37:44 +01:00
Rémy Voet (ryv) 0787f150b1 [FIX] core: make index naming without conflicts
There are two issues with the index naming convention used by the ORM:

Problem 1: it is possible to have naming conflict for indexes.  For
instance, the name 'slide_channel_tag_group_sequence_index' is used for
both fields slide.channel.tag.group.sequence and
slide.channel.tag.group_sequence.  Only the first index will be created.

Solution 1: we separate the model and field names with a double
underscore instead of a single one, which is the same strategy as with
LEFT JOIN aliases.  This is correct because model names don't contain
such double underscores or underscores as prefix or suffix (it is not
forbiden but model name should follow the 'dot notation'.)

Problem 2: index names can be longer than 63 chars, but PostgreSQL
silently truncates it.  This doesn't actually break anything (PostgreSQL
also truncates values when we check the existence of indexes) but it can
lead to using the same name twice.  There is hopefully not any example
in our code.

Solution 2: if the name is too large, we truncate it and pad it with a
hash of the complete name to match 63 characters, which is also the
strategy used for LEFT JOIN aliases.

task-2984730

closes odoo/odoo#100736

Related: odoo/upgrade#3957
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-11-07 15:57:28 +01:00
Krzysztof Magusiak 167cf8fff8 [FIX] tools.image: Update IMAGE_MAX_RESOLUTION
New phones go up to 48MP (Samsung Galaxy A22).
Also the error message was saying 4.5 instead of 45.

opw-3020614, opw-3020502

Close #104424

closes odoo/odoo#104546

X-original-commit: 8b19107c69c648dbfba2a9304c04f0f55aac4b46
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-11-02 09:45:40 +01:00
xmo-odoo 84e6be4150 [REM] core: deprecated osv-memory-age-limit option
Part-of: odoo/odoo#98138
2022-10-26 19:03:47 +02:00
Xavier Morel 18446f23f1 [REM] core: support for the deprecated <report> and <act_window> tags
Part-of: odoo/odoo#98138
2022-10-26 19:03:46 +02:00
Chong Wang (cwg) 105e0b9ef2 [FIX] core: search translated fields with characters needed to be escaped
The trigram index function jsonb_path_query_array("column_name", '$.*')::text
uses all translations' representations to build the indexed text. So the
original text needs to be JSON-escaped correctly to match it.

X-original-commit: 7547df664945dddcb839e4903068f7f25ecfc08c
Part-of: odoo/odoo#103031
2022-10-11 14:10:16 +02:00
Chong Wang (cwg) bde30899a4 [FIX] core: new PO file overrides old ir_translation
Before: while upgrading to 16.0, the PO file is loaded before migrating
ir_translate. As a result, old translations from ir_translation
override the new translations imported by new PO files.

After: translations imported by the PO file take priority over the
translations migrated from the ir_translation table.

X-original-commit: 178270eff48221537a4efba1ad8fa0b53a1516f2
Part-of: odoo/odoo#103031
2022-10-11 14:10:15 +02:00
Antoine Dupuis 5a09449ddc [FIX] tools: fix filename resolution in XSD imports
With PRs odoo/89145 and enterprise/26393 came new methods for
retrieving XSD files and using them for XML validation.

The new retrieval method expects modules to provide a 'prefix' that
is prepended to the XSD's filename. For example, l10n_cl_edi will
name its XSD files 'l10n_cl_edi.<filename>.xsd'.

However, this messes things up when one XSD file needs to import
another. For example, l10n_cl_edi.DTE_v10.xsd has the statement
'<xs:include schemaLocation="SiiTypes_v10.xsd"/>'

Currently, the filename resolver has no way of knowing that this
should resolve to 'l10n_cl_edi.SiiTypes_v10.xsd', not
'SiiTypes_v10.xsd'.

In addition, the new retrieval method saves the ZIP archives
received over the network under the '<filename.xsd>'. Thus
'SiiTypes_v10.xsd' might actually be a ZIP-encoded file.

So, we need to do something to fix the imports.
Here are two possible solutions:
1. We scrap this 'prefix' stuff and either save the ZIP files under
   a different name, or we just don't save them.
2. Or, we provide a mechanism for indicating a prefix to the filename
   resolver.

Personally, I don't see the point in saving the ZIP files, and this
'prefix' stuff seems pointless. So I prefer solution 1.

But, because I assume there must be a reason to all of that 'prefix'
stuff, here is an implementation of solution 2.

I'd be keen to know the reason, btw.

EDIT:
In addition to the first issue described above, we have the second
issue that some XSD files returned by the Chilean SII are encoded
using ISO-8859-1 encoding (e.g. SiiTypes_v10.xsd). If we leave them
in this encoding, then LXML isn't able to parse them when performing
imports.

closes odoo/odoo#102601

Solution: convert the files to UTF-8 before storing them.
X-original-commit: 75555df56475b457331938453657c1f73d231e33
Related: odoo/enterprise#32482
Signed-off-by: Josse Colpaert <jco@odoo.com>
2022-10-07 14:37:14 +02:00
Walid HANNICHE (waha) 82265c52ca [FIX] tools: fix underline and italic in logged notes
Steps to reproduce:
- in any app log a note using full composer
- add styles to the text (underline, strike-through, italic)

Bug:
styles except bold are removed

Fix:
added missing styles to the whitlelist

opw-2956374

closes odoo/odoo#102056

X-original-commit: eb07ab104d5f47574e14db811108a7d0cbbbe4b2
Signed-off-by: David Monjoie (dmo) <dmo@odoo.com>
2022-10-04 16:01:06 +02:00
Damien Abeloos 1f6005070f [IMP] tools: keep additional data when sanitizing
Purpose is to allow to have ``data-behavior-props`` and ``data-prop-name``
on HTML nodes and keep it even when HTML content is sanitized. Those attributes
will be used notably in Knowledge application to store some structured content
e.g. used to populate props.

Prepares Task-2796156
Prepares odoo/enterprise#29423

X-original-commit: 7ddb4685567c115ca4e14fa7e5f845b0cb33fd7c
Part-of: odoo/odoo#101694
2022-09-30 10:33:01 +02:00
Lucas Lefèvre f07cd645c9 [FIX] tools: don't extract missing spreadsheet data source name
Previous versions of spreadsheet files don't have a name
for list and pivot datasources.

The spreadsheet template "pipeline dashboard" in `document_spreadsheet_crm`
is such a file.
As a consequence, exporting the source terms from this module fails
with a `KeyError: 'name'`

Part-of: odoo/odoo#101659
2022-09-30 09:31:52 +02:00
Lucas Lefèvre 723d8f8b50 [FIX] tools: extract translatable terms from spreadsheet dashboard
This commit extract translatable content from spreadsheet data.
- chart titles and description
- pivot and list names
- argument of _t() functions
- labels of links

X-original-commit: d210a55f48b316fdf38fcbe5ae5a70cf0e2c3814
Part-of: odoo/odoo#101120
2022-09-26 14:43:21 +02:00
Martin Trigaux 5150d003fd [I18N] *: export 16.0 source terms
closes odoo/odoo#101053

X-original-commit: 7d4c9d9736cc2ff0ee91dc17ea0f952c9a3dc65a
Related: odoo/enterprise#31752
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2022-09-24 15:00:56 +02:00
Martin Trigaux ffbd1ff062 [IMP] tools: do not export attribute content
Nodes like
<attribute name="t-on-dragenter.stop.prevent">() => state.dropzoneVisible = true</attribute>
should not be exported in the translations files but
<attribute name="string">Hello World</attribute>
should be kept

It was already the case in the xml processing method for server side
templates.
Since 16.0, client side QWeb views also use the same inheritance
mechanism than the one on the server side so the exception needs to be
replicated.

X-original-commit: e05d1192481935b2cd7aae7e939f98e60e002899
Part-of: odoo/odoo#101053
2022-09-24 15:00:56 +02:00
Xavier-Do 33b44ce84b [FIX] config: don't save deprecated longpolling-port
Longpolling port is replaced with gevent port and is deprecated
This commit avoid saving the value.

Not really usefull but when saved the value was None leadind to an error
when casting to int. The default value should be an int.

closes odoo/odoo#100991

X-original-commit: adac9a9ceaa173c0181a4fc57d0ea83b7a37fc71
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2022-09-23 14:37:17 +02:00
Chong Wang (cwg) a61c0e3fc4 [FIX] tools: export pot files
Before this commit, it was not possible to export the .pot files

Part-of: odoo/odoo#100573
2022-09-20 13:48:49 +02:00
ef00294e71 [IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation.  Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table.  The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages).  Empty text is allowed in translation values, but not NULL.

Here are examples for a field with translate=True:

    NULL
    {"en_US": "Foo"}
    {"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
    {"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}

Like before, writing False to the field makes it NULL, i.e., False in all
languages.  However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.

Here are examples for a field with translate=xml_translate:

    NULL
    {"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}

Change for callable(translate) fields: one can now write any value in any
language on such a field.  The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values.  Basically the
structure of the value must remain the same in all languages, like before.

Reading a translated field is now both simpler and faster than the former
implementation.  We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:

    SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...

The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages).  For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.

Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language.  Before this change, only the source value of the
field ('en_US') could be indexed.

Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages.  We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.

Code translations are no longer stored into the database.  They become static,
and are extracted from the PO files when needed.  The worker simply uses a cache
with extracted code translations for performance.  This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker.  Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).

Performance summary:
 (+) reading 'model' translated fields is faster
 (+) reading 'model_terms' translated fields is much faster (no need to inject
     translations into the source value)
 (+) searching translated fields with operator 'ilike' is much faster when the
     field is indexed with 'trigram'
 (+) updating translated fields requires less ORM flushing
 (-) importing translations from PO files is 2x slower

Some extra fixes:
 - make field 'name' of ir.actions.actions translated; because of the PG
   inheritance, this is necessary to make the column definition consistent in
   all models that inherit from ir.actions.actions.
 - add some backend API for the web/website client for editing translations
 - move methods get_field_string() to model ir.model.fields
 - move _load_module_terms to model ir.module.module
 - adapt tests in test_impex, test_new_api
 - because env.lang is injected into SQL queries, its returned value is
   now guaranteed to correspond to a valid active language or None
 - remove wizard to insert missing translations (no longer makes sense)

task-id: 2081307

Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-09-15 22:37:50 +02:00
Raphael Collet 50767ef90e [IMP] core: simplify code for column conversion in fields 2022-09-15 22:30:56 +02:00
Raphael Collet 70c776f21f [FIX] core: CLI --load-language was broken 2022-09-15 22:30:56 +02:00
Rémy Voet (ryv) 4087bcdc5e [IMP] core: add unaccent to trigram indexes
We added trigram index for char fields since https://github.com/odoo/odoo/pull/83015.
But if `unaccent` is installed in the database (and isn't force to
`False` on field), these new trigram indexes are pointless and cost a
lot for nothing (almost nothing, it still can be used for equality
operator but in this case a btree will be far more efficient).

The simple way to fix it is to add `unaccent(<column>)` in the index
trigram definition, but unfortunately `unaccent` is not immutable and
may therefore not be indexed.  In order to make `unaccent` indexable, we
must declare it as immutable (see
https://stackoverflow.com/questions/11005036/does-postgresql-support-accent-insensitive-collations/11007216#11007216
for more information and how to do that).

With this patch, trigram indexes are created with `unaccent(<column>)`
if the function `unaccent` is available in the database, and for the
fields that are not declared with `unaccent=False`.  Moreover, we issue
a warning when `unaccent` is available but is not immutable, in which
case most trigram indexes will be useless.

odoo/upgrade#3736
task-2551518

closes odoo/odoo#95943

Signed-off-by: Rémy Voet <ryv@odoo.com>
2022-09-05 18:34:53 +02:00
std-odoo 87307a9010 [IMP] base: add new "Properties" fields
Purpose
=======

Add a new field "Properties" to be able to light customization of workflows
based on a parent model. Those properties acts in some ways like Odoo fields
without requiring specific columns e.g. add new properties on tasks of a
specific project.

Usage
=====

Define properties on a parent model (e.g. project) with

```
    attributes_definition = fields.PropertiesDefinition('Message Properties')
```

It defines properties available on children: types, default, value, model for
relational properties,...

Use it on children records (e.g. task) with

```
    attributes = fields.Properties(
        string='Properties',
        definition='parent_id.attributes_definition',
    )
```

Technical
=========

Parent | Properties definition
------------------------------
The properties definition is stored on the parent, on a JSON field.
This definition contains the type of the properties, the default value,
the model of the many2one,...
```
[
    {
        'name': 'name',
        'string': 'Name',
        'type': 'char',
        'default': 'Default Name',
    }, {
        'name': 'partner_id',
        'string': 'Partner',
        'type': 'many2one',
        'comodel': 'res.partner',
    },
]
```

Child | Properties values
-------------------------
The value is stored on the child, using a Properties field.
```
{
    'name': 'Mitchel',
    'partner_id': 1337,
}
```

When we read this field, we will automatically read the definition on
the parent, and merge both JSON into one, so the web client has the
value of each property, and their definition.

```
[
    {
        'name': 'name',
        'string': 'Name',
        'type': 'char',
        'default': 'Default Name',
        'value': 'Mitchel',
    }, {
        'name': 'partner_id',
        'string': 'Partner',
        'type': 'many2one',
        'comodel': 'res.partner',
        'value': 1337,
    },
]
```

Integrity
---------
If we remove a property on the parent, we won't update the child value.

Instead, when we read the child properties, we will filter them based
on the parent. So the removed properties will be removed the next time
we write on the field.

In the same logic, the many2one existence is checked when we read the
field. There's no foreign key between the integer stored in the JSON
in the SQL row corresponding to the record in database.

Write
-----
We can write on the Properties field with a list of field definition
+ value.

Some types are not JSONifiable (like the date, datetime), they are
stored as string in database and parsed when we read the value.

In order to update the parent definition by writing on the child,
you need to add the dict key `definition_changed` or
`definition_deleted`. This is because we need to be able to know
if the definition has been changed without doing extra SQL queries.

Access rights
-------------
A user can add a many2one / many2many property to a model only if he
has the access rights to it.

Many2one / Many2many
--------------------

The model choice of a many2one / many2many properties was subject to
changes.

First implementation stored models in both parent and children to easily
spot changes and avoid complex queries when fetching records, trying to
synchronize them, ...

As this leads to storing a lot of duplicated content we choose to instead
reset the value on the child if the model has been change. We generate a
new name for the property. So it behaves like if we removed the property
and created a new one.

To be able to restore the old value (e.g. if by mistake we changed the
model, and go back to the old model), we store the initial states.

Task-2852259

Part-of: odoo/odoo#95184
2022-08-29 23:46:06 +02:00
Raphael Collet 07744c64e9 [FIX] core: in ir.model.fields, discard deleted fields from registry
This patch discard the fields to delete from data structures like field
triggers, field inverses, and fields to compute.

This is quite useful when uninstalling modules.  This commit fixes the
uninstallation of modules base_setup, bus, mail, web_tour.

closes odoo/odoo#98668

X-original-commit: 49398a769b2ec4965bf148f10b5e13d0a6c1acff
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-08-25 22:46:04 +02:00
Pratik Raval bb24757d08 [IMP] crm: detect leads based on similar phone/mobile number
Currently, the 'similar lead detection' mechanism only considers email,
contact name, and partner name while finding duplicate leads in CRM.

After this commit, it will also consider the mobile/phone number for
the same. Note that the mobile numbers and phone numbers both will be
matched with each other while finding duplicate leads.

Task-2817884

closes odoo/odoo#88372

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2022-08-25 19:56:48 +02:00
Florian Charlier db7c9ba7c5 [IMP] tools, website_{*}: increment multiple fields at once
* = blog, forum, slides

For performance reasons, allow to increment multiple fields of the same record
within the same query.

With python tests.

Task-2663320
Part of odoo/odoo#79615
2022-08-25 14:24:53 +02:00
tsm-odoo 29468b1272 [IMP] config: deprecate longpolling port in favor of gevent port
This commit is part of the websocket integration in Odoo.
The longpolling port does not make sense anymore: longpolling has been dropped.
This commit deprecate the `--longpolling-port` option and replace it by the
`--gevent-port` option.

closes odoo/odoo#75510

Related: odoo/enterprise#23184
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2022-08-23 17:55:11 +02:00
tsm-odoo f8e794a864 [IMP] bus, tools: add rate limiting to incoming websocket messages
This commit is part of the websocket integration in Odoo.
It focuses on implementing rate limiting to the incoming
websocket messages. When opening a connection, a burst of messages
is allowed. When requests are received too fast, an exception is
raised and the websocket is closed.

Two config parameters are added to customize the rate limtier
behavior:
   - websocket_rate_limit_delay: Integer specifying the seconds that
     should space out two requests.
   - websocket_rate_limit_burst: Integer specifying how many websocket
     frames can be accepted in excess of the specified rate.

Part-of: odoo/odoo#75510
2022-08-23 17:55:09 +02:00
tsm-odoo e06bb9a42d [ADD] bus: add websocket implementation
This commit is the first commit of the websocket integration in Odoo.
It focuses on the implementation of the websocket protocol as per RFC6455.

The implementation is tested thanks to the autobahn test suite.

A config parameter is available to customize the websocket connection:
   - websocket_keep_alive_timeout (default 600): Integer specifying how
     many seconds a websocket connection should be kept alive

Part-of: odoo/odoo#75510
2022-08-23 17:55:09 +02:00
Xavier Morel 9c0fa1efe3 [CHG] core: deprecate get_module_filetree
It's pretty much unused and fairly complicated.

Also deprecate `listdir` entirely since `get_module_filetree` is the
only extant user of the recursive listdir.

closes odoo/odoo#98034

Related: odoo/enterprise#30403
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-08-23 07:36:24 +02:00
Ivan Yelizariev c0bbe22642 [FIX] core: apply user's tz on formatting Datetime field as Date
STEPS in v14:

1. Set user's timezone to Asia/Qatar
2. Inventory > Config > Setting > Enable Lots & Serial Numbers, Expiration Dates, Display Lots & Serial Numbers on Delivery Slips, Display Expiration Dates on Delivery Slips
3. Create a product (storable, tracking by lots, expiration date)
4. Manually update the on-hand quantity for the product
5. Create a lot for the product created in Step 3 > Set the expiration date to 3/30/2025 00:00:00
6. Create an SO with the product > Confirm it to Delivery
7. Validate the Delivery > Print the Delivery Slip

Before this commit the expiration date is rendered as 03/29/2025 instead of 03/30/2025

opw-2901367

closes odoo/odoo#98371

Signed-off-by: Julien Castiaux <juc@odoo.com>
2022-08-22 16:48:40 +02:00
Xavier Morel 4cd0119eff [CHG] core: move traverse_container out of utils
It's really not a general purpose utility, it's just a way of forcing
a lazy collection's evaluation.

Alternatively, add a helper for this to / alongside `lazy`? Note that
the lazy object(s) may not be the top-level.

closes odoo/odoo#98023

Related: odoo/upgrade#3776
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-08-22 16:48:28 +02:00
Xavier Morel 7f14631fe8 [CHG] core: deprecate all exec_pg_* functions
They're not used much and they don't seem much of an advantage
compared to just calling `subprocess` with the discovery utility
functions.

While at it, fix `dump_db` to not unnecessarily have an open stdin to
`pg_dump`.

Part-of: odoo/odoo#98023
2022-08-22 16:48:28 +02:00
Xavier Morel 719e7d46e4 [CHG] core: mark a bunch of utilities as deprecated
Remove UnquoteEvalContext directly because it's unused and super specialised.

Part-of: odoo/odoo#98023
2022-08-22 16:48:27 +02:00
Shawcker 6cdb99f887 [ADD] tools: add XSD loading related methods in xml_utils
Currently, there is no globally available way to load xsd files.
Modules that use xsd files to validate xml files all load the files with their own methods, but in the end they all do the same thing. It is redundant and hard to maintain. On top of that, new modules that need to load such xsd files need to implement it again.

This commit adds an easy way to load xsd files (either directly .xsd files or from .zip archives) and save them as ir.attachment to use with _check_with_xsd method for xml validation purposes.
It also adds a function to validate an XML file with an XSD. This function allows for a reloading method to be called if the XSD file was not found in database.
In order to avoid excessively downloading XSD files (during tests for instance), the 'skip_xsd' flag can be set to True in the context. This will skip the XSD validation (and thus download).

task id=2782053

closes odoo/odoo#89145

Related: odoo/enterprise#26393
Signed-off-by: Laurent Smet <las@odoo.com>
2022-08-22 16:48:12 +02:00
Romain Derie a462d87c4b [FIX] *: set correct type on html field in xml created records
When a record is created through xml data, its HTML fields should
receive a `type="html"` attribute, not a `type="xml"` attribute.

When important XML data with XML type instead of HTML type will have 2
differences:
- The field value will be prefixed by `<?xml version="1.0"/>`
- If the HTML contains multiple root nodes, the value will be wrapped in
  a `<data/>` tag.

See `_fix_multiple_roots()` and the `xml_import` class for more details.

closes odoo/odoo#98239

Related: odoo/enterprise#30491
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-08-19 11:52:49 +02:00
Xavier-Do 421d500842 [FIX] profiler: manage frame without lineno
In some case the frame may have a filename and corresponding file but
no lineno. This may lead to an error

File ".../odoo/tools/profiler.py", line 571, in _add_file_lines
    line = filelines[lineno - 1]
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

This commit simply fixes this by skipping the logic if we have no lineno.

closes odoo/odoo#98217

X-original-commit: 44715af97413da3608e99f211c361a498f25d964
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2022-08-17 11:21:50 +02:00
Xavier Morel 5c5876d74f [REM] osutil: deprecated code
We deprecated all those in 14.0, so 16.0 seems like a good time to
remove them.

`getppid` was not deprecated but it doesn't seem useful to keep

closes odoo/odoo#97987

Around: windows support was added to getppid in Python 3.2.
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-08-16 09:19:14 +02:00
Adrien Widart 320025ee63 [FIX] base: convert barcode type if incorrect encoding
To reproduce the issue:
(Need stock_barcode)
1. Create a product:
    - Barcode: 1234567890123
2. Print the label
3. Try to scan the barcode and check the value read.

Error: The value is 1234567890128, the last digit is incorrect (8
instead of 3)

When printing a barcode, we use the library 'report-lab' to generate a
barcode image from a value and a barcode type. In case of EAN-13, if the
value contains a non-digit character, it will raise an error. We then
catch the error and retry to generate the barcode according to the
barcode type Code128:
https://github.com/odoo/odoo/blob/87698d90f02bfe93c6e643f4876a5ccd74788eff/odoo/addons/base/models/ir_actions_report.py#L569-L575
However, if the value contains only digits, the method will use the 12
first digits:
https://github.com/mattjmorrison/ReportLab/blob/dade0f303cb6fcdbe535c4cc92e6102c2417b699/src/reportlab/graphics/barcode/eanbc.py#L187-L188
and will then add the last one, the check digit, which is computed by
the library. This explains why, in the above use case, the barcode value
returned by the scanner is not the same than the expected one.

Note: Similar behavior with type EAN-8

OPW-2902150

closes odoo/odoo#97052

X-original-commit: a2c7470f8fd46b2520f7b8f0750c63216273ce96
Related: odoo/enterprise#30160
Signed-off-by: Steve Van Essche <svs@odoo.com>
Signed-off-by: Adrien Widart <awt@odoo.com>
2022-08-11 12:51:05 +02:00
Martin Trigaux ffc525419a [IMP] base: avoid render with env mixup
The render API was confusing as mixing the access to the report and
the rendering env.

The ambiguity was present for code such as
`report.sudo()._render(record_ids)` where it was not clear if the
`sudo()` is needed to access to `report` or to `record_ids`. For low
priviledge users (such as portal or public), it was common to use
`report.with_user(SUPERUSER_ID)._render(record_ids)`.

This PR changes the render methods signature to be `api.model`. The
`report_ref` can be:
- ir.actions.report external id
- ir.actions.report id
- ir.actions.report recod
- `report_name` value

This will allow to call the report methods with any user and no longer
need to use `with_user(1)` to render reports as public user.

Task-id 2670865

closes odoo/odoo#91341

Related: odoo/upgrade#3650
Related: odoo/enterprise#27323
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2022-08-02 11:48:46 +02:00