Commit Graph
82 Commits
Author SHA1 Message Date
Ivan Rasputin 1711e91257 [IMP] core: test exporting of translation files
Static qweb files are not tested for syntax errors.
This commits adds a hacky way to do this job: we test exporting of translation files.

Inspired by the following PRs, that were made as a response for errors caught by sentry

* https://github.com/odoo/enterprise/pull/43975
* https://github.com/odoo/odoo/pull/128221

closes odoo/odoo#128445

Related: odoo/enterprise#44074
Signed-off-by: Raphael Collet <rco@odoo.com>
2023-09-15 09:22:04 +00:00
Chong Wang (cwg) ddfdba9424 [FIX] core: update model terms for en_US and other
For model_terms translated fields if a translation for a lang(fr_FR) has never
been defined

before this commit
when update translations for both en_US and fr_FR, the new translation for fr_FR
cannot be saved.

after this commit
new translations can be correctly saved when en_US and fr_FR are updated at the
same time.

closes odoo/odoo#135103

X-original-commit: ef4b195eeac178a14eb47f4e6cc61ddc97f78866
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Chong Wang (cwg) <cwg@odoo.com>
2023-09-11 23:59:51 +00:00
Rémy Voet (ryv) 3c62ca1eb9 [REM] core: remove name_get API
Rationale
=========

Since v8, the `display_name` field is present on all models. By default,
`display_name` uses `name_get` which has pretty much the same purpose
(return record name used by the web client). Gradually, many (backend)
developers (and the ORM: https://github.com/odoo/odoo/commit/6da1c3ac4c036eac289597602976538e243cb939)
started using `display_name` (more convenient than
`record.name_get()[0][1]`) but it still had the `name_get` override.
It becomes more complex than necessary and poeple start to misunderstand
the two (and sometimes override both, leading to inconstiencies between
`display_name`/`name_get`).

To simplify the ORM and the API, we decided to keep only one of them,
the `display_name` field:
- It is much more convenient from a backend point of view
(`record.name_get()[0][1]` vs `record.display_name`)
- It is cached during the same transaction (and invalidated if
its dependencies change)
- It can be overridden like any other compute field (override
`_compute_display_name` with any extra dependencies)
- `name_get` is replaced by `read(['display_name'])`
(API perceptive), which can actually be more efficient
(if `display_name`'s depends are correct, the ORM will only fetch the
fields it needs instead of every prefetchable field)

Changes
=======

- Deprecates `name_get` for the v17 and based the method on
`display_name` (the opposite of before)
- Converts all usage of `name_get`
- Overrides of `name_get` are now overrides of `_compute_display_name`
- For `res.partner`, rename the field store `display_name` into
`complete_name` because `display_name` context-dependent and it makes
no sense to have a compute store that is context-dependent.
- Previously, it was possible to return multiple names for the same
record with `name_get`, but it was tricky and most of the usage of
this `name_get` didn't take this into account. The only example of
this is the `name_get` of `product.product`
(now use `", ".join(<names>)`).

Part-of: odoo/odoo#122085
2023-06-28 17:41:19 +02:00
Alvaro FuentesandChong Wang f7fca3faf9 [FIX] core: do not udpate translation terms from text to xml
Steps to reproduce the issue:
* In a clean v16 db install base_setup
* Log in into DB, switch language to French (install and activate it)
* Upgrade to saas-16.1

Issue: The main settings page under Companies settings (Sociétés in Frech) shows
```
<span class="o_form_label">Mise en page du document</span> <span class="fa fa-lg fa-building-o" title="Les valeurs définies ici sont spécifiques à l'entreprise." aria-label="Values set here are company-specific." groups="base.group_multi_company" role="img"/>
```
instead of just
```
Mise en page du document
```

The issue comes from this difference, 16.0:
https://github.com/odoo/odoo/blob/fb02720aed0b0be00df8f5d0e1932b948c300b92/addons/base_setup/views/res_config_settings_views.xml#L78-L79
vs saas-16.1:
https://github.com/odoo/odoo/blob/80bc702ecfe6f704af05f9830d272b162018f2ad/addons/base_setup/views/res_config_settings_views.xml#L79

Note that `Document Layout` is used in two different contexts. In 16.0
it comes within an xml/html block containing a `<span>` tag, and more
importantly it is the **text** part of the xml tag. In saas-16.1 the
same `Document Layout` is used as an **attribute** of the `setting` tag.
Thus we cannot blindly assign the whole term (block with tags) to the
attribute when upgrading to saas-16.1, it is only safe to update the
translation from xml to text. Updating a text entry with something that
seems to have other xml elements is unsafe and can lead to the issue
showcased here.

For more context, at the time of updating the terms here is the
situation:
* `closest_matches` is `['Document Layout']`
* `closest_term` is `Document Layout`
* `old_term` is
  ```
  <span class="o_form_label">Document Layout</span>
                                        <span class="fa fa-lg fa-building-o" title="Values set here are company-specific." aria-label="Values set here are company-specific." groups="base.group_multi_company" role="img"/>
  ```

closes odoo/odoo#123540

X-original-commit: 6cd49293fe4d0d86fbbfa2476e113deadeba7348
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
Signed-off-by: Alvaro Fuentes Suarez (afu) <afu@odoo.com>
Co-authored-by: Chong Wang(cwg) <cwg@odoo.com>
2023-06-02 23:38:58 +02:00
Chong Wang (cwg) 65e50310f7 [IMP] core: allow update empty translations
before this commit
When users open the translation dialog for an empty field, and directly update
and save all translations in the translation dialog, nothing will be saved for
backend

after this commit:
these translations will be saved.

opw-3297748

closes odoo/odoo#122008

X-original-commit: ae643ab48b3447f372bfe08fbad7515f370b7cbe
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2023-05-23 14:52:08 +02:00
Chong Wang (cwg) e03754a9d4 [FIX] core: write terms to en_US if not activated
Before this commit:
when en_US is not activated and the user changes terms slightly for
ir.ui.view.arch, the new term will be treated as a typo fix or style change, and
won't be populated to other languages. As a result, in the form view, arch_base
field which displays the en_US translation of the arch_db will still be the
content before the change(wrong and strange).

After this commit:
when write a model_terms field when en_US is not activated, its en_US value will
always be overwritten.

opw-3265418

closes odoo/odoo#122006

X-original-commit: e7fd2361ab83f3a02ada09c18094adc5c7517227
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2023-05-23 14:52:05 +02:00
Chong Wang (cwg) b92415023d [IMP] core: prefetch all translations
a new context `prefetch_langs=True` allows ORM to prefetch all translations of
translated fields while fetching.

For example
The activated languages are 'fr_FR' and 'nl_NL'
In the database the value is '{"en_US": "English", "fr_FR": "French"}'::jsonb
after fetch with `prefetch_langs=True` the raw cache value will become
{'en_US': 'English', 'fr_FR': 'French', 'nl_NL': 'English'}

closes odoo/odoo#116947

Signed-off-by: Raphael Collet <rco@odoo.com>
2023-04-26 18:12:08 +02:00
Chong Wang (cwg) 26b9b656c6 [FIX] core: allow write translation for non-existing record
before this commit:
    record = env['model.name'].browse(id)
if record doesn't exist in the database and call
   record.translated_field_name = value
Then _get_stored_translation will raise
   TypeError: 'NoneType' object is not subscriptable

after this commit:
like write non-translated field, the value can be written to the cache, but not
the database and no error will be raised.
Note: The feature is only for the original ORM 'write', if the 'overriden write'
reads other fields of the non-existing record, a MissingError will be raised.

closes odoo/odoo#119202

X-original-commit: 3ba7ca28a68acccb8eb25f117900c1aa1980264a
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2023-04-20 15:22:01 +02:00
Chong Wang (cwg) 0cef635102 [FIX] core: fix copy translations
after odoo #113888
when copy translations from one record to another, translations for
non-installed languages may raise error.

These translations may be
1. created before the langauge is deactivated
2. en_US which is always available for non falsy translated field value

this commit drops translations for uninstalled languages except 'en_US' when
copy and prevents raising error when users want to translate en_US when en_US is
not activated

closes odoo/odoo#115711

X-original-commit: 7bb1825ddbf2340882bef5ed1d9c877f78a2b815
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2023-03-20 15:31:27 +01:00
Chong Wang (cwg) 4e282f10fc [IMP] core: void model translation
Before this commit, there is no way to "void" model translations, i.e.,
discard a translation and let the value fall back on the 'en_US' value.
The API of methods write() and update_field_translations() can only
overwrite the translations for the specified languages.

After this commit, calling update_field_translations() with a falsy
value except the empty string discards the corresponding translation
value, and let the value of the field in the given language fall back on
the 'en_US' value of the field.

X-original-commit: 5434fb845c393327db377abf872c448f4860a7d3
Part-of: odoo/odoo#111869
2023-02-03 16:32:34 +01:00
Benoit Socias 0bc3ae0ed5 [FIX] base: ignore spaces around text content when matching translation
When the same text appears several times inside a translated field but
nested in different HTMLs, the matching for each one is done
independently if various spacing appear in the HTML.

This commit strips the spaces around the matched texts so that texts
that are synchronized on purpose do not become desynchronized.
Doing this leads to collisions on the keys of `text2term`, it therefore
also has to replace it with a dictionary of text to list of terms.

opw-3098819

closes odoo/odoo#109798

X-original-commit: 6cec590a2dad43063d3bb747838393a8df13aa04
Signed-off-by: Benoit Socias (bso) <bso@odoo.com>
2023-01-13 17:32:35 +01:00
Chong Wang (cwg) eab341aec3 [FIX] core: drop mismatched and illegal model terms
The model terms translations assume the number of terms in each language of a
model_term translated field should be the same.
However, sometimes the assumption cannot be promised for sake of bad
translations

This commit
1. drops illegal model term translations while translating
2. drops mismatched terms at run-time in case the database has been contaminated

closes odoo/odoo#107373

X-original-commit: f9ab5ca3e99b2883ada18a8f6deb12d45608b43f
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-12-07 18:29:18 +01:00
Chong Wang (cwg) a0bd7d0d72 [IMP] core: improve translation import speed
this commit improves the performance of importing translations by
1. groupup udpating data for different languages and different modules
2. read model_terms translations data directly with xmlid to get rid of fetching
   data from ir_model_data

for modules:sale_management, point_of_sale, industry_fsm, helpdesk, mrp, mrp_plm
(62 installed modules)
time for test_language_install is 45.14% less

for all modules:
time for test_language_install is 68.03% less

closes odoo/odoo#107332

X-original-commit: 727ef9e0a2ca93089b472eb1313117fee8dc876d
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-12-06 17:12:21 +01:00
Chong Wang (cwg) d57d8534bb [FIX] core: fix search with accent
For a trigram indexed translated field field_x,
domain leaf ('field_x', '=', value) and ('field_x', 'ilike', pattern)
will miss records if the pattern contains accent characters

This commit set ensure_ascii=False for json.dumps to avoid escaping special
characters

opw-3048182

closes odoo/odoo#105240

X-original-commit: 26b5c8c2baeaee9fcd5384aafa5f9b39543d6903
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Wang Chong (cwg) <cwg@odoo.com>
2022-11-07 21:37:44 +01:00
Chong Wang (cwg) 105e0b9ef2 [FIX] core: search translated fields with characters needed to be escaped
The trigram index function jsonb_path_query_array("column_name", '$.*')::text
uses all translations' representations to build the indexed text. So the
original text needs to be JSON-escaped correctly to match it.

X-original-commit: 7547df664945dddcb839e4903068f7f25ecfc08c
Part-of: odoo/odoo#103031
2022-10-11 14:10:16 +02:00
ef00294e71 [IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation.  Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table.  The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages).  Empty text is allowed in translation values, but not NULL.

Here are examples for a field with translate=True:

    NULL
    {"en_US": "Foo"}
    {"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
    {"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}

Like before, writing False to the field makes it NULL, i.e., False in all
languages.  However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.

Here are examples for a field with translate=xml_translate:

    NULL
    {"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}

Change for callable(translate) fields: one can now write any value in any
language on such a field.  The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values.  Basically the
structure of the value must remain the same in all languages, like before.

Reading a translated field is now both simpler and faster than the former
implementation.  We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:

    SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...

The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages).  For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.

Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language.  Before this change, only the source value of the
field ('en_US') could be indexed.

Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages.  We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.

Code translations are no longer stored into the database.  They become static,
and are extracted from the PO files when needed.  The worker simply uses a cache
with extracted code translations for performance.  This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker.  Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).

Performance summary:
 (+) reading 'model' translated fields is faster
 (+) reading 'model_terms' translated fields is much faster (no need to inject
     translations into the source value)
 (+) searching translated fields with operator 'ilike' is much faster when the
     field is indexed with 'trigram'
 (+) updating translated fields requires less ORM flushing
 (-) importing translations from PO files is 2x slower

Some extra fixes:
 - make field 'name' of ir.actions.actions translated; because of the PG
   inheritance, this is necessary to make the column definition consistent in
   all models that inherit from ir.actions.actions.
 - add some backend API for the web/website client for editing translations
 - move methods get_field_string() to model ir.model.fields
 - move _load_module_terms to model ir.module.module
 - adapt tests in test_impex, test_new_api
 - because env.lang is injected into SQL queries, its returned value is
   now guaranteed to correspond to a valid active language or None
 - remove wizard to insert missing translations (no longer makes sense)

task-id: 2081307

Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-09-15 22:37:50 +02:00
Raphael Collet 7a248f5933 [IMP] base: add test to measure the installation of a language on a database
The main purpose of the test is to provide statistics for the runbot, in
order to detect unexpected variations in the loading of translations.

closes odoo/odoo#99636

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2022-09-07 02:17:53 +02:00
Raphael Collet 9c3b9a4926 [FIX] core: automatically flush upon invalidation for cache consistency
Because method _read() no longer updates existing values in memory,
those values in memory must be consistent with the database.

On the other hand, if pending updates are not performed on the database
before fetching values, it means that the corresponding database values
cannot be put in cache.  This implies that one cannot empty the cache
without flushing the corresponding fields.

In order to avoid mistakes, flush automatically before invalidating the
cache.  This makes the invalidation methods safe by default, and avoids
cargo-culting which would systematically associate invalidation to
flushing, which may eventually be less performant.

Part-of: odoo/odoo#66938
2022-07-05 11:35:00 +02:00
Raphael Collet 60a1452a40 [REF] base: adapt code to new flush API
Part-of: odoo/odoo#87527
2022-05-25 18:00:47 +02:00
Raphael Collet 68aa10fc24 [FIX] base: do not load translations twice
This fixes former pull request odoo/odoo#78287.

The language installation wizard was invoking toggle_active() on the
target languages, which loads translations for those languages (with
overwrite=False), and then loading translations again (with expected
parameter overwrite).

Simply don't use toggle_active(), and directly set the active field
instead, in order to load translations once with the expected parameter
overwrite.

closes odoo/odoo#90421

X-original-commit: 3e4733859fbe2465c386ec65b7be995b152c8c48
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-05-03 17:53:58 +02:00
Martin Trigaux 2617b3f072 [FIX] base: include model_terms in check
Translations of type model_terms are similar to model and the access
rights can be checked on the referenced record instead of
ir.translation

closes odoo/odoo#90101

X-original-commit: 992a919d09c3ba224a64241f4a6c16c1a086eb4b
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2022-04-29 13:13:00 +02:00
Denis Ledoux b03c227e88 [REF] models: refactor fields_view_get, load_views
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.

e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
  in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`

The goal of this revision is to change that so it sends the list of all fields
only once.

In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
  - `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
  - `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`

The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
  As it no longer contains the fields,
  the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
  which is a dict with as key the model name and as values
  the model fields description. It contains the fields description
  for all models implied in the view:
  the model of the main view and the model of all one2many and many2many fields.

With this change, the fields description will only be sent once by model
implied in the view.

In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.

- one2many and many2many fields views are passed directly in the main view
  architecture rather than being put in the `views` key
  of the field description.
  This is actually easier to treat by the web client,
  and this will allow in a future work to cache an entire view in one block
  of text rather than having to combine multiple cached blocks of text
  to return one view.
- one2many and many2many fields which do not have directly embedded views
  have their views directly injected in the architecture,
  so the web client doesn't have to do RPC calls to `load_views`
  for each one2many and many2many fields not having embedded views.
  For instance, this allow to reduce the number of RPC calls to `load_views`
  from 8 to 1 when loading the form of `product.product`.
  Currently, this behavior is limited to 1 level deep but we consider making it
  go all the way down in future works. We did not do it for the moment because
  in certain cases it rises the processing time and the size (bytes) too much.
  e.g. the sale.order view can be 5 levels deep,
  meaning you can reach 4 dialogs on top the main view.
  ```
  sale.order form > order_line > sale.order.line form > invoice_lines >
  account.move.line form > asset_ids > account.asset form >
  depreciation_move_ids > account.move form.
  ```
  This will also benefit in future works to cache an entire view in one block
  of text rather to having to combine multiple cached block of text
  to get one view.
- `fields_view_get` becomes `get_view`.
  As it no longer returns the fields description,
  keeping the `fields` in the name `fields_view_get` no longer makes sense.
  Hence removing `fields` from the method name, it becomes `view_get`.
  As it gets renamed anyway, we take the opportunity to rename it `get_view`,
  which is more in line with the general getter/setter guidelines
  in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
  This is not mandatory, there is no technical reason to rename `load_views` as
  it practically sends the same info as before,
  the view architectures and their fields description. Just in another way.
  We just take the opportunity of this pull request to suggest a cleaner API:
  `_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
  `_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
  in `_get_view` and `get_view`.
  The rationale is that submenu was already no longer used (deprecated)
  and the mobile options is introduced.
  The mobile options is necessary to tell the server to send the mobile views
  for x2many fields (kanban instead of tree).
  Instead of adding a new argument each time we add a new option to
  `fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
  to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
  view information. Now, `get_view` returns a tuple with the view architecture
  as an `etree` node, and the view as a browse record. The rationale is that all
  overrides of `_fields_view_get` were about modifying the arch only
  (e.g. changing the address format/re-organizing the address related field
  nodes of the partner according to the company country).
  To do so, all these overrides were doing `etree.fromstring` to parse the arch
  which was sent in text to convert it to an `etree`,
  then operations were done on the `etree`,
  and then `etree.tostring` was called to convert back the arch to string.
  With this change of signature to send the arch as an `etree`,
  all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
  allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
  has been performed in `get_view`:
  - `fields` is removed, as explained above,
  - `view_id` is renamed `id`,
  - `name` is removed, it was unused by the web client,
  - `type` is removed, it was unused by the web client,
  - `field_parent` is removed, it was unused by the web client,
  - `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
  (now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
  as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
  `fields_view_get`, `_fields_view_get` and `load_views` are provided,
  with deprecation warnings in them.

- The web client could cache the model fields description
  (as it already caches the views),
  so it doesn't need to fetch them again if it asks for another view of a model
  for which he already has the fields description.
  If we do so, `get_views` could return only the list of models used by
  the views, without the fields description as of now,
  and the web client would then call `fields_get` independently only for
  the models for which it doesn't have yet the fields description.
  This would avoid the server to return the fields description
  and to call `fields_get`, which is costly, for each `get_views`,
  therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
  unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
  This is already done for qweb views, it's not done for back-end views.
  Therefore the postprocessing of the views is performed for each `get_views`,
  which is costly, while the view architecture doesn't change for users
  belonging to the same groups, according to the groups implied by the view.

This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.

Part-of: odoo/odoo#87522
2022-04-29 09:57:44 +02:00
Raphael ColletandMartin Trigaux c490f6fef0 [FIX] base: delete orphan translations without access right errors
Deleting an orphan translation is mostly impossible, since it requires
'write' access on the corresponding records, and checking access rights
fails because the records no longer exist!

Simply enable operations on orphan translations by ignoring the
corresponding records that do not exist.  And make sure that the patch
does not allow to create orphan translations.

closes odoo/odoo#86869

X-original-commit: 454aa2587dcda2582ae58650019669ff0e8baa2c
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Martin Trigaux <mat@odoo.com>
2022-03-22 08:47:54 +01:00
a6f310fca3 [FIX] base: ignore tags in translations matching
When comparing existing translations to new source value
html tag would invalidate existing translations.
The html tags used for styling (coloring, italic, ...) should not
invalidate the translation as the meaning of the text as not changed.

Changed the logic to compare translations based on textual content only.

+ add some tests.

task-2667950

closes odoo/odoo#83895

X-original-commit: 1fe4b0cc1981382cc3d944ec44276f0aee90945d
Signed-off-by: Antoine Guenet <age@odoo.com>
Signed-off-by: Geelen Sébastien (sge) <sge@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
Co-authored-by: xmo-odoo <xmo@odoo.com>
2022-02-03 11:45:55 +00:00
Raphael Collet b840d33946 [FIX] base: translations of field labels in views
The commit e123aa2f30 optimizes the call
to fields_get() made in the NameManager to post-process the combined
arch of a view.  The optimization consists in avoiding translations,
with the assumption that the result is only used for validations.  It
turns out that the regular post-processing actually returns the field
descriptions as part of the result of fields_view_get().

We fix this mistake by keeping the optimization only when the flag
'validate' is true, so that the regular post-processing of a view
returns field descriptions with the expected translations.

closes odoo/odoo#72216

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-06-16 09:25:11 +00:00
Fabien PinckaersandRaphael Collet 085e972e4a [IMP] core: clean up and speed up XML/HTML translation
The former algorithm was recreating a full etree from scratch, while
parsing the content to translate.  Instead, we parse the etree and
update only the content that to be translated.

Before:
| In [2]: %timeit views.invalidate_cache(); views.mapped('arch_db')
| 578 ms ± 7.83 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

After:
| In [2]: %timeit views.invalidate_cache(); views.mapped('arch_db')
| 208 ms ± 2.79 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Reading a view is 2.63x faster: from 504 ms, to 191 ms (sum of all the
1345 views, when installing website_sale).  Large views, like web pages
goes up to 3x-4x faster; small views are ~2x faster.

As views are cached, it only impact the loading to put in cache, but all
XML/HTML fields get the same performance gain.

closes odoo/odoo#68182

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2021-03-23 15:35:47 +00:00
Ivan Yelizariev 3513b059e1 [FIX] odoo/fields.py: fix removing translations on writing ''
cache_value could be empty string rather than None. I'm not sure why it was
different before.

STEPS:
* install an app that adds menu Products (e.g. Sales)
* activate second language and switch to it
* create new product, set some value to field description ("Internal Notes"),
save
* click edit, remove description, save

closes odoo/odoo#64752

Before: the value is not removed
After: value is empty, translations are removed
X-original-commit: 3ef5e63f62f0f84d6e92cb55eb112ab2f2924faa
Signed-off-by: Ivan Yelizariev // IEL <yelizariev@users.noreply.github.com>
2021-01-19 15:02:08 +00:00
Raphael Collet 6319e0c0ce [IMP] base: speed up tests on translations
Simply load translations in `setUpClass` instead of `setUp`.
2020-11-24 13:23:32 +00:00
nie cc114ab1ac [FIX] base: Contacts still have deactivated language
Steps to reproduce the bug:
- Install web_studio, contacts
- Activate language English (UK)
- Switch all users' language to UK
- Deactivate language English (US)
- Via Studio, add a related field to a contact: Self > Language

Bug:
Traceback: contacts still have en_US as language and it's deactivated.

opw:2350362

closes odoo/odoo#60279

X-original-commit: 47ec3585f50f030025b39031807be50a9299bcd6
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Signed-off-by: backspac <backspac@users.noreply.github.com>
2020-10-19 13:36:27 +00:00
Raphael ColletandChristophe Simonis c99873d149 [FIX] core: improve cache consistency of translated fields
X-original-commit: 7460d65387edd9d8578e048da6ee634ef6a2e71e
Co-authored-by: Christophe Simonis <chs@odoo.com>
2020-04-30 17:30:21 +00:00
Martin Trigaux 78007d0b56 [FIX] base: insert empty translation
When creating new translations, do not prepopulate the translation
with src.
Take the following scenario:
- set a value in a translatable field in English
- open the translation popup -> creates all ir.translation
- notice an error in English, close the popup
- correct the error in English
- reopen the popup -> existing translations not modified

Add test about reseting

closes odoo/odoo#50147

X-original-commit: 619eccc7a533eedfb3aec0c3740ead1006e0f2f7
Related: odoo/enterprise#10194
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2020-04-24 12:00:24 +00:00
Nicolas Lempereur d118cefd97 [FIX] base: translate_xml update src on orig lang update
This is a changement succeeding 13.0 7eb603b26c, in that commit the issue
was solved but only in the case of en_US base language of translation
which can not be the case on a website.

opw-2153422
closes #42269

closes odoo/odoo#42467

X-original-commit: 191075454563c7e15b22d20b21afeb8bcc5601c4
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
2019-12-30 11:58:03 +00:00
Nicolas Lempereur 200045d337 [FIX] base: update en_US translation on src update
On the website when you modified a string with not much change, we
modify the translation source so they are not dropped.

But we also do it for an en_US translation, so if we have a translation:

lang=en_US/src=helloworld/value=helloworld

if we change `helloworld` by `helloworld!` the translation would become:

lang=en_US/src=helloworld!/value=helloworld

which would make it seem like the change did not work at all.

Without change, modifed test fail on second:

    self.assertEqual(view.with_env(env_us).arch_db, archf % terms_en)

with assertion:

    AssertionError: '<form string="X">Bread and cheeze</form>'
                 != '<form string="X">Bread and cheese</form>'

opw-2153422
closes #41714

closes odoo/odoo#41809

X-original-commit: 7eb603b26cc591b85ceb11210f5db8308d2ba27a
Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com>
2019-12-12 12:23:09 +00:00
Kevin Baptiste 6cbe824871 [REV] web: reverts update to fontawesome 5.11.2
This reverts commit ff1c35513a.

closes odoo/odoo#41480

X-original-commit: 116057b26e71db4692280463669f3e80d813ddcc
Related: odoo/enterprise#7110
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2019-12-09 10:33:36 +00:00
Kevin Baptiste ff1c35513a [IMP] web: update to fontawesome 4.7.0 to 5.11.2
FontAwesome 5 introduced new names for some icons as described on
https://fontawesome.com/how-to-use/on-the-web/setup/upgrading-from-version-4#name-changes

This commit replaces the old names to the new ones.

closes odoo/odoo#35826

Taskid: 2050241
Related: odoo/enterprise#5180
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2019-11-28 10:05:12 +00:00
Martin Trigaux f5e2038386 [IMP] base: use less tools method
Instead, updating the translations of a module can be done directly
on the ir.module.module record
Remove one call to _update_translation by the actual creation of the
language
2019-11-19 11:36:52 +01:00
Martin Trigaux 49fbab5329 [IMP] base: split and deprecate load_lang
load_lang was a kind of hybrid method trying to active or creating a
language if not found. This was error prone.
Instead rely on two methods with clear purpose:
ResLang._create_lang(lang, lang_name=None)
  - create a new res.lang entry using the locale of the server
    return the res.lang record to match the API of _activate_lang

ResLang._active_lang(code)
  - activate the given code lang

Most of the time, _active_lang is what is expected

tools.trans_load_data and IrTranslation._load_module_terms no longer
activate the language if not active.
Loading the translations should be explicit on an activated language,
it is too error prone to silently activate/create a language if not
found.
Remove lang_name from trans_load_data as no longer needed.
2019-11-19 10:37:01 +01:00
Nans Lefebvre 4dcae232f3 [FIX] fields, test_translate: multilanguage without en_US
Before, en_US was almost always the default in case of 'no language'.
Now, that should be superseded by lang=None;
yet still, en_US plays the role of source language.

In a multilanguage database where en_US was not installed,
writing a translation with lang=None would crash.
This could prevent module installation.

A test is added to cover that case.

In the case where we write a record with lang=None, the source is the
same as the record; in that case the cache contains the value for the
field.name under the context keys en_US and None, so we need to
invalidate the cache to avoid getting back the old value.

co-authored with mart-e

opw 2088487
opw 2083710

closes odoo/odoo#39130

X-original-commit: 1a5a999c2dd69c9a6a2dacb59e8bf45b09c6bc13
Signed-off-by: Nans Lefebvre (len) <len@odoo.com>
2019-10-21 14:31:10 +00:00
Christophe Simonis 64e43808b7 [MERGE] forward port branch saas-12.5 up to 58a83d1222 2019-09-20 17:34:45 +02:00
Martin Trigaux 543a5523a3 [FIMP] base: remove all translations when writing False
This "bugprovement" resolve the old issue of writing on a translatable
field a falsy value.

Before this commit, only the translation value was removed. When no
translation is present (or when the value is not set), the value is
retrieved directly from the source field.

Before 18d9c2cab2, it was still possible to go around this issue by
removing manually all ir.translation entries and writing with a user
in en_US.
This is no longer possible as writing on a translatable field always
uses translations when in multi-language environment, without
exception of en_US.

When writing a falsy value on a field, the expected behaviour is to
reset the value and see that falsy value after saving, not the old
value stored on the source model.

Remove all translations and force to update the column.

Removing one individual translation is still possible in the
translation popup.

Task id: 2062415

closes odoo/odoo#37077

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2019-09-18 14:56:41 +00:00
Julien Castiaux 4f03a5f136 [FIX] *: remove old deprecated modules/functions
PEP-594 is deprecating a bunch of modules. As part of the cleanup, we
are also dealing with long deprecated modules, functions and aliases.

* `assert_` -> `assertTrue`
* `assertEquals` -> `assertEqual`
* `assertNotEquals` -> `assertNotEqual`
* `assertAlmostEquals` -> `assertAlmostEqual`
* `assertRaisesRegexp` -> `assertRaisesRegex`
* `assertRegexpMatches` -> `assertRegex`
* `base64.encodestring` -> `base64.encodebytes`
* `base64.decodestring` -> `base64.decodebytes`
* `inspect.getargspec` -> `inspect.signature`
* `inspect.formatargspec` -> `inspect.signature`
* `logging.warn` -> `logging.warning`

closes odoo/odoo#36863

Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-09-17 11:36:42 +00:00
Raphael Collet 9920f20e4c [IMP] models: ORM speedup
This branch is the combination of several optimizations in the ORM:

* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;

* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;

* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);

* make method `modified` take advantage of inverse fields to inverse
dependencies;

* filter records by evaluating a domain on records in Python;

* a computed field with `readonly=False` behaves like a normal field
with an onchange method;

* computed fields are computed in superuser mode by default.

Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.

closes odoo/odoo#35659

Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
2019-08-20 12:43:59 +00:00
Martin TrigauxandRaphaël Collet 7593b887df [REF] fields: use ir.model.fields.selection
The selection values of a selection field are now stored in database in the
model ir.model.fields.selection

This will allow to have a modular approche on selections and each selection
is now linked to the module that declared it.
Previously to this change, the selections were linked to the field, meaning
uninstalling a module had no impact on the selections stored on database.

With this change, the selections will now be translated in the correct module
(having an external id) and the records having a used selection will now be
reset to null.

closes odoo/odoo#30228

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>


Co-authored-by: Raphaël Collet <rco@odoo.com>
2019-08-19 11:44:34 +00:00
Martin Trigaux 18d9c2cab2 [FIX] base: single lang behaviour
If a db is in single lang (en_US or not) but still has translations, the
translation should be updated in addition to updating the referenced record
Add tests to formalise the expected behaviour:

If one language only (en_US or not), when writing on a translatable field:
- record field should updated
- value of potential existing en_US translation should be updated
- src of potential existing translations should be updated
- no new translation should be created

If en_US and fr_FR, when writing on a translatable field in en_US:
- record field should updated
- value of potential existing en_US translation should be updated
- src of potential existing translations should be updated
- new en_US translation should be created if was not present

If en_US and fr_FR, when writing on a translatable field in fr_FR:
- record field should not be updated
- value of potential existing fr_FR translation should be updated
- src of potential existing translations should not be updated
- new fr_FR translation should be created if was not present

Adapt test_new_api test
get_installed is ormcached, just putting active = True is not enough
2019-08-01 09:06:25 +00:00
Martin Trigaux 66ef641f33 [REF] base: remove field source of ir.translation
Instead, make sure the field src is always up to date
Add tests

Changes in _write:
- Replace _set_ids (to be deprecated) call by _upsert_translations as it works
  in batch
- Call _upsert_translations for any language, including en_US
  In case an English translation already existed for a record, only the master
  value (on the reference record) was updated but the user kept seeing the
  translation value (was revealed 73a7534bfc).
- Read src_trans without language
  Similar as above, if an English translation already existed, the translation
  was used for the result of the read and not the new value that has just been
  inserted into the database
- Add _set_source method
  When updating a master record of a translated field, the src field must be
  updated, including in different languages.
  Before it was ok that the src field was out of date as the source was computed

Changes in copy_translation:
- set src as the new value without lang
  update the comment to reflect reality since 489494e733
  src will contain the English version if no changes were made and it will
  contain the modified value if copy was overriden

Changes in upsert_translations:
- Do not force a module, comment, state
  Only src, res_id, name, value and lang must be given. Optional values will no
  longer be set to null if not given
2019-08-01 09:06:25 +00:00
Martin Trigaux 5d57e1862f [MERGE] Forward port of saas-12.4 to master up to beba36416f
closes odoo/odoo#34924

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2019-07-16 15:45:24 +00:00
Martin Trigaux cf8e74a1a7 [MERGE] Forward port of 12.0 to saas-12.2 up to 1bd434da34
closes odoo/odoo#34901

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2019-07-16 09:14:17 +00:00
Xavier-Do 95b4f2ab4b [IMP] core: improve test_tags
Add the support to be able to specify exlicitely module, class and method name in test_tags

Before this commit, there is no distinction between module tag and test tag. Meaning
that --test-tag module_name will replace the default +standard tag and execute all test
of the module.

This commit tries to keep the initial behaviour, while addind new features, like explicit
modules tag with /module_name, as well as class (:class) and method (.method)

Some usage examples:
--test_tags /module will execute all standard test of module
--test_tags :class will execute all standard test with class name 'class'
--test_tags .method will execute all standard test with method name 'method'
--test_tags external/module will execute all external test of module
--test_tags */module will execute all tests of module,
--test_tags */module,-standard will execute all non standard tests of module,
--test_tags -/website:TestUiTranslate.test_admin_tour_rte_translator will disable only rte translator test

closes odoo/odoo#34756

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-07-11 09:03:14 +00:00
Raphael Collet 038001a293 [FIX] base: method load_module_terms not aimed at being public 2019-07-08 13:51:35 +00:00
Martin Trigaux 73a7534bfc [IMP] base: display field translation in a popup
The goal of this change is to allow the user to stay in the form view and edit
the translations in one go.

Generate also English translations to make the source=English a bit less true.
It was already the case for callable translations method

Adapt the test that was ambigious (verified no French translation was duplicated
but an English translation is created)
2019-07-02 09:53:08 +00:00