Commit Graph
120 Commits
Author SHA1 Message Date
Xavier Morel 9ebbfdac73 [FIX] *: incorrect translations markings
Fixes a large number of cases where strings are translated then
formatted, instead of letting `_()` do the formatting internally,
which allows it to recover from incorrect translations (missing,
broken, or extra placeholders).

Also

- removes translation markers entirely when there's nothing to
  translate e.g. `_("%s - %s")` is not useful
- fixes a few messes which lead to only partial translatability
  (DRY is generally a bad idea when translations are involved, even
  more so when you don't make the variable part translatable)
- fixes a few nearby issues noticed at the same time
- replaces a few `"%s"` by `%r`, which should automatically quote
  strings relatively appropriately
- fixes translated strings which use `\` to escape a newline (in order
  to fill-paragraph): `\` escapes only the newline, if the
  continuation string is indented this results in a bunch of spaces
  ending in the string to translate, which is pretty garbage for the
  translator, using implicit concatenation works much better

Note: some of the updates revert f-string parameters to %, because
babel (2.9) apparently has trouble with f-strings and blows up trying
to extract them.

Not in scope:

Helping translators fix translatable strings e.g. any translation
string with more than one placeholder probably should use keyword
placeholders

- Provides more context / data to the translator to make sense of the
  sentence.
- Allows reordering the translated terms, which can be necessary
  depending on the sentence and language.

closes odoo/odoo#139314

Related: odoo/enterprise#49311
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-10-23 16:45:09 +00:00
Martin Trigaux 75858d3509 [IMP] *: use correct indentation in manifest summary
The summary is a short char field. It should not contain carriage
returns.
The description is the longer text field.
Remove unnecessary spaces in both.

Automatically dedent the description to avoid this issue poping up
again in future modules.

closes odoo/odoo#138214

Related: odoo/enterprise#48695
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2023-10-11 08:18:49 +00:00
Martin Trigaux 22ab49e343 [IMP] *: use file_path and file_open
Replace all the calls to get_resource_path to the better file_path or
directly use file_open when not needed

Doing both a get_resource_path and file_open means checking twice that
the file exists.
Doing a simple path concatenation before a file_open is safe.
If given to another method (e.g. etree.parse), calling file_path is
the prefered method.

Note that get_resource_path used to return False when the file does
not exists while file_path/file_open raises a FileNotFoundException

closes odoo/odoo#135607

Related: odoo/upgrade#5187
Related: odoo/enterprise#47475
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2023-10-06 14:33:43 +00:00
Chong Wang (cwg) 007d2bde2a [REF] translation: better get_po_paths
make the tool function get_po_paths to reduce duplicated code

Part-of: odoo/odoo#134785
2023-09-18 22:17:02 +00:00
niyasraphy 251e558ff2 [IMP] base: remove activate module server action
before this commit, from list view users can install
module using the button in list view and from the
action button.

initially the Install button was not available in the
list view and only option to install multiple apps was
from the action button.

but with the introduction of the button in list header
there is no need for an another server action to
perform the same.

after this commit, the activate modules server action
will be removed from the code and its related test
and also newly added Install button will be renamed
to "Activate" to align with the button in kanban and
form.

closes odoo/odoo#133544

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-04 05:49:50 +00:00
Xavier Morel 8d06889ec3 [FIX] base: encoding guessing of html module descriptions
I missed a critical issue in #133708: various users had discovered
they could already fix description issues by adding an XML declaration
to their document which is very cool (though technically not really
valid).

What is a lot less cool is that lxml gets *extremely* unhappy when
asked to parse *strings* with an encoding declaration, raising a
ValueError, so the purported fix breaks on any module which does that,
which seems to include a lot of OCA modules.

Gate the encoding guessing by bailing if the document has an XML
declaration, in which case we just assume the author knows what
they're doing and we leave them alone. For extra safety, check the
encoding declaration in ascii and utf16. Could also have checked for
BOMs, but lxml seems to not care about them overly much (in fact it
seems to prefer them decoded which is odd).

Also same as non-utf8 descriptions, mark XML declarations as
deprecated (because it's a hack to make UTF8 descriptions work which
is not necessary anymore).

closes odoo/odoo#133968

Reported-by: @rezak400
X-original-commit: fd353d7d0104431208b91603431e41ef4a6e54bb
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-01 16:46:03 +00:00
Xavier Morel 431b4e69e3 [FIX] base: correctly parse utf8 html module descriptions
Apparently `lxml.html.document_fromstring` (and possibly other
`lxml.html` loaders) parses byte-strings as latin1 regardless of their
actual encoding, maybe because python2, maybe because there's a super
legacy html4 parser underlying it.

Either way that means ever since loading
`static/description/index.html` files was added 10 years
ago (4bf6a7ea4c) `_get_desc` has been
loading these files in latin1 rather than the utf8 most people would
expect.

Add an explicit decoding phase to try and load html description files
in UTF8. Fall back to latin1 in case there are description files which
are genuinely in latin1, or even just some random-ass broken stuff
which very much isn't utf8 (the extended-ascii encodings -- of which
latin1 is one -- will happily accept and mangle any input as every
byte value is valid, utf8 is a lot more structured).

Closes #127846

closes odoo/odoo#133859

X-original-commit: 4dbc3b00e587f3d64cfd964a685f2bddd1b499ad
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-09-01 11:39:56 +00:00
Martin Trigaux 35a118eff5 [IMP] base: move helpers to tools
closes odoo/odoo#67316

Signed-off-by: Rémy Voet (rvy) <rvy@odoo.com>
2023-06-08 12:11:07 +02:00
Martin Trigaux f4d5752724 [IMP] tools: load also es_MX
The translations on Spanish (Mexico) are more active than on Spanish
Moreover, most other countries speaking a variant of Spanish are
located in Latin America and are closer to es_MX than es.
To solve this, when installing for instance es_AR, the translations
will be loaded in the following order:
1 es
2 es_MX
3 es_AR

In case a term is translated in both es and es_MX (and not es_AR), the
later will be used

closes odoo/odoo#121415

Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2023-07-28 17:53:53 +02:00
Xavier-Do 595aa24843 [IMP] registry: multiple ormcache
One of the main issue with ormcache is that the invalidation clears
everything, meaning that some value, slow to compute but with a long
lifetime, can be removed from the cache because an easy to invalidate
value is cleared, like after writting or creating a product has an
example.

Most example in the code will try to invalidate the cache of the models
doing something like `env['ir.qweb'].clear_caches()` but it is
finally equivalent to `env.registry.clear_cache()`, and cross worker.

The idea is to have multiple cache, maybe with specific sizes for a
specific purpose.

Having one per model is maybe a bad idea because it will be difficult
to size the LRU correcly, and it is too dynamic. Checking invalidation
may be expensive.

The proposed solution is closed allow a limited number of named caches,
using onse sequence per cache. This is actually close to the
cache_longterm.

We want to discourage using a specific cache for one use case in
the buisness code. Adding a cache shouldn't be something easy, doable
in stable.

Note that we could also change the invalisation mecanism using an
insert only table. We an check the sequence of this table, but also
fetch all invalidation messages.
Another possible improvement, especially if we have more than x cache is
to have a global sequence, checking signaling would mean to check the
main sequence, and only the other ones if the main one changed.

Note that this poc is inspired from the long term cache but not all
use case where applie yet.

Part-of: odoo/odoo#119813
2023-07-18 11:42:26 +02:00
Saurabh Choraria 2476a5b1bb [FIX] base: prevent traceback when icon file is not found
When user tries to import a module which does not consist of an icon and then
when user tries to access that module the error occurs.

Steps to reproduce:
1. Install web_studio.
2. Import a module(without icon) from apps > import module menu.
3. Now search that module in apps and click on module info.
4. The error will occur.

Applying this commit will fix this issue.

sentry-4206999375

closes odoo/odoo#124540

X-original-commit: 4b0221800ab75d5573e7066bf8beee14488062dd
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-06-12 12:41:15 +02:00
Julien Castiaux c207b40b51 [FIX] base: reniew request registry after uninstall
Install and then uninstall the utm module via the web client, you get a
traceback because the ir.http override of the utm module is still
present in the registry altought the module is not installed anymore.

The problem affects all modules that override the _post_dispatch method
of ir.http, it is not limited to UTM.

The problem is that, after the uninstallation, a new registry (without
the uninstalled modules) is created but the old registry was still used
by the HTTP stack.

closes odoo/odoo#122519

Closes: #121755
X-original-commit: 979844600a0bc5ea8b63cf4d58c7aba5133e82f3
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
2023-05-25 19:16:40 +02:00
william-andre 9f13817425 [IMP] base: allow to add country flags on module kanban
The icons made for localization require tedious manual work where it
could be done easily with some css.

task-3166075

Part-of: odoo/odoo#108617
2023-05-10 04:14:49 +02:00
Xavier Morel c7826675a8 [FIX] base: restore deletion of dependencies on field removal
odoo/odoo#111651 improved and optimised triggers but dropped the
in-place cleanup of field dependencies. As a consequence, during
module uninstallation if a stored computed field is removed (because
it's part of a module being uninstalled), and one of its dependencies
is subsequently altered (e.g. it's itself removed, or written to) the
second update will break as the query trying to find out which
dependent records to update will error, either because of trying to
select / filter on a missing column, or because of trying to fetch
in a missing table.

The simplest examples of this issue are computed fields with a
dependency on `ir.model`:

- In `calendar`, `calendar.event.res_model` is a related on
  `res_model_id.model`, this prevents the removal of *any* `ir.model`
  record if it gets uninstalled.

  As a result the `calendar.attendee` and `calendar.event` tables
  don't get removed (just emptied of all their non-automatic fields),
  their records remain as well, and when the non-automatic fields get
  re-added during installation re-instating the NOT NULL constraints
  fails, breaking the uninstall/reinstall test.

- In `payment`, `payment.provider.module_state` is a related on
  `module_id.state`, this breaks *during* uninstallation, as after
  `module_uninstall` first calls `_module_data_uninstall` which
  removes the field, then it *updates the modules being uninstalled*
  (sets their state), which tries to find out which
  `payment.provider`'s `module_state` is should update, which breaks
  because the `module_id` column has been removed.

  This second one was worked around in odoo/odoo#118900, by marking
  modules as uninstalled before actually gutting them, but as it turns
  out the "actual" fix is needed anyway. So revert the workaround, and
  actually fix the issue.

closes odoo/odoo#119130

X-original-commit: 48a420efcf7c7b46416bab006003553cc9d23846
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-20 09:33:48 +02:00
Xavier Morel 98b5505305 [FIX] base: mark modules as uninstalled before gutting them
Before this, uninstalling the `payment` module (or any of its
dependencies) is broken: as payment.provider has a dependency on
ir.module.module.state, marking the modules causes a lookup of the
payment provides to update, but the table was removed by
`_module_data_uninstall`, so the lookup blows up.

This is a consequence of odoo/odoo#111651 which improved and optimised
triggers but dropped the in-place cleanup of the triggers tree.

Thus while the columns & tables get removed from the database the
in-memory structures (registry, models, fields, ..., as well as the
trigger and dependency caches) are not so the python side will happily
try to look up stuff which has been nuked if accessed at the wrong
moment (which is any moment between the start of
`_module_data_uninstall` and the creation of a new registry, really).

As `_module_data_uninstall` is nothing but a giant pile of dodgy state
anyway, making modules as uninstalled before it executes doesn't seem
like a huge deal. It may cause unnecessary extra recomputation for the
few models which depend on modules, but that doesn't seem like a major
issue, at worst it makes uninstallation a touch slower but they're not
a huge performance concern at the moment (they're more of a
correctness one).

closes odoo/odoo#118959

X-original-commit: 460efeb623ec62c980d187710bd4e7614af0e7bd
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2023-04-19 12:13:32 +02:00
Louis Wicket (wil) 9afe7c74c9 [IMP] *: remove "French spacing" 👺
According to Wiktionary, French spacing is "the archaic practice (though
still current in French) of inserting a space around colons, semicolons,
question marks, and exclamation marks". This is not standard practice in
English and most languages of the world.

The purpose of this commit is to start purging the code from this typo,
as it may reflect poorly on the software for some people.

closes odoo/odoo#114533

Related: odoo/enterprise#37853
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
2023-03-14 15:52:10 +01:00
Rémy Voet (ryv) 880daf6e59 [REM] base: remove useless module_nr field of ir.module.category
The last usage was an unused tree, previously removed.

closes odoo/odoo#109420

Related: odoo/enterprise#35586
Related: odoo/upgrade#4192
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
2023-01-10 18:54:48 +01:00
niyasraphy 4c43882f30 [FIX] base: prevent uninstalling of web module
closes odoo/odoo#108088

X-original-commit: 688acadeec159d713f75384d35d22e6c88f4b00a
Signed-off-by: Julien Castiaux <juc@odoo.com>
2022-12-16 13:48:25 +01:00
Chong Wang (cwg) a0bd7d0d72 [IMP] core: improve translation import speed
this commit improves the performance of importing translations by
1. groupup udpating data for different languages and different modules
2. read model_terms translations data directly with xmlid to get rid of fetching
   data from ir_model_data

for modules:sale_management, point_of_sale, industry_fsm, helpdesk, mrp, mrp_plm
(62 installed modules)
time for test_language_install is 45.14% less

for all modules:
time for test_language_install is 68.03% less

closes odoo/odoo#107332

X-original-commit: 727ef9e0a2ca93089b472eb1313117fee8dc876d
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-12-06 17:12:21 +01:00
Ivan Yelizariev 0339a506f8 [REM] base,web: delete menu Apps Store, Updates
Those menus are supposed to be integrations with apps.odoo.com, but it doens't
work and not maintained.

Existing menus that open apps store links should be enough:

* Apps >> Apps >> Theme Store --> https://apps.odoo.com/apps/themes
* Apps >> Apps >> Third-Part Apps --> https://apps.odoo.com/apps/modules

---

close #72902
close #74602

closes odoo/odoo#106379

Related: odoo/enterprise#34324
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2022-11-24 13:28:55 +01:00
Raphael Collet f0ca3f32d2 [FIX] core: ignore imported modules when loading registry
The existing code was generating misleading errors for imported Odoo
modules that could not be loaded.  Although there was a specific hack
for module 'studio_customization', imported modules were not handled
properly.  This patch adds the right condition in the SQL query in the
module that introduces imported modules.

closes odoo/odoo#106098

X-original-commit: e1cfc1f74000e55473f7a26f0df5a13c4d5094c0
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
2022-11-21 21:18:53 +01:00
niyasraphy 32073ef3da [FIX] base: name search for ir_module_module model
name, shortdec, summary fields are added to name search based on the fields set in the filter domain of existing search view of apps menu

closes odoo/odoo#105457

X-original-commit: a33165a78ab888b8c690b5424ed997b94ee8be05
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-11-09 16:08:10 +01:00
Yannick Tivisse 7f443e8542 [IMP] base: Prevent button_immediate_install on non loaded registries
If called in cascade, this could lead to a situation in which a same
thread is trying to create recursively several registries when loading
the sames modules, which obviously cannot work in any way.

Use button_install instead, that will simply update the module state
while waiting to be installed in a proper way, instead of forcing
a registry creation that will mess the whole process.

X-original-commit: 6b52844a8dc78788cbd948c7e2f90156ee8b89d7
Part-of: odoo/odoo#101086
2022-09-26 12:39:46 +02:00
ef00294e71 [IMP] core: store translated fields as JSONB columns
Translated fields no longer use the model ir.translation.  Instead they store
all their values as JSON, and store them into JSONB columns in the model's
table.  The field's column value is either NULL or a JSON dict mapping language
codes to text (the field's value in the corresponding language), and must
contain an entry for key 'en_US' (as it is used as a fallback for all other
languages).  Empty text is allowed in translation values, but not NULL.

Here are examples for a field with translate=True:

    NULL
    {"en_US": "Foo"}
    {"en_US": "Foo", "fr_FR": "Bar", "nl_NL": "Baz"}
    {"en_US": "Foo", "fr_FR": "", "nl_NL": "Baz"}

Like before, writing False to the field makes it NULL, i.e., False in all
languages.  However, writing "" to the field makes its value empty in the
current language, but does not discard the values in the other languages.

Here are examples for a field with translate=xml_translate:

    NULL
    {"en_US": "<div>Foo<p>Bar</p></div>", "fr_FR": "<div>Fou<p>Barre</p></div>"}

Change for callable(translate) fields: one can now write any value in any
language on such a field.  The new value will be adapted in all languages, based
on the mapping of terms between languages in the old values.  Basically the
structure of the value must remain the same in all languages, like before.

Reading a translated field is now both simpler and faster than the former
implementation.  We fetch the value of the field in the current language by
coalescing its value with the 'en_US' value of the field:

    SELECT id, COALESCE(name->>'fr_FR', name->>'en_US') AS name ...

The raw cache of the field contains either None or a dict which is conceptually
a subset of the JSON value in database (except for missing languages).  For the
sake of simplicity, most cache operations deal with the dict and return the text
value in the current language.

Trigram indexes have been adapted to the new storing strategy, and should enable
to search in any language.  Before this change, only the source value of the
field ('en_US') could be indexed.

Computed stored translated fields are not supported by the framework, because of
the complexity of the computation itself: the field would need to be computed in
all active languages.  We chose to not provide any hook to compute a field in
all languages at once, and the framework always invokes a compute method once to
recompute it.

Code translations are no longer stored into the database.  They become static,
and are extracted from the PO files when needed.  The worker simply uses a cache
with extracted code translations for performance.  This is reasonable, since
fr_FR code translations for all modules takes around 2MB of memory, and the
cache can be shared among all registries in the worker.  Changing code
translations requires to update the corresponding PO file and reloading the
worker(s).

Performance summary:
 (+) reading 'model' translated fields is faster
 (+) reading 'model_terms' translated fields is much faster (no need to inject
     translations into the source value)
 (+) searching translated fields with operator 'ilike' is much faster when the
     field is indexed with 'trigram'
 (+) updating translated fields requires less ORM flushing
 (-) importing translations from PO files is 2x slower

Some extra fixes:
 - make field 'name' of ir.actions.actions translated; because of the PG
   inheritance, this is necessary to make the column definition consistent in
   all models that inherit from ir.actions.actions.
 - add some backend API for the web/website client for editing translations
 - move methods get_field_string() to model ir.model.fields
 - move _load_module_terms to model ir.module.module
 - adapt tests in test_impex, test_new_api
 - because env.lang is injected into SQL queries, its returned value is
   now guaranteed to correspond to a valid active language or None
 - remove wizard to insert missing translations (no longer makes sense)

task-id: 2081307

Co-authored-by: Fabien Pinckaers <fp@openerp.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-09-15 22:37:50 +02:00
Fabien Pinckaers 781a771295 [IMP] remove duplicated index
The following indexes were in double:
- ir_module_module_name_index, ir_module_module_name_uniq
- ir_model_obj_name_uniq, ir_model_model_index
- ir_config_parameter_key_index, ir_config_parameter_key_uniq
- decimal_precision_name_index, decimal_precision_name_uniq
- account_edi_proxy_client_user_id_client_index, account_edi_proxy_client_user_unique_id_client
- mail_channel_rtc_session_channel_member_id_index, mail_channel_rtc_session_channel_member_unique
- loyalty_card_code_index, loyalty_card_card_code_unique

Part-of: odoo/odoo#99795
2022-09-08 12:46:50 +02:00
Denis Ledoux 2dccc0d031 [IMP] base: faster get_bindings
The cache key of _get_bindings was not super efficient.
The result of _get_bindings is cached,
but its performance was altered by the cache
key which requires to fetch the user groups for each call to
_get_bindings.
Besides, as there is a lot of possible group
combination, this resulted in a lot of possible cache keys,
and therefore a lot of cached values.

This revision aims to make _get_bindings more efficient
by:

- do not use the groups in the cache keys (less cached values)
- filter out actions not available to the user groups after
  retrieving them from the cache
- use has_group to do the above, which is itself cached as well,
  and therefore do not need to fetch the user groups
  at each call to get_bindings.

In addition, move get_bindings from `get_view`
to `get_views`. If there was 3 views asked by `get_views`
(let's say kanban, list, form)
`get_bindings` was being called 3 times, through `get_view`
with each time the same arguments and therefore the same result :-).
Moving it to `get_views` allows to call it only once for all view types
requested, and for the web client it doesn't change much,
as it always request the toolbar/get_bindings through `get_views` only.

In addition, add the lang to the cache of _get_bindings.
it was actually a bug not to put it: if you had 2 users
with the same group set, using 2 different languages,
the user accessing first the get_bindings would cache
the action names within his language, and then the second
user would see the action name within the language of the first user
:-).

Before
```py
In [1]: %time for i in range(1000): self.env['ir.actions.actions'].get_bindings('res.partner'); self.env.invalidate_all();
CPU times: user 790 ms, sys: 104 ms, total: 893 ms
Wall time: 1.7 s
```

After
```py
In [1]: %time for i in range(1000): self.env['ir.actions.actions'].get_bindings('res.partner'); self.env.invalidate_all();
CPU times: user 23.5 ms, sys: 9.12 ms, total: 32.7 ms
Wall time: 36.9 ms
```

Part-of: odoo/odoo#99417
2022-09-05 16:54:01 +02:00
Vincent Schippefilt 9a5d575406 [IMP] base: unused fields in ir_module_dependency
The table ir_module_dependency was created manually with tracking fields,
but the CRUD was done manually, without the ORM.
Removing the fields that are always null will not have any impact
whatsoever but we should still do it to be correct.

closes odoo/odoo#89268

Related: odoo/upgrade#3455
Signed-off-by: Rémy Voet <ryv@odoo.com>
2022-06-16 18:31:56 +02:00
Raphael Collet 3ae2c7db72 [FIX] base: avoid invalidation that slows down updating module dependencies
closes odoo/odoo#92356

Signed-off-by: Rémy Voet <ryv@odoo.com>
2022-05-27 16:42:55 +02:00
Raphael Collet 60a1452a40 [REF] base: adapt code to new flush API
Part-of: odoo/odoo#87527
2022-05-25 18:00:47 +02:00
Christophe Monniez 65c8814a2f [FIX] various: replace deprecated currentThread method
CurrentThread is now really deprecated in Python 3.10 ... Time to
change.

Part-of: odoo/odoo#91927
2022-05-23 08:29:52 +02:00
Denis Ledoux b03c227e88 [REF] models: refactor fields_view_get, load_views
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.

e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
  in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`

The goal of this revision is to change that so it sends the list of all fields
only once.

In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
  - `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
  - `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`

The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
  As it no longer contains the fields,
  the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
  which is a dict with as key the model name and as values
  the model fields description. It contains the fields description
  for all models implied in the view:
  the model of the main view and the model of all one2many and many2many fields.

With this change, the fields description will only be sent once by model
implied in the view.

In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.

- one2many and many2many fields views are passed directly in the main view
  architecture rather than being put in the `views` key
  of the field description.
  This is actually easier to treat by the web client,
  and this will allow in a future work to cache an entire view in one block
  of text rather than having to combine multiple cached blocks of text
  to return one view.
- one2many and many2many fields which do not have directly embedded views
  have their views directly injected in the architecture,
  so the web client doesn't have to do RPC calls to `load_views`
  for each one2many and many2many fields not having embedded views.
  For instance, this allow to reduce the number of RPC calls to `load_views`
  from 8 to 1 when loading the form of `product.product`.
  Currently, this behavior is limited to 1 level deep but we consider making it
  go all the way down in future works. We did not do it for the moment because
  in certain cases it rises the processing time and the size (bytes) too much.
  e.g. the sale.order view can be 5 levels deep,
  meaning you can reach 4 dialogs on top the main view.
  ```
  sale.order form > order_line > sale.order.line form > invoice_lines >
  account.move.line form > asset_ids > account.asset form >
  depreciation_move_ids > account.move form.
  ```
  This will also benefit in future works to cache an entire view in one block
  of text rather to having to combine multiple cached block of text
  to get one view.
- `fields_view_get` becomes `get_view`.
  As it no longer returns the fields description,
  keeping the `fields` in the name `fields_view_get` no longer makes sense.
  Hence removing `fields` from the method name, it becomes `view_get`.
  As it gets renamed anyway, we take the opportunity to rename it `get_view`,
  which is more in line with the general getter/setter guidelines
  in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
  This is not mandatory, there is no technical reason to rename `load_views` as
  it practically sends the same info as before,
  the view architectures and their fields description. Just in another way.
  We just take the opportunity of this pull request to suggest a cleaner API:
  `_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
  `_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
  in `_get_view` and `get_view`.
  The rationale is that submenu was already no longer used (deprecated)
  and the mobile options is introduced.
  The mobile options is necessary to tell the server to send the mobile views
  for x2many fields (kanban instead of tree).
  Instead of adding a new argument each time we add a new option to
  `fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
  to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
  view information. Now, `get_view` returns a tuple with the view architecture
  as an `etree` node, and the view as a browse record. The rationale is that all
  overrides of `_fields_view_get` were about modifying the arch only
  (e.g. changing the address format/re-organizing the address related field
  nodes of the partner according to the company country).
  To do so, all these overrides were doing `etree.fromstring` to parse the arch
  which was sent in text to convert it to an `etree`,
  then operations were done on the `etree`,
  and then `etree.tostring` was called to convert back the arch to string.
  With this change of signature to send the arch as an `etree`,
  all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
  allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
  has been performed in `get_view`:
  - `fields` is removed, as explained above,
  - `view_id` is renamed `id`,
  - `name` is removed, it was unused by the web client,
  - `type` is removed, it was unused by the web client,
  - `field_parent` is removed, it was unused by the web client,
  - `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
  (now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
  as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
  `fields_view_get`, `_fields_view_get` and `load_views` are provided,
  with deprecation warnings in them.

- The web client could cache the model fields description
  (as it already caches the views),
  so it doesn't need to fetch them again if it asks for another view of a model
  for which he already has the fields description.
  If we do so, `get_views` could return only the list of models used by
  the views, without the fields description as of now,
  and the web client would then call `fields_get` independently only for
  the models for which it doesn't have yet the fields description.
  This would avoid the server to return the fields description
  and to call `fields_get`, which is costly, for each `get_views`,
  therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
  unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
  This is already done for qweb views, it's not done for back-end views.
  Therefore the postprocessing of the views is performed for each `get_views`,
  which is costly, while the view architecture doesn't change for users
  belonging to the same groups, according to the groups implied by the view.

This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.

Part-of: odoo/odoo#87522
2022-04-29 09:57:44 +02:00
Stefan RijnhartandRaphael Collet 4627e83d4b [FIX] base: updating 'base' should update all modules
If an installed module is only present in the dependency graph through a new,
uninstalled dependency, it was not be selected for upgrade when updating the
`base` module (c.q. `--update all`)

closes odoo/odoo#85704

X-original-commit: 6c80e7cfeafaaa4fb7c0d93cbe94c66068fc3170
Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2022-03-03 11:04:16 +00:00
Yannick TivisseandVictor Feyens 18952cdc76 [IMP] *: Convert single create method into multi
Taskid: 2703085
Part-of: odoo/odoo#80824
Co-authored-by: Victor Feyens <vfe@odoo.com>
2021-12-14 19:13:18 +00:00
Julien Castiaux 2e29a93503 [REF] core: remove http.addons_manifest
The http.addons_manifest is a map {module: manifest_dict} that is
populated upon the first http request. This map is basically a module
manifest cache with an extra `addons_path` key, the path of the module
on the file-system. This cache is eagerly populated upon the first http
request, the map is empty in non-http contextes (e.g. cron) which have
been a source of bugs (e.g. 50c8eb1).

A manifest cache is necessary because reading and parsing python files
from the file-system is not that cheap but there is no reason that cache
is located in `odoo.http`. A thin cache layer now wraps
`load_information_from_description_file()`/`load_manifest()` and is
lazily populated.

The `http.addons_manifest` have been removed. The extra `addons_path`
key is now present in the "normal" manifest. The `read_manifest()` was
hardly used so it has been deprecated. The only way to retrieve a
manifest is now `load_information_from_description_file()` which was
renamed `load_manifest()` (no cache) and `get_manifest()` (cache).

Side note about performances, the cache is necessary. Addons manifest
are read-only and reading + parsing python files from the file system is
not a cheap operation. Running the e-commerce tour
`@website_sale.test_04_admin_website_sale_tour` without cache on
`load_manifest()` requires 68,29 secs to complete on my laptop,
exceeding the default 1-minute time frame allowed in tests. Using a
cache the time is down to 36,53 secs. The performance impact is huge.

Part-of: odoo/odoo#79977
2021-12-14 12:39:54 +00:00
Victor Feyens 34dd6a2dba [IMP] base: provide a cached method to fetch module records
Following the API for ir.model records, the same logic is provided for
ir.module.module records.

This allows to avoid a bunch of queries when modifying/opening settings.

Part-of: odoo/odoo#62249
2021-11-26 09:44:57 +00:00
Raman Salihi 4cad597041 [FIX] base: use display_name instead of name in assert_log_admin_access
`assert_log_admin_access` uses the `name` field of `ir.module.module` for logging.
`assert_log_admin_access` was added to `install_demo` method of `ir.demo` in d345fb649e3e3c1bf141dca49b45503b8b3a7164 but `ir.demo` does not have `name` field, hence causing AttributeError.
The proposed solution here is to use `display_name` instead

closes odoo/odoo#79398

X-original-commit: 9d6f54e7f7219cbbda41129e35682f6a76aae1d4
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Raman Obaid Ahmed (raob) <raob@odoo.com>
2021-11-05 11:00:54 +00:00
Raphael Collet 3a4d518303 [FIX] core: missing cursor commit in module installation
Right after some module operation (install, uninstall), the framework
looks for some ir.actions.todo to execute.  Currently those actions
cannot be found, because the cursor seems to see the database in the
state it was just before the operation.  Simply add the commit() which
was removed by revision 1595c0ee27.

closes odoo/odoo#75980

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-09-06 12:03:42 +00:00
Raphael ColletandXavier Dollé 1595c0ee27 [REF] core: replace thread-local "envs" by cursor-bound "transaction"
Refactor the Environments object into a Transaction object, which is
bound to one cursor, and is no longer shared among several cursors.

The following methods/properties have been changed:
 - Environment.envs no longer works (because of the design change);
 - Environment.manage() is deprecated (no longer useful);
 - Environment.reset() is now an instance method;
 - env.clear_upon_failure() is deprecated in favor of cr.savepoint().

closes odoo/odoo#75598

Related: odoo/enterprise#20451
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Xavier Dollé <xdo@odoo.com>
2021-09-03 15:45:46 +00:00
Ivan Yelizariev 99787182e4 [FIX] base: skip unneeded update_list call
`update_list` is an expensive operation and could be avoided when no modules are
updated. Particually, it speeds up initial theme installation on website, which
has following code:

```
    themes.filtered(lambda m: m.state == 'installed').button_upgrade()
```

Benchmark for Odoo 15: `/website/configurator_apply` spends 1.3 sec for unneeded
`update_list` call.

closes odoo/odoo#75140

X-original-commit: 82ba4994e54fc46cc55a463629727e840349bb6e
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Ivan Yelizariev // IEL <yelizariev@users.noreply.github.com>
2021-08-16 11:01:44 +00:00
Stéphane Bidoul 9a131b2f78 [FIX] base: do not check dependencies of uninstallable addons
When an installed module is not installable, we don't want
to check its dependencies because they may not be available.
Erroring out on such dependencies is not needed because
the module will not be loaded anyway. On the contrary, such
errors prevent starting a database which would otherwise
function normally.

Such errors occur when doing incremental migrations where
it happen that addons are installed (from the previous version)
but not migrated yet and therefore not installable.
When we migrate lower level dependencies Odoo marks
higher level addons as "to upgrade" even if they are not installable.
That is usually harmless, except when such addons have
dependencies that are themselve not available.
This PR fixes that.

closes odoo/odoo#73206

X-original-commit: a11e8a32883b0d0deaeb2497daa585eb3d3c686e
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
2021-07-05 08:45:13 +00:00
Xavier Morel e0ad9dcf28 [FIX] base: confusion between werkzeug.urls and urls local
As part of the Python 3 compatibility effort,
01e3514147 moved all uses of urllib(2)
to werkzeug & requests as the packages were renamed and reorganised
between P2 and P3.

For `module.py`, it looks like I confused the `urls` local variable
for an already imported `werkzeug.urls` and just tried calling
`url_parse` on that. Which doesn't work, because `urls` is a
dict.

Fixes #71076

closes odoo/odoo#71120

X-original-commit: 0cfe528de82c1de8971e5bb5f66103447eae781c
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
2021-05-20 18:27:30 +00:00
8cc066173d [IMP] *: Improve assets management
This commit changes the way assets are declared in Odoo modules.

Before: assets were declared in template files. Template bundles were
generated from primary templates, so technically any qweb template could
have been called as an asset bundle, with the 't-call-assets' directive.

Being standard qweb templates, they had access to standard HTML tags
(script, link, with or without raw scripts or style definition), qweb
directives (t-call, t-raw, etc.) and could be inherited by other
templates.

Now: assets are defined in the module's manifest and generated by the
't-call-assets' directive.

More information on the new system can be found on the updated user
documentation (see the "JavaScript Reference" section).

Task: 2352566

Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
2021-03-31 13:57:17 +02:00
Mathias Markl 6ca12a2545 [IMP] base: performance improvement when installing a module
closes odoo/odoo#67316

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-03-16 11:50:29 +00:00
Mathias Markl f6d8275533 [IMP] base: performance improvement when installing a module
closes odoo/odoo#67959

X-original-commit: c0afa00d7cdbe98e4c67d2987a2b808bedd474ac
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2021-03-16 14:00:32 +00:00
Yannick Tivisse 6b6d5931a8 [FIX] base: Fix bad call to super using the same name than a variable
closes odoo/odoo#66515

Related: odoo/enterprise#16482
Related: odoo/upgrade#2178
Signed-off-by: Yannick Tivisse (yti) <yti@odoo.com>
2021-02-24 07:19:22 +00:00
Xavier-Do 87857b1066 [FIX] base: do not update module studio_customization
Since odoo/enterprise@937b9214cb module
'studio_customization' has a dependency on module 'web_studio', in order
to automatically remove customizations when uninstalling 'web_studio'.
As a side effect, this marks 'studio_customization' to be upgraded when
'web_studio' is upgraded.  An error is logged, but the module stays in
state `to upgrade`, thereby blocking the execution of cron jobs.

The fix simply adds an explicit (though not very elegant) condition on
the module name to filter out module 'studio_customization' when marking
modules in method `button_upgrade`.

A cleaner fix would be to use the `imported` flag, but we don't want to
risk breaking existing features in stable and will be done in master
instead.

Slightly linked to closed #43880

closes odoo/odoo#64538

X-original-commit: 572e05284c1a7587e989acebd43f3c07e7e99fb1
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-01-14 12:08:54 +00:00
Adrian Torres 679185f3e8 [IMP] *: replace all raises inside unlink by api.ondelete
With this commit, all instances of errors being raised inside
`BaseModel.unlink` overrides are moved into methods decorated with
`api.ondelete` which is safer.
2020-12-04 09:16:16 +00:00
Victor Feyens c482fdbf05 [IMP] various: improve code style and performance by improve `all()` usage
PURPOSE

Clean code. Be more performance oriented.

SPECIFICATIONS

Improvements applied in this commit

  * not all() --> any(not) for earlier returns;
  * all([generator]) --> all(generator) to avoid unnecessary list casting.
    This code construct is better managed by all;

This commit will probably not have a big performance effect on standard
production databases. However each performance and cleaning improvement
is welcomed.

LINKS

Task ID-2328619

closes odoo/odoo#56810

X-original-commit: 1cc6bb1231401ea7f501d2f5b5e9641ec8734850
Related: odoo/enterprise#12802
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
2020-08-31 13:59:27 +00:00
Xavier-Do 4da3f9cf1c [FIX] base: avoid warning when computing desc of to_buy modules
post upgrade tests will trigger 'module not found' warning when testing modules views
because of 'fake' modules without real file path.

closes odoo/odoo#56335

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2020-08-21 16:07:00 +00:00
Victor Feyens f287d29ad5 [FIX] base: ir_module computes on new record
Do not crash when the computes are displayed on an unsaved record.
This bugfix is needed for enterprise/web_studio test tours.
2020-08-20 14:23:15 +00:00