The goal of this revision is to re-use the environment among the
different steps of the registry loading,
instead of creating a new environment for each step.
1. Simply To avoid to repeat the line
`env = api.Environment(cr, SUPERUSER_ID, {})`
multiple times in the code
2. This also allows to share the context among the different
steps. This is not yet used in this revision, but it could
be, for instance to avoid the current repetition to add the keys
`install_module`, in `convert_csv_import` and `xml_import._tag_record`
Part-of: odoo/odoo#108254
The existing code was generating misleading errors for imported Odoo
modules that could not be loaded. Although there was a specific hack
for module 'studio_customization', imported modules were not handled
properly. This patch adds the right condition in the SQL query in the
module that introduces imported modules.
closesodoo/odoo#106098
X-original-commit: e1cfc1f74000e55473f7a26f0df5a13c4d5094c0
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
Issue:
Imported module is not able to search existing static files to replace them
Cause:
After changing the new loc calculation, ir.attachment will link to the
ir.model.data. However, those attachments will not able to search anymore
without sudo. Therefore, the system will try to create the attachment again.
It will return error because constraint "ir_model_data_module_name_uniq_index"
will prevent ir.model.data to create again.
Solution:
Use sudo to search the attachment.
closesodoo/odoo#93752
X-original-commit: 8f604847eb1ec96955ee724ad7b882c74160b9d7
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Context
-------
Web design specific require a lot of work and need maintenance.
So far the style files were not taken into account despite the
amount of maintenance they require.
With this commit they are going to be counted,
knowing they still can be excluded in the manifest.
Imported module allow to deploy frontend assets: stylesheet,
javascript, xml template and Qweb view that require
as well some maintenance. They are going to be counted
Implementation
--------------
- Add method to parse css and scss file
- Include .scss and .css file in the count
- Add external_id to attachment that store
frontend asset of imported module
- Find all attachment with .js, .css, .scss, .xml
from imported module
- Find qweb view from imported module
- cound the content of the attachment and the qweb view
closesodoo/odoo#86816
X-original-commit: d2e0b48271dbd8cd7bbdbc5c2c1649c55c51417b
Signed-off-by: Christophe Simonis <chs@odoo.com>
Signed-off-by: Thibault Francois <tfr@odoo.com>
The http.addons_manifest is a map {module: manifest_dict} that is
populated upon the first http request. This map is basically a module
manifest cache with an extra `addons_path` key, the path of the module
on the file-system. This cache is eagerly populated upon the first http
request, the map is empty in non-http contextes (e.g. cron) which have
been a source of bugs (e.g. 50c8eb1).
A manifest cache is necessary because reading and parsing python files
from the file-system is not that cheap but there is no reason that cache
is located in `odoo.http`. A thin cache layer now wraps
`load_information_from_description_file()`/`load_manifest()` and is
lazily populated.
The `http.addons_manifest` have been removed. The extra `addons_path`
key is now present in the "normal" manifest. The `read_manifest()` was
hardly used so it has been deprecated. The only way to retrieve a
manifest is now `load_information_from_description_file()` which was
renamed `load_manifest()` (no cache) and `get_manifest()` (cache).
Side note about performances, the cache is necessary. Addons manifest
are read-only and reading + parsing python files from the file system is
not a cheap operation. Running the e-commerce tour
`@website_sale.test_04_admin_website_sale_tour` without cache on
`load_manifest()` requires 68,29 secs to complete on my laptop,
exceeding the default 1-minute time frame allowed in tests. Using a
cache the time is down to 36,53 secs. The performance impact is huge.
Part-of: odoo/odoo#79977
The `ir_import_module` module is used by users to upload custom module
data via a zip file. The zip file can contain xml data, xml views and
static files.
Before this commit, the "import module" feature of this module was
broken since the imported module's manifest was lost after the import
was performed.
Now, 'ir.asset' records are created along with the attachments pointing
to the imported static files.
closesodoo/odoo#80120
X-original-commit: ec77577796e72707fbb2077f05067428cebc3c7f
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Julien Mougenot (jum) <jum@odoo.com>
Commit 59eee6ba5f introduced a cleanup
mechanism for external data modules being uninstalled from a database,
this meant deleting the ir.module.module entry created when first
installing the aforementioned external module.
However, the above patch has one small oversight: the access of the
`imported` field introduced by the `base_import_module` module is done
after the `super().module_uninstall()` call which, in the case of the
uninstall of the `base_import_module` module, will delete the `imported`
column and the following call to filtered will fail because the column
has already been deleted and the registry hasn't been reloaded yet.
The solution to this, as explained in the code comment, is to simply
compute the `modules_to_delete` before the call to `module_uninstall()`.
closesodoo/odoo#59418
X-original-commit: c0226087b82ffda3971bb953d8d00b8c5e67642c
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Using a few regex like
\((_\(.*%s.*)(\) % )([\w\[\]][\w .\[\]\(\)'"]*)\)
($1, $3))
Old syntax is still compatible but starts the migration to the new
syntax that catches error.
Uninstalling an imported module leaves behind an ir.module.module entry
which is entirely useless: it cannot be reinstalled without re-uploading
the module.
A quick solution is provided here by simply cleaning up after the
uninstallation and deleting the ir.module.module records of imported
modules that just got removed.
Another possibility could be to store the uploaded module for good in
an attachment and re-load it from there upon re-installation, but that
would lead to some conflict with Studio, as uninstalling the
studio_customization module usually means you want to get rid of it for
good.
* walksymlinks is useless, os.walk got followlinks in 2.6
* tempdir is redundant with tempfile.TemporaryDirectory added in 3.2
* listdir(recursive=False) is just os.listdir (I guess recursive=True
is somewhat useful)
* zip_dir is *not* redundant with shutil.make_archive:
- zip_dir handles the archived root differently
- zip_dir sorts files, make_archive sorts directories
- make_archive explicitly adds entries for directories (including
empty), sorts directories, doesn't sort files; zip_dir sorts
files (including with a custom key function) but ignores directories
- zip_dir filters out a bunch of trash files
closesodoo/odoo#39573
Signed-off-by: Christophe Simonis <chs@odoo.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Multi is the default api for methods, it is not necessary to explicitly
decorate methods with it, adds clutter and most people use it because
they see that the rest of the code uses it.
Done with `find . -type f -name '*.py' | xargs sed -i '/@api.multi/d'`
The old tree views don't really exist anymore, this odd pseudo-flag to
dispatch between "list" and "tree" tree views has no reason to remain.
Task 1937686
closesodoo/odoo#31243
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit removes the field `datas_fname` from `ir.attachment` as
it was unnecessary and most of the time the duplicate of `name` or
`url`.
Task #1909865closesodoo/odoo#32976
Signed-off-by: Martin Geubelle (mge) <mge@openerp.com>
Currently, when trying to import a module, if there is an error
related to dependencies, it is not shown in a decent format. For
example, technical names of the modules are being displayed in
a single line, separated by comma.
This commit makes the dependency error message readable by
displaying short description of the modules (instead of
technical names), each on a new line.
task-1830483
closesodoo/odoo#33029
Signed-off-by: Martin Geubelle (mge) <mge@openerp.com>
If the zipfile contains some garbage directories (e.g. leftover empty
directories from git, `__MACOSX` metadata folder, …) it seems
unnecessary to log an error, just skip the directory and don't mark it
as a proper / successful module.
closesodoo/odoo#31639
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
Check that it makes custom binary fields into attachment as that's the
main reason for the change: when users create binary fields via Studio,
they're necessarily db-stored (as the interface doesn't allow altering
the attachment attribute and it's unclear how we'd handle users
switching it on/off every time), which significantly bloats their
database (and burns storage & backup space), especially as the primary
use case for binary fields is adding images and documents to records.
* check that binary fields are properly created as attachment=True
* add attachment=False on fields where that seems relevant (most but not
all of the fields previously using the default)
* remove occurrences of attachment=True
closesodoo/odoo#29308
- Create a simple module, set version to 2.0
- Import the zip file
The 'Latest Version' (field `installed_version`) is set to 11.0.1.0,
while one would expect 11.0.2.0.
There are several reasons for this. First, the method
`get_values_from_terp` doesn't return the version, so at this point the
information is already lost. Then, the field `installed_version` is a
computed field which retrieves the version from the `__manifest__.py`
file. In the case of an imported module, there is no such file in which
we can look up for the value (even though the imported module contained
one).
To avoid this, we explicitly set the 'Installed Version' (field
`latest_version`) to the version retrieved at import. Since this field
refers to the version installed in the database, it makes sense to use
it this way. Moreover, in the case of imported modules, we use the value
of `latest_version` for `installed_version`.
Fixes#23988
opw-1833522
Before this commit, while performing a check on whether a customization
being imported was done using studio or not, we would assume that the
.zip would, at the very least, contain a data folder.
This is not the case as loose files can be imported and the data folder
is only generated by studio customizations.
To solve this, we walk over the whole zip instead of walking over the
data/ folder only (which may be non-existant), so loose files, files
inside the data folder and potentially files in other folders can be
parsed without raising unexpected errors.
Previous to this commit, when importing a studio module with unmet
dependencies that were not studio the error message would just display
the Studio is required to install Studio customizations.
With this commit it should now correctly display the missing
dependencies by checking if only web_studio is in the set of unmet
dependencies
opw-777642
Previous to this commit, anyone could import studio customizations into
a community db, this was troublesome since some people would just
install studio, do their customizations and then move them into a
community db, not paying anything at all.
This commit fixes this by checking whether the exported views were
created with studio or not and if studio is installed
Now that we're closer to switching to P3 for good, these helpers have
outlived their usefulness, and mostly add noise.
All remaining dict.iter*() or dict.view*() must be converted to the
normal keys(), values() or items() calls.
Whenever the result is likely to be used for more than the scope of a
loop, or when the dict needs to be modified during iteration, the calls
must be wrapped in a ``list()``, to protect the new P3 semantics.
Those cases are very exceptional.
Also removed some dead code or improved the API to remove unnecessary
conversions.
* StringIO removed from stdlib, replace with io
* try to correctly handle BytesIO/StringIO (one is for bytes the other
is for text)
* fix base64: Python 3 removed bytes-encoding and bytes-bytes
codecs (via #encode) so replace all calls to str.encode('base64'),
also b64encode is a bytes->bytes conversion so attempt to properly
handle that
issue #8530
As a consequence of rev. 76cd8d2558,
imported modules were unable to access their resource files during
import.
Rather than further modifying the file_open API to whitelist paths
(the whole thing needs a redesign in master), we temporarily
whitelist the temporary directory by including it in the global
addons_paths, making sure to undo it afterwards.
This gives all lower level function access the resource files via
file_open, without having to pass around whitelisted paths
through many different calls.
The import_module() method does not need to be public,
so let's mark it private. It's not called by anyone
except import_zipfile().
After 76cd8d2558 it would
fail anyway because addons_path would not be prepared by
import_zipfile().
In Python 3:
* various builtins and dict methods were changed to return
view/iterable objects rather than lists
* and the separate Python 2 view/iterable builtins and methods were
removed altogether
This is problematic when using these items as list (which the happens
repeatedly in Odoo), but more viciously when iterating *multiple times*
over them (which also happens, which I've messed up multiple times while
writing this, and which is a pain to debug even when you've just created
the issue).
Convert all code using these to semantics-matching cross-version
helper functions to get the LCD behaviour between P2 and P3, and
forbid the builtins via lint.
issue #8530
Also restrict XML data attribute evaluation context
even for real module data files. This will prevent
accidentally depending on context parameters that
would not be available inside base_import_module.
Denis Ledoux