When we are importing a module and we need to create an attachment from
a file in static. We get an error if the file path of the file contains
a space.
We get this error becuase it takes the path to generate an xml id and
the name of an xml_id cannot contains spaces.
closesodoo/odoo#142113
Signed-off-by: Pierre Masereel (pim) <pim@odoo.com>
Since we have multiple categories for the Industries apps. We are now
getting them from apps.odoo.com. And we are using the ids from apps in
the domain that is sent back to apps.
closesodoo/odoo#141173
Signed-off-by: Pierre Masereel (pim) <pim@odoo.com>
With normal module, the icon is always loaded from the filesystem but
when importing a module that contains a module icon (by default in
static/description/icon.png) it crash with a `FileNotFound` exceptions
because the image is not available on the filesystem but as an
ir.attachment.
This ensure that for imported module we load the icon from the
corresponding imported attachment
closesodoo/odoo#141133
Signed-off-by: Pierre Masereel (pim) <pim@odoo.com>
Since we need to get the list of dependencies for the saas (and not noly
a Text of the module names, we split the function to be able to easily
get the list of dependencies.
closesodoo/odoo#140994
Signed-off-by: Pierre Masereel (pim) <pim@odoo.com>
We now have the possbility to publish modules that only contains XML on
apps.odoo.com and we want to display in the apps of the database to ease
the installation.
TASK-3186716
closesodoo/odoo#140245
Signed-off-by: Pierre Masereel (pim) <pim@odoo.com>
Fixes a large number of cases where strings are translated then
formatted, instead of letting `_()` do the formatting internally,
which allows it to recover from incorrect translations (missing,
broken, or extra placeholders).
Also
- removes translation markers entirely when there's nothing to
translate e.g. `_("%s - %s")` is not useful
- fixes a few messes which lead to only partial translatability
(DRY is generally a bad idea when translations are involved, even
more so when you don't make the variable part translatable)
- fixes a few nearby issues noticed at the same time
- replaces a few `"%s"` by `%r`, which should automatically quote
strings relatively appropriately
- fixes translated strings which use `\` to escape a newline (in order
to fill-paragraph): `\` escapes only the newline, if the
continuation string is indented this results in a bunch of spaces
ending in the string to translate, which is pretty garbage for the
translator, using implicit concatenation works much better
Note: some of the updates revert f-string parameters to %, because
babel (2.9) apparently has trouble with f-strings and blows up trying
to extract them.
Not in scope:
Helping translators fix translatable strings e.g. any translation
string with more than one placeholder probably should use keyword
placeholders
- Provides more context / data to the translator to make sense of the
sentence.
- Allows reordering the translated terms, which can be necessary
depending on the sentence and language.
closesodoo/odoo#139314
Related: odoo/enterprise#49311
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Revision odoo/odoo@cabb9e7e57 introduced a
regression: This is no longer possible to import a data module
using `<field file="..."/>` in their data file.
This revision targets to restore the feature as expected.
The unit tests added covers the feature, so that regression
no longer happens in the future.
It introduces a new concept of temporary directory `file_open`
can read from.
e.g.
```py
with odoo.tools.file_open_temporary_directory(self.env) as module_dir:
with zipfile.ZipFile('foo.zip', "r") as z:
z.extract('foo/__manifest__.py', module_dir)
with odoo.tools.file_open('foo/__manifest__.py', env=self.env) as f:
manifest = f.read()
```
Note that `file_open` will be allowed to read from that temporary
directory only if `env` is passed to `file_open`,
and if the `env` is part of the same transaction/request than the `env`
passed to `file_open_temporary_directory`.
This is to avoid having users, whether from other databases,
or even the same database,
trying to access these directories not belonging to them.
e.g. If an admin uploads sensitive data in this temporary directory,
no one than him must be allowed to read from these files, not even
another user from his database.
closesodoo/odoo#126326
X-original-commit: ea5cfe2e9493e4ef5a9b81851af452c8c51e407e
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
The revision dd3f918cafebe26e5aa9dff1b2e484b7ed662140
introduced a regression:
it was no longer possible to set an icon in the import module,
and if no icon is provided,
the icon should fall back to the base module icon,
which wasn't the case either.
In this last case, it even leaded to a crash:
```
File "/home/odoo/src/14.0/odoo/odoo/addons/base/models/ir_module.py", line 251, in _get_icon_image
with tools.file_open(path, 'rb') as image_file:
File "/home/odoo/src/14.0/odoo/odoo/tools/misc.py", line 176, in file_open
return _fileopen(name, mode=mode, basedir=base, pathinfo=pathinfo, basename=basename, filter_ext=filter_ext)
File "/home/odoo/src/14.0/odoo/odoo/tools/misc.py", line 211, in _fileopen
raise ValueError("Unknown path: %s" % name)
Exception
ValueError: Unknown path: /base/static/description/icon.png
```
opw-3340652
closesodoo/odoo#124661
X-original-commit: 12ccfcf8d2ee936ec066de81c2dce2dd17348cb6
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
We no longer want to support importing data modules from within
the addons path.
Only modules imported with an archive and extracted in a temporary
folder are supported.
closesodoo/odoo#121881
For performance reasons, instead of extracting the whole
zip, extract only the files which are actually used
during in the `_import_module`, in case people
put additional crap in the archive which are ignored
during the import. That way, we avoid useless I/O.
Also, adding the temporary directory in the addons
path wasn't thread-safe.
This revision changes this to make the module
import feature thread-safe.
closesodoo/odoo#80120
Signed-off-by: Pierre Masereel <pim@odoo.com>
The goal of this revision is to re-use the environment among the
different steps of the registry loading,
instead of creating a new environment for each step.
1. Simply To avoid to repeat the line
`env = api.Environment(cr, SUPERUSER_ID, {})`
multiple times in the code
2. This also allows to share the context among the different
steps. This is not yet used in this revision, but it could
be, for instance to avoid the current repetition to add the keys
`install_module`, in `convert_csv_import` and `xml_import._tag_record`
Part-of: odoo/odoo#108254
The existing code was generating misleading errors for imported Odoo
modules that could not be loaded. Although there was a specific hack
for module 'studio_customization', imported modules were not handled
properly. This patch adds the right condition in the SQL query in the
module that introduces imported modules.
closesodoo/odoo#106098
X-original-commit: e1cfc1f74000e55473f7a26f0df5a13c4d5094c0
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Raphael Collet <rco@odoo.com>
Issue:
Imported module is not able to search existing static files to replace them
Cause:
After changing the new loc calculation, ir.attachment will link to the
ir.model.data. However, those attachments will not able to search anymore
without sudo. Therefore, the system will try to create the attachment again.
It will return error because constraint "ir_model_data_module_name_uniq_index"
will prevent ir.model.data to create again.
Solution:
Use sudo to search the attachment.
closesodoo/odoo#93752
X-original-commit: 8f604847eb1ec96955ee724ad7b882c74160b9d7
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Context
-------
Web design specific require a lot of work and need maintenance.
So far the style files were not taken into account despite the
amount of maintenance they require.
With this commit they are going to be counted,
knowing they still can be excluded in the manifest.
Imported module allow to deploy frontend assets: stylesheet,
javascript, xml template and Qweb view that require
as well some maintenance. They are going to be counted
Implementation
--------------
- Add method to parse css and scss file
- Include .scss and .css file in the count
- Add external_id to attachment that store
frontend asset of imported module
- Find all attachment with .js, .css, .scss, .xml
from imported module
- Find qweb view from imported module
- cound the content of the attachment and the qweb view
closesodoo/odoo#86816
X-original-commit: d2e0b48271dbd8cd7bbdbc5c2c1649c55c51417b
Signed-off-by: Christophe Simonis <chs@odoo.com>
Signed-off-by: Thibault Francois <tfr@odoo.com>
The http.addons_manifest is a map {module: manifest_dict} that is
populated upon the first http request. This map is basically a module
manifest cache with an extra `addons_path` key, the path of the module
on the file-system. This cache is eagerly populated upon the first http
request, the map is empty in non-http contextes (e.g. cron) which have
been a source of bugs (e.g. 50c8eb1).
A manifest cache is necessary because reading and parsing python files
from the file-system is not that cheap but there is no reason that cache
is located in `odoo.http`. A thin cache layer now wraps
`load_information_from_description_file()`/`load_manifest()` and is
lazily populated.
The `http.addons_manifest` have been removed. The extra `addons_path`
key is now present in the "normal" manifest. The `read_manifest()` was
hardly used so it has been deprecated. The only way to retrieve a
manifest is now `load_information_from_description_file()` which was
renamed `load_manifest()` (no cache) and `get_manifest()` (cache).
Side note about performances, the cache is necessary. Addons manifest
are read-only and reading + parsing python files from the file system is
not a cheap operation. Running the e-commerce tour
`@website_sale.test_04_admin_website_sale_tour` without cache on
`load_manifest()` requires 68,29 secs to complete on my laptop,
exceeding the default 1-minute time frame allowed in tests. Using a
cache the time is down to 36,53 secs. The performance impact is huge.
Part-of: odoo/odoo#79977
The `ir_import_module` module is used by users to upload custom module
data via a zip file. The zip file can contain xml data, xml views and
static files.
Before this commit, the "import module" feature of this module was
broken since the imported module's manifest was lost after the import
was performed.
Now, 'ir.asset' records are created along with the attachments pointing
to the imported static files.
closesodoo/odoo#80120
X-original-commit: ec77577796e72707fbb2077f05067428cebc3c7f
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Julien Mougenot (jum) <jum@odoo.com>
Commit 59eee6ba5f introduced a cleanup
mechanism for external data modules being uninstalled from a database,
this meant deleting the ir.module.module entry created when first
installing the aforementioned external module.
However, the above patch has one small oversight: the access of the
`imported` field introduced by the `base_import_module` module is done
after the `super().module_uninstall()` call which, in the case of the
uninstall of the `base_import_module` module, will delete the `imported`
column and the following call to filtered will fail because the column
has already been deleted and the registry hasn't been reloaded yet.
The solution to this, as explained in the code comment, is to simply
compute the `modules_to_delete` before the call to `module_uninstall()`.
closesodoo/odoo#59418
X-original-commit: c0226087b82ffda3971bb953d8d00b8c5e67642c
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Adrian Torres (adt) <adt@odoo.com>
Using a few regex like
\((_\(.*%s.*)(\) % )([\w\[\]][\w .\[\]\(\)'"]*)\)
($1, $3))
Old syntax is still compatible but starts the migration to the new
syntax that catches error.
Uninstalling an imported module leaves behind an ir.module.module entry
which is entirely useless: it cannot be reinstalled without re-uploading
the module.
A quick solution is provided here by simply cleaning up after the
uninstallation and deleting the ir.module.module records of imported
modules that just got removed.
Another possibility could be to store the uploaded module for good in
an attachment and re-load it from there upon re-installation, but that
would lead to some conflict with Studio, as uninstalling the
studio_customization module usually means you want to get rid of it for
good.
* walksymlinks is useless, os.walk got followlinks in 2.6
* tempdir is redundant with tempfile.TemporaryDirectory added in 3.2
* listdir(recursive=False) is just os.listdir (I guess recursive=True
is somewhat useful)
* zip_dir is *not* redundant with shutil.make_archive:
- zip_dir handles the archived root differently
- zip_dir sorts files, make_archive sorts directories
- make_archive explicitly adds entries for directories (including
empty), sorts directories, doesn't sort files; zip_dir sorts
files (including with a custom key function) but ignores directories
- zip_dir filters out a bunch of trash files
closesodoo/odoo#39573
Signed-off-by: Christophe Simonis <chs@odoo.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Multi is the default api for methods, it is not necessary to explicitly
decorate methods with it, adds clutter and most people use it because
they see that the rest of the code uses it.
Done with `find . -type f -name '*.py' | xargs sed -i '/@api.multi/d'`
The old tree views don't really exist anymore, this odd pseudo-flag to
dispatch between "list" and "tree" tree views has no reason to remain.
Task 1937686
closesodoo/odoo#31243
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit removes the field `datas_fname` from `ir.attachment` as
it was unnecessary and most of the time the duplicate of `name` or
`url`.
Task #1909865closesodoo/odoo#32976
Signed-off-by: Martin Geubelle (mge) <mge@openerp.com>
Currently, when trying to import a module, if there is an error
related to dependencies, it is not shown in a decent format. For
example, technical names of the modules are being displayed in
a single line, separated by comma.
This commit makes the dependency error message readable by
displaying short description of the modules (instead of
technical names), each on a new line.
task-1830483
closesodoo/odoo#33029
Signed-off-by: Martin Geubelle (mge) <mge@openerp.com>
If the zipfile contains some garbage directories (e.g. leftover empty
directories from git, `__MACOSX` metadata folder, …) it seems
unnecessary to log an error, just skip the directory and don't mark it
as a proper / successful module.
closesodoo/odoo#31639
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
Check that it makes custom binary fields into attachment as that's the
main reason for the change: when users create binary fields via Studio,
they're necessarily db-stored (as the interface doesn't allow altering
the attachment attribute and it's unclear how we'd handle users
switching it on/off every time), which significantly bloats their
database (and burns storage & backup space), especially as the primary
use case for binary fields is adding images and documents to records.
* check that binary fields are properly created as attachment=True
* add attachment=False on fields where that seems relevant (most but not
all of the fields previously using the default)
* remove occurrences of attachment=True
closesodoo/odoo#29308
- Create a simple module, set version to 2.0
- Import the zip file
The 'Latest Version' (field `installed_version`) is set to 11.0.1.0,
while one would expect 11.0.2.0.
There are several reasons for this. First, the method
`get_values_from_terp` doesn't return the version, so at this point the
information is already lost. Then, the field `installed_version` is a
computed field which retrieves the version from the `__manifest__.py`
file. In the case of an imported module, there is no such file in which
we can look up for the value (even though the imported module contained
one).
To avoid this, we explicitly set the 'Installed Version' (field
`latest_version`) to the version retrieved at import. Since this field
refers to the version installed in the database, it makes sense to use
it this way. Moreover, in the case of imported modules, we use the value
of `latest_version` for `installed_version`.
Fixes#23988
opw-1833522
Before this commit, while performing a check on whether a customization
being imported was done using studio or not, we would assume that the
.zip would, at the very least, contain a data folder.
This is not the case as loose files can be imported and the data folder
is only generated by studio customizations.
To solve this, we walk over the whole zip instead of walking over the
data/ folder only (which may be non-existant), so loose files, files
inside the data folder and potentially files in other folders can be
parsed without raising unexpected errors.
Previous to this commit, when importing a studio module with unmet
dependencies that were not studio the error message would just display
the Studio is required to install Studio customizations.
With this commit it should now correctly display the missing
dependencies by checking if only web_studio is in the set of unmet
dependencies
opw-777642
Previous to this commit, anyone could import studio customizations into
a community db, this was troublesome since some people would just
install studio, do their customizations and then move them into a
community db, not paying anything at all.
This commit fixes this by checking whether the exported views were
created with studio or not and if studio is installed
Now that we're closer to switching to P3 for good, these helpers have
outlived their usefulness, and mostly add noise.
All remaining dict.iter*() or dict.view*() must be converted to the
normal keys(), values() or items() calls.
Whenever the result is likely to be used for more than the scope of a
loop, or when the dict needs to be modified during iteration, the calls
must be wrapped in a ``list()``, to protect the new P3 semantics.
Those cases are very exceptional.
Also removed some dead code or improved the API to remove unnecessary
conversions.
* StringIO removed from stdlib, replace with io
* try to correctly handle BytesIO/StringIO (one is for bytes the other
is for text)
* fix base64: Python 3 removed bytes-encoding and bytes-bytes
codecs (via #encode) so replace all calls to str.encode('base64'),
also b64encode is a bytes->bytes conversion so attempt to properly
handle that
issue #8530
As a consequence of rev. 76cd8d2558,
imported modules were unable to access their resource files during
import.
Rather than further modifying the file_open API to whitelist paths
(the whole thing needs a redesign in master), we temporarily
whitelist the temporary directory by including it in the global
addons_paths, making sure to undo it afterwards.
This gives all lower level function access the resource files via
file_open, without having to pass around whitelisted paths
through many different calls.
Pierre Masereel