It's pretty much unused and fairly complicated.
Also deprecate `listdir` entirely since `get_module_filetree` is the
only extant user of the recursive listdir.
closesodoo/odoo#98034
Related: odoo/enterprise#30403
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The http.addons_manifest is a map {module: manifest_dict} that is
populated upon the first http request. This map is basically a module
manifest cache with an extra `addons_path` key, the path of the module
on the file-system. This cache is eagerly populated upon the first http
request, the map is empty in non-http contextes (e.g. cron) which have
been a source of bugs (e.g. 50c8eb1).
A manifest cache is necessary because reading and parsing python files
from the file-system is not that cheap but there is no reason that cache
is located in `odoo.http`. A thin cache layer now wraps
`load_information_from_description_file()`/`load_manifest()` and is
lazily populated.
The `http.addons_manifest` have been removed. The extra `addons_path`
key is now present in the "normal" manifest. The `read_manifest()` was
hardly used so it has been deprecated. The only way to retrieve a
manifest is now `load_information_from_description_file()` which was
renamed `load_manifest()` (no cache) and `get_manifest()` (cache).
Side note about performances, the cache is necessary. Addons manifest
are read-only and reading + parsing python files from the file system is
not a cheap operation. Running the e-commerce tour
`@website_sale.test_04_admin_website_sale_tour` without cache on
`load_manifest()` requires 68,29 secs to complete on my laptop,
exceeding the default 1-minute time frame allowed in tests. Using a
cache the time is down to 36,53 secs. The performance impact is huge.
Part-of: odoo/odoo#79977
The license was missing in most enterprise manifest so
the decision was taken to make it explicit in all cases.
The missing licenses were add in all version starting from
12.0 in repos odoo, enterprise and design-themes.
Starting from this commit, when a license is not defined,
a warning will be triggered.
closesodoo/odoo#74347
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
This legacy package was supposed to be an alias to `odoo.upgrade`.
However, depending on how your import its sub-packages (and in which
order), we were ending with the module being loaded multiple times,
breaking the expectation of a singleton.
```python
In [1]: from odoo.addons.base.maintenance.migrations import util as m1
In [2]: import odoo.addons.base.maintenance.migrations.util as m2
In [3]: m1
Out[3]: <module 'odoo.upgrade.util' from '/Users/chs/devel/odoo/odoo/stable/odoo/addons/base/maintenance/migrations/util.py'>
In [4]: m2
Out[4]: <module 'odoo.addons.base.maintenance.migrations.util' from '/Users/chs/devel/odoo/odoo/stable/odoo/addons/base/maintenance/migrations/util.py'>
In [5]: from odoo.addons.base.maintenance.migrations import util as m3
In [6]: m3
Out[6]: <module 'odoo.addons.base.maintenance.migrations.util' from '/Users/chs/devel/odoo/odoo/stable/odoo/addons/base/maintenance/migrations/util.py'>
In [7]: m2 == m3
Out[7]: True
In [8]: m1 == m3
Out[8]: False
In [9]:
```
Now, with this import hook, we ensure that the modules imported from
`odoo.addons.base.maintenance.migrations` are aliases to ones imported
from `odoo.upgrade`.
```python
In [1]: import odoo.addons.base.maintenance.migrations.util as m2
In [2]: m2.__name__
Out[2]: 'odoo.upgrade.util'
```
closesodoo/odoo#71351
X-original-commit: 0d0458a0f370f872caacac26361f2c4730c2cbba
Signed-off-by: Christophe Simonis <chs@odoo.com>
Co-authored-by: Christophe Simonis <chs@odoo.com>
This commit changes the way assets are declared in Odoo modules.
Before: assets were declared in template files. Template bundles were
generated from primary templates, so technically any qweb template could
have been called as an asset bundle, with the 't-call-assets' directive.
Being standard qweb templates, they had access to standard HTML tags
(script, link, with or without raw scripts or style definition), qweb
directives (t-call, t-raw, etc.) and could be inherited by other
templates.
Now: assets are defined in the module's manifest and generated by the
't-call-assets' directive.
More information on the new system can be found on the updated user
documentation (see the "JavaScript Reference" section).
Task: 2352566
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
The "active" field is a non-working deprecated alias to "auto_install",
it does not work and was confusing users (see #59850). It has been
removed.
closesodoo/odoo#62086
Task: 2361729
Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
post upgrade tests will trigger 'module not found' warning when testing modules views
because of 'fake' modules without real file path.
closesodoo/odoo#56335
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
The `PackageLoader` jinja's loader purpose is to discover template files
given a python module path. It was necessary to register a special hook
into the import scheme of Odoo to support `import crm`, `import openerp`
and `import openerp.addons` like module path.
Since the support for those old module paths is deprecated since v13 and
the jinja's team shows desire to remove support to `pkg_resource` utils
as highlighted by #50552, it is better to remove it.
closesodoo/odoo#53515
Task: 2282681
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Currently there are a few issues with testing reporting (at the
command-line):
1. there is a global report after at_install tests, but it gets
"scrolled off" by long post_install tests, and is thus easy to
miss
2. with test tags, it's easy to fat finger a typo and run 0 tests,
which look like everything's running fine (no failure)
To improve this, print a global report at shutdown (in
`--stop-after-init` mode if tests are enabled) which recapitulates the
test results *and prints a warning if no tests were run at all*.
Also update the reporting collection to make this more reliable:
* have `run_unit_tests` return `None` if it has run no tests, the
assertion reporting machinery counts this as neither success nor
failure which is exactly what we want
* have load_test only report a success *if files were actually
loaded* (by having `load_data` return that information)
Task 2301268
closesodoo/odoo#54812
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The `initialize_sys_path` is the function responsible of getting the
custom Odoo import hooks to work, i.e. addons import via
`import odoo.addons`. This function is called by some other module
related functions to ensure the paths are correctly setup. We are
confident it is useless to re-initialize the path in many places.
The `load_openerp_module` function is called during server bootup by
both `odoo-bin server` and `odoo-bin shell`, both command parse the
configuration before starting the server which initialize the paths
already.
Most call to `get_module`, `get_module_path` and `get_resource_path` are
done in models or controllers where a registry is setup already. There
is one notable exception which is the subcommand discovery done during
the bootup, for that specific case, we initialize the paths with a
partially loaded configuration.
closesodoo/odoo#49715
Task: 2200956
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Every call of initialize_sys_path was adding two new hooks to the
sys.meta_path, resulting in very long loading times on databases having
a lot of modules.
For example, 2.27s instead of 752 on a database having 130 installed
modules.
References:
- odoo/odoo#45780
- odoo/odoo#45662closesodoo/odoo#53121
X-original-commit: 2444fde7f852787d87589a93c4bc385d76ff20e9
Signed-off-by: Christophe Simonis <chs@odoo.com>
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Use `warnings.warn` to log a warning entry on usage of deprecated
import hook instead of `logging.warning`. The `warnings` can be more
easily configured to show/hide class of warnings or to limit warning
emission, plus it is possible to log a single stack entry whereas
logging can just log the entire call stack.
Bring back the various openerp import hooks by reverting 9e1f13bac12
closesodoo/odoo#50604
Task: 2234749
X-original-commit: 5840202802c2ff6ca1ca8fd6f3892900481db5da
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Signed-off-by: Christophe Simonis <chs@odoo.com>
Performances from a general point of view can be difficult to track.
This commit proposes to improve logs in two ways:
The current logs only use the sql_counter, wich will only be updated
when a cursor is closed. In a test-enable install, this counter
is actually the queries of the tests wince the install cursor is
open untill the end. The first fix is to use bot sql_counter and
sql_log_count to have total queries untill now on closed cursor,
but also the current number of queries of the current cursor.
This means that the new log format will be
{nb} modules loaded in {time}, {loading_querie} (+{test_cr_queries}) queries
instead of
{nb} modules loaded in {time}, {tests_cr__queries}queries
Nothe that in the current version, {nb} is actually the total number of
loaded modules until now.
This commit also add an equivalent end log by module and change the
loglevel of module start on install (mainly usefull if an error occurs
before anything else is logged hidding the module causing this error.)
A cleaner runbot logger is also added, in order to be abble to call
_logger.runbot( instead of _logger.log(25. This will clarify the purpose
of such a log level.
closesodoo/odoo#47283
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
A future upgrades improvement will be to add tests in upgrades
modules. This small imp will scan in upgrade for some tests,
specific tooling will be added to upgrade repository in odoo/upgrade#878closesodoo/odoo#47030
X-original-commit: 59b5a03bdf7ae1ad8851a67d278edc31a26a96a8
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Some of the implicit side-effects got missed, namely that under some conditions
(e.g. using an odoo subcommand) `initialize_sys_path` can be called before the
config has been loaded at all, resulting in the first call not properly setting
up things, and one of the subsequent calls fixing things up.
Since this breaks workflows right now, quickly fix it, we'll re-investigate
how to fixup the entire thing in order to restrict & enforce a single call.
closesodoo/odoo#45844
X-original-commit: 6cb4c829e1559bcf836e4b573051760565b7d0c1
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
On a standard odoo install, `MigrationManager._get_files` represent
more than 4% of an install. This is because the legacy
odoo/base/maintenance/migration path was added to upgrade.__path__
once by module, making the get_filed check 574^2 os.path.exists.
This commit adds a check on initialize_sys_path to call it only once,
and merge legacy path with upgrade-path management in order to benefit
of the `up not in upgrade.__path__` check. This second part of the fix
will also remove the local dir from the upgrades paths.
A further improvement would be to fix MigrationManager in order to skip
_get_file work on a fresh install, (wip by rco-odoo)
closesodoo/odoo#45699
X-original-commit: 497330a695ed52a33f9c9b9c27b149446de0db29
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Since this option is soon to be used, this rename is a last tweek
to make it more logical to use since it will point to a single
upgrade dir most of the time.
closesodoo/odoo#44593
X-original-commit: 1c8e2809fb296abce6114b7da906d48a240df418
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Migration have long been only accessible thanks to a symlink from
`odoo.base.maintenance` to our private migration repository. Thank to
the change of bbb1a8f it is now possible to give a load the
migrations scripts from a path given in options.
The `initialize_sys_path` function has been updated to hooks the new
paths or the legacy symlink and to provide aliases to the previous
import logic to ensure backward compatibility.
`odoo.upgrades` (`community/odoo/upgrades`) is a new namespace that hook
all `--upgrades-paths` directories or the
`community/odoo/base/maintenance/migrations` symlink if none is previded.
`odoo.addons.base.maintenance.migrations` has been made an alias to
`odoo.upgrades`.
The `odoo.upgrades` is the desired method for accessing migrations
scripts and should be used by all new scripts.
closesodoo/odoo#44117
Task: 2178274
X-original-commit: d963cc05acd882729c4eb5ab940dae2a2197e55a
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
This commit partially backports bpo-24412, which allows the definition of
class cleanups (addClassCleanup) and module cleanups (omitted),
similar to instance cleanups (addCleanup).
This is useful for tests that override unittest's setUpClass and
could crash during its execution: If this happens, it is possible that a
bunch of crap is left in the database or even worse, the cursor becomes
completely fucked; Thanks to the addClassCleanup, we can undo the damage
done by the setUpClass.
Another benefit is that it is called unconditionally after tearDownClass
is called, so it can also be called as a replacement and/or safer
tearDownClass.
`openerp` module/addons imports has been deprecated in v13 by 7c47eb1
for removal in v14.
If you were still using the removed aliases, please substitute all
`import openerp` by `import odoo` and `import openerp.addons` by
`import odoo.addons`.
If you were still using the removed `ad_paths` proxy, please use the
python standard `odoo.addons.__path__`.
closesodoo/odoo#37007
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
[PEP-594] is deprecating the `imp` module, that module is used in
`module.py` in order to dynamically import addons using any of the
`odoo.addons` or `openerp.addons` import anchor.
We are deprecating `openerp` module/addons imports in v13 in order to
remove the support in v14 and greatly simplify how modules/addons are
loaded. If you are still using the old `import openerp` or `import
openerp.addons`, `import odoo` and `import odoo.addons` are drop-in
replacements.
The `odoo.modules.module.ad_paths` addon paths list has been deprecated
too. The list is now accessible on `odoo.addons.__path__` where they
are now directly loaded [2].
See also:
[PEP-594]: https://python.org/dev/peps/pep-0594/
[2]: https://packaging.python.org/guides/packaging-namespace-packages/closesodoo/odoo#36597
Task: 2003936
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Before this commit --log-level had no impact on test logs.
We need to check logger "isEnabledFor" before makeRecord
since logger will make this check in error,warn,log...
dedicated function before making call to _log.
We also need to use modules.py runner instead of unittest
logger in OdooTestRunner (test will allways be a suite there)
closesodoo/odoo#36142
Signed-off-by: Romain Libert (rli) <rli@odoo.com>
Since 735ee54 the logs are improved by getting additional informations
from the traceback. When the test fails during the setUpClass, the test
object received by the getCallerInfo method is not a TestCase instance
but an _ErrorHolder. Although they should have the same API, as stated in
unittest documentation, the _ErrorHolder does not have a
_testMethodName attribute leading to a warning.
With this commit, the warning is skipped in that particular case.
closesodoo/odoo#36108
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Since #34996, in some case, the error detail was not logged keeping
only the final summary "x errors, x failures" when running TestSuite.
Making fail test test_cache_invalidation for instance won't show
a detailed message when breaking test_01_project_tour will.
This issue occurs when using subtest, like with assertQueryCount,
@users decorator, test_all_l10n, and test_youtube_urls.
Since Testresult addSubTest append directly to error and failures
instead of calling addError and addFailure, we need to ovewrite
addSubtest too.
closesodoo/odoo#35270
Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
1. Make test logs clearer & remove redundancies
Instead of having an ERROR log right when the test fails then print
the useful / relevant information at the end of the test suite,
immediately print the traceback. Keep the final summary. Also avoids
having to wait for the entire test suite to end before a dev' can know
the failure details of a specific test.
Done by working at a lower level and replacing the custom
test stream mess by a custom Result class which prints and formats the
information we want. Replace TextTestRunner by a bare-bones custom
Runner object to tie it in.
2. Provide useful location information on test failure
Leverage the work above to log the test function's failure location:
previously logging would point to within TestStream which is not
useful.
Here, on failure the traceback is used to discover the caller info and
point to the test line which fails instead. similar to unittest's
_exc_info_to_string (https://github.com/python/cpython/blob/93e8aa62cfd0a61efed4a61a2ffc2283ae986ef2/Lib/unittest/result.py#L173).
3. Replace direct logging in browser_js by raising errors
Properly marks the test as in error, and the error traceback points to
the tour definition / launcher (python side) rather than common.py
and/or module.py.
Also removes unused dbname parameter that was added in
/278ed718e9805edf088642ba10d3b7c4e5716c31/openerp/modules/module.py#L361
for nor visible reason
closesodoo/odoo#34996
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Traceback coming from tests are currently logged
line by line. This will implies that runbot will have one
ir_logging entry per line which is not practical. More than
that, the log prefix can make the traceback less readable
because of line returns and difficult to copy paste.
This commit simply remove this feature. After discussion with odo
and chs, we will also remove the docstring from test shortDescription
since most of the time this information is not clear and can be accessed
in source code if needed.
closesodoo/odoo#33911
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Currently, auto_install is triggered when all dependencies get
installed, but there are cases where one would want such trigger on
only a subset thereof.
e.g. we want `website_sale_dashboard` to auto-install when
`website_sale` is installed. Currently, it requires `web_dashboard` to
also be auto-installed otherwise `website_sale_dashboard` would "wait"
for both dependencies to be explicitly installed before the
auto-install triggers. That's despite `web_dashboard` not being very
useful on its own. More generally this is an issue with technical
modules which need to be marked as auto_install so as not to block
e.g. bridge modules from automatically installing.
This change allows setting `auto_install` to a subset of `depends`:
* if auto_install is set to `False`, the module does not get
automatically installed (no change in semantics)
* if auto_install is set to `True`, the module gets automatically
installed if and only if all its dependencies are installed (also no
change in semantics)
* if auto_install is set to a list of dependencies, the module will be
installed when all *these* dependencies are installed, other
dependencies (excluded from auto_install) will be installed
alongside as a consequence
* auto_install can be set to an empty list, in this case the module
will always be automatically installed regardless of its
dependencies (and will force their installation).
So after this change, `web_dashboard`'s auto_install can be set to
`False` (such that it's not installed if no module defining dashboards
is installed) and `website_sale_dashboard`'s manifest can be edited
to:
'auto_install': ['website_sale']
possibilities:
# no automatic installation
'depends': ['a', 'b'],
'auto_install': False
# automatic installation if both a and b are installed
'depends': ['a', 'b'],
'auto_install': True
# automatic installation if both a and b are installed (explicit)
'depends': ['a', 'b'],
'auto_install': ['a', 'b']
# automatic installation if b is installed, a will get forcefully
# installed if it isn't yet
'depends': ['a', 'b'],
'auto_install': ['b']
# always automatically installed, will cause the installation of
# its dependencies even if they're not marked explicitly
'depends': ['a', 'b'],
'auto_install': []
Task 1851328
closesodoo/odoo#29431
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Odoo no longer supports python 2, thus some of these helpers can and
have been replaced by python 3 built-ins, therefore there is no need for
them to stay defined.
The removed helpers are:
* izip, imap and ifilter
* unichr, text_type
* implements_to_string, implements_iterator
* string_types, integer_types
* to_native
The python 2 shims have also been removed, and only the python 3 helpers
have been kept, because they can still be usable (i.e. accepting
both bytes and str for functions that can only accept one of the two)
[REM] pyjsparser: remove PY3 shims
They're no longer necessary as Odoo doesn't officially support python 2
anymore.
closesodoo/odoo#28519
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
This is necessary to get --dev=xml working on windows, because the xml
import uses normcase in file_open and the case normalized filename is
then used to find the addon path (which was not found before on windows)
With this fix arch_fs is now correctly filled in windows and --dev=xml
works.
closesodoo/odoo#28331
Purpose: When running tests, all the tests for the installed/updated
files are done. This commit adds a 'tagged' decorator that can be used to
tag tests. Combined with a new 'test-tags' CLI option, it adds the ability
to filter which tests are executed. For example, @tagged('slow') will
add a tag 'slow' to the test. The CLI option 'test-tags="slow"' will
only run tests tagged 'slow'.
One can use prefixes to select cases with tags.
'+' or no prefix means that the tests tagged with this tag are selected
for execution. '-' prefix will exclude the tests tagged with this tag.
Exclusion takes precedence over inclusion.
Also, by default, all Odoo tests cases are tagged 'standard' and with
the technical name of the module.
This means that when selecting tests with the 'test-tags'
parameter, if '-standard' is not specified, all tests tags are
going to be executed.
When tagging tests, one can remove such automatic tag by prefixing the
tag name with '-'. E.g. @tagged('-standard') will remove the standard
tag from the test.
Another example, if one wants to test the 'sale' module alone,
even without adding any 'tagged' decorator thos tests can be selected
like that: --test-tags="sale"
Tests are selected or deselected using a TagsSelector. When instanciated,
a string is passed with comma separated tests selectors like
'+slow,-standard'. When the 'check' method is called with a test as argument,
it returns True or False if the test has to be executed or not.
Before this fix, you can break a test class by simply adding an
incorrect import like this:
`from gloubiboulga import Casimir`
--> No error message, the test is simply not run
This is due to the fact that we want to ignore ImportError's... only
when there is no `.tests` submodules (actually, we ignored them all!)
This commit fixes the condition and re-enables error logging when tests
actually encounter ImportErrors
So that was a fun one: mock.patch calls would regularly fail refusing
to find the addon in odoo.addon (e.g. essentially getattr(odoo.addon,
'account_budget' deep within the bowels of mock).
Turns out the answer is that our import hooks would not be used for
many imports: while in Python 2, sys.meta_path is empty and the
default finders are run after all meta_path finders fail as noted by
the documentation[0].
However when the import system was rewritten in Python 3.3[1]
meta_path was "despecialised" and the default finders were moved to
meta_path rather than be a hidden part of the import machinery[2]:
> sys.meta_path and sys.path_hooks now store all of the meta path
> finders and path entry hooks used by import. Previously the finders
> were implicit and hidden within the C code of import instead of
> being directly exposed.
The result of this change is that ``sys.meta_path.append`` means the
default finders should take priority and the custom ones should be
fallback. This is the exact opposite of what we want.
Fix issue by ``sys.meta_path.insert``-ing our finders at the start of
the path rather than appending them at the end. This should change
nothing in Python 2 but seems to fix the issue in P3.
[0]
https://docs.python.org/2/library/sys.html?highlight=meta_path#sys.meta_path
[1] https://docs.python.org/3/whatsnew/3.3.html#importlib
[2] https://docs.python.org/3/whatsnew/3.3.html#visible-changes
In Python 3:
* various builtins and dict methods were changed to return
view/iterable objects rather than lists
* and the separate Python 2 view/iterable builtins and methods were
removed altogether
This is problematic when using these items as list (which the happens
repeatedly in Odoo), but more viciously when iterating *multiple times*
over them (which also happens, which I've messed up multiple times while
writing this, and which is a pain to debug even when you've just created
the issue).
Convert all code using these to semantics-matching cross-version
helper functions to get the LCD behaviour between P2 and P3, and
forbid the builtins via lint.
issue #8530
Xavier Morel