Commit Graph
802 Commits
Author SHA1 Message Date
Thibault Francois fc65ae4dfe [FIX] cloc: do not count studio qweb
Since
https://github.com/odoo/odoo/commit/d2e0b48271dbd8cd7bbdbc5c2c1649c55c51417b,
qweb view from imported module are counted. the issue is that qweb view
created with the wysiwig editor of studio got an external ID from
studio_customization module which is declared as imported

So those views were wrongly counted for the maintenance fee.

Solution
--------
Exclude qweb view from studio_customization module

closes odoo/odoo#87424

X-original-commit: 77db024140e4f7c9ae8e6b5eb95de857afbab0da
Signed-off-by: Christophe Simonis <chs@odoo.com>
2022-03-29 10:56:40 +02:00
Gorash 7ddf8e4d21 [IMP] base: modify the structure of generated methods
This flat structure makes caching easier if odoo adds shared caches for
example. In addition, the use of idempotent names makes it possible to
compare methods during development.

closes odoo/odoo#85110

Related: odoo/design-themes#554
Related: odoo/enterprise#24622
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2022-03-29 10:56:17 +02:00
Gorash 880954ebfc [IMP] *: remove _render from ir.ui.view and simplify report
There were inconsistencies in the calls to `_render`.
* the view context could contain information that misled developers.
Indeed, the context and value of the view are not supposed to be found
in the rendering. Thus by calling `ir.qweb` with the name of the
template, we ensure that there is no unwanted information and in
addition the cache key is that of the name of the template which saves
a query.
* the context used for rendering was modified by a method on
`ir.ui.view`, except this is not information used by this model. There
is now a `_prepare_environment` method residing on `ir.qweb`. This
method allows to modify the value dictionary as well as the context in
which the rendering will be done. This preparation of the data as well
as my security check is done only once per rendering. This also saves
some queries
* Freeze options for rendering were inconsistent. It could be that
options on which rendering depends were not part of the cache key. Thus,
depending on the user who generated the generation of the rendering
function, there was or was not information in the template. For example
for automatic branding. This is no longer possible, because it is the
context that is used. The options serving as a cache key are only
recorded for information (for the profiling system for example). A
simplification of the `ir.qweb.field` models could be made.

The report rendering and call `ir.qweb` instead of `ir.ui.view`.

Part-of: odoo/odoo#85110
2022-03-29 10:56:15 +02:00
Thibault Francois dc8f06624f [ADD] cloc: count loc in style file and from imported module
Context
-------

Web design specific require a lot of work and need maintenance.
So far the style files were not taken into account despite the
amount of maintenance they require.

With this commit they are going to be counted,
knowing they still can be excluded in the manifest.

Imported module allow to deploy frontend assets: stylesheet,
javascript, xml template and Qweb view that require
as well some maintenance. They are going to be counted

Implementation
--------------

- Add method to parse css and scss file
- Include .scss and .css file in the count

- Add external_id to attachment that store
  frontend asset of imported module
- Find all attachment with .js, .css, .scss, .xml
  from imported module
- Find qweb view from imported module
- cound the content of the attachment and the qweb view

closes odoo/odoo#86816

X-original-commit: d2e0b48271dbd8cd7bbdbc5c2c1649c55c51417b
Signed-off-by: Christophe Simonis <chs@odoo.com>
Signed-off-by: Thibault Francois <tfr@odoo.com>
2022-03-21 17:57:05 +01:00
Fabio Barbero 47041f2d45 [IMP] base: allow user to activate multiple languages at once
Purpose
=======
Add "Activate" button when selecting multiple languages, select multiple
languages when clicking on "Add languages" in settings.

Specifications
=============
`lang` variable in `base.language.install` changed from Selection to
Many2many to allow multiple languages being activated at once.
Hide globe icon for language fields from view mode (only visible when
editing). Remove state in base.language.install since it's no longer
need to keep track of the installation step.

Task-2662548

closes odoo/odoo#78287

Related: odoo/upgrade#2921
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
2022-03-14 08:42:58 +00:00
John Laterre (jol) 32dee3dd5e [IMP] account: add Vendor Bill ref. on Invoice PDF
The goal is to add a digital stamp on the Invoice PDF
stating the name of its related Vendor Bill.

Auditors may request printed copies of all (original) Invoices
as well as the possibility to relate each document
to a posted entry (Vendor Bill).

task-2755372

closes odoo/odoo#86127

X-original-commit: f0ea39791c70cc63fe39665514e5344a56ed5b71
Signed-off-by: William André (wan) <wan@odoo.com>
Signed-off-by: John Laterre <jol@odoo.com>
2022-03-09 15:19:04 +00:00
Thibault Francois e1983aa413 [FIX] base{_import_module}: cloc should not count generated code
* Studio generate computed field "count" when the user add a button
in the button box. Since those lines are written by studio with drag and drop
it should not be counted for the maintenance subscription.

We can detect them because field created by the user in studio start with x_studio
and fields created outside studio does not have studio_customization
xml_id

* Add test to make no standard module introduce customization in the
database during the install the will be counted by cloc

see https://github.com/odoo/enterprise/pull/22664

closes odoo/odoo#85763

X-original-commit: 47ae24081fe5c0c86ce8507229dfb550ec585633
Signed-off-by: Christophe Simonis <chs@odoo.com>
Signed-off-by: Thibault Francois <tfr@odoo.com>
2022-03-04 11:16:52 +00:00
Julien Castiaux c0647b5c52 [REF] core: HTTPocalypse (14) changes all addons
This commit is the 14th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.

* `request.uid = x` => `request.update_env(user=x)`.
* `request.context = x` => `request.update_env(context=x)`.
* `request.context = dict(request.context, x=y)`
   => `request.update_context(x=y)`.
* `request.cr = None` => `request.cr.close()`.
* `http.mono_db()` => `request.db`.
* `http.dispatch_rpc()` => `service.dispatch_rpc()`.
* `@service.model.check` => `service.model.retrying()`.
* `request.endpoint`
   => `env['ir.http']._match(request.httprequest.path)[0].endpoint`.
* `request.routing_iteration `=> `removed`.
* `request.jsonrequest` => `request.dispatcher.jsonrequest`.

Note that `request.params` is now set much later in the process. If you
are in a situation where you values from the query string or the
http body you can use `request.get_http_params()`.

Note that using the new `request.future_response`, it is possible to
add headers and cookies on the response object before the response
object is initialized. Please note that headers/cookies saved on
the future response will NOT be injected in case of error.

PR: odoo#78857
Task: 2571224
2022-02-24 13:30:51 +00:00
Julien Castiaux 347a3ccf76 [REF] core: HTTPocalypse (4) route binding
This commit is the 4th commit of a comprehensive refactor of our HTTP
framework. See odoo/odoo#78857 for complete historic, discussions and
rationnals.

Complete refactor of the "registry" of controllers. The `ControllerType`
metaclass have been replaced by an abstract class with a py3.7
`__init_subclass__`. The `Endpoint` class is gone too, replaced by a
clever usage of `functools.partial`. This refactor is "pure", it doesn't
add any new feature, it only merely adds a few warnings.

The four `@route`, `Controller`, `_generate_routing_rule`, `routing_map`
work as follow:

1. A reference to each immediate child class of `Controller` (not grand-
children) is registered in a global list indexed by module (thus a
dictionnary) everytime the server starts. Remember that every first-
child (not grand-children) of `Controller` is the primary controller,
the one that can later be extended by other controllers (the grand-
children) in other modules. Remember that it is possible to get each
class's children via the `__subclasses__` dunder method.

2. Every controller method that is decorated with `@route` is granted an
attribute: `original_routing`, a dictionnary containing the `@route`
arguments. When a controller method has the `original_routing`
attribute, this method is called an `endpoint`.

3. `_generate_routing_rules` receives the list of installed module
names, this list is topologicaly sorted according to the modules
dependencies. That is `base` comes before `web` in this list. The
objectif of this function is to pair each route to an endpoint whoose
class's MRO respect the above-mentioned order. Whe achieve this by
carefully crafting classes at runtime, classes inheritating from the
correct "source code" controllers in accordance to the topology. This
method is also responsible of merging each method's `original_routing`
into one `routing` dictionnary, the very `rule.endpoint.routing` dict
that is used through the rest of the http framework.

4. Each route-endpoint pair is saved into a `routing_map`, an object
that bind each route to its endpoint. This object exposes a `match`
method used to find back the endpoint given its route (=http path).

In addition to this refactor, we added some new helpers in our tools,
among them `submap` that implement a kind of `dict() - set() -> dict()`
operator. Filtering a dict on a set of keys is a common operation but
the python standard library lacks a dedicated operator.

PR: odoo#78857
Task: 2571224
2022-02-24 13:30:48 +00:00
Samuel Degueldre 284ce4b7a4 [IMP] tools, test_js_transpiler: add support for import default + named
Previously, importing the default export on the same line as the named
exports was not supported, this commit adds support for that in order to
allow removing duplicate imports.

before:
import something from "path";
import { A } from "path";

after:
import something, { A } from "path";

Part-of: odoo/odoo#84530
2022-02-15 09:26:53 +00:00
Victor Feyens 9a0bf4785d [IMP] base: display query counts on profiling records
closes odoo/odoo#83746

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2022-02-08 10:03:59 +00:00
Christophe Simonis beb6a719bc [FIX] core: do not translate html attributes without actual content
The issue has been introduced by 1fe4b0cc1981382cc3d944ec44276f0aee90945d.

A simple example is an element with attribute "string" like

    <field name="foo" string=" "/>

When the translation process is parsing the value of the attribute as
some HTML fragment html.from_string(" "), it raises an exception.  This
was not a problem before the commit above, but it now is.  For instance,
updating the view "event.view_event_form" triggers the error.

closes odoo/odoo#84144

X-original-commit: 5ffc1704162d8918985964b08f4b2b49e6054958
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Christophe Simonis <chs@odoo.com>
2022-02-08 00:07:26 +00:00
a6f310fca3 [FIX] base: ignore tags in translations matching
When comparing existing translations to new source value
html tag would invalidate existing translations.
The html tags used for styling (coloring, italic, ...) should not
invalidate the translation as the meaning of the text as not changed.

Changed the logic to compare translations based on textual content only.

+ add some tests.

task-2667950

closes odoo/odoo#83895

X-original-commit: 1fe4b0cc1981382cc3d944ec44276f0aee90945d
Signed-off-by: Antoine Guenet <age@odoo.com>
Signed-off-by: Geelen Sébastien (sge) <sge@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
Co-authored-by: xmo-odoo <xmo@odoo.com>
2022-02-03 11:45:55 +00:00
Gorash e830953570 [IMP] IrQweb: refactoring and add technical documentation
Major changes:
- Remove some of the recursively when compiling
- Compile attributes became a directive
- Compile options became a directive
- `t-field` compilation now uses the same logic as `t-out`
- Simplification of ``t-if`` directive compilation
- Improved handling of errors wrapped by QWebException
- Constants defined outside the class

    Odoo
     ┗━► _render (returns MarkupSafe)
        ┗━► _compile (returns function)                                        ◄━━━━━━━━━━┓
           ┗━► _compile_node (returns code string array)                       ◄━━━━━━━━┓ ┃
              ┃  (skip the current node if found t-qweb-skip)                           ┃ ┃
              ┃  (add technical directives: t-tag-open, t-tag-close, t-inner-content)   ┃ ┃
              ┃                                                                         ┃ ┃
              ┣━► _directives_eval_order (defined directive order)                      ┃ ┃
              ┣━► _compile_directives (loop)    Consume all remaining directives ◄━━━┓  ┃ ┃
              ┃  ┃                              (e.g.: to change the indentation)    ┃  ┃ ┃
              ┃  ┣━► _compile_directive                                              ┃  ┃ ┃
              ┃  ┃    ┗━► t-if            ━━► _compile_directive_if                 ━┫  ┃ ┃
              ┃  ┃    ┗━► t-foreach       ━━► _compile_directive_foreach            ━┛  ┃ ┃
              ┃  ┃    ┗━► t-inner-content ━━► _compile_directive_inner_content ◄━━━━━┓ ━┛ ┃
              ┃  ┃    ┗━► t-options       ━━► _compile_directive_options             ┃    ┃
              ┃  ┃    ┗━► t-call          ━━► _compile_directive_call               ━┫ ━━━┛
              ┃  ┃    ┗━► t-att           ━━► _compile_directive_att                 ┃
              ┃  ┃    ┗━► t-tag-open      ━━► _compile_directive_open          ◄━━┓  ┃
              ┃  ┃    ┗━► t-tag-close     ━━► _compile_directive_close         ◄━━┫  ┃
              ┃  ┃    ┗━► t-out           ━━► _compile_directive_out             ━┛ ━┫ ◄━━┓
              ┃  ┃    ┗━► t-field         ━━► _compile_directive_field               ┃   ━┫
              ┃  ┃    ┗━► t-esc           ━━► _compile_directive_esc                 ┃   ━┛
              ┃  ┃    ┗━► t-*             ━━► ...                                    ┃
              ┃  ┃                                                                   ┃
              ┗━━┻━► _compile_static_node                                           ━┛

closes odoo/odoo#81024

Related: odoo/enterprise#22942
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2022-02-03 08:09:31 +00:00
Gorash 9ce5bc8881 [IMP] IrQweb: merge Qweb engine file qweb.py and ir_qweb.py
QWeb is the primary templating engine used by Odoo. It is an XML
templating engine and used mostly to generate XML, HTML fragments and
pages.

To create new XML template, please see :doc:`QWeb Templates documentation
<https://www.odoo.com/documentation/15.0/developer/reference/frontend/qweb.html>`

In **input** you have an XML template giving the corresponding input
etree. Each etree input nodes are used to generate a python function.
This fonction is called and will give the XML **output**.
The ``_compile`` method is responsible to generate the function from the
etree, that function is a python generator that yield one output line at a
time. This generator is consumed by ``_render``. The generated function is
orm cached.

In the graphic below you can see theresume of the call of the methods
performed in the IrQweb class.

    Odoo
     ┗━► _render (returns MarkupSafe)
        ┗━► _compile (returns function)                                        ◄━━━━━━━━━┓
           ┗━► _compile_node (returns code string array)                       ◄━━━━━━━┓ ┃
              ┃  (add technical directives: t-inner-content, t-tag)                    ┃ ┃
              ┣━► _directives_eval_order (defined directive order)                     ┃ ┃
              ┃                                                                        ┃ ┃
              ┣━► _compile_directives                              (recursive) ◄━━━━┓  ┃ ┃
              ┃  ┣━► _compile_directive                                             ┃  ┃ ┃
              ┃  ┃    ┗━► t-if            ━━► _compile_directive_if                ━┫  ┃ ┃
              ┃  ┃    ┗━► t-foreach       ━━► _compile_directive_foreach           ━┫  ┃ ┃
              ┃  ┃    ┗━► t-*             ━━► ...                                  ━┛  ┃ ┃
              ┃  ┃    ┗━► t-inner-content ━━► _compile_directive_inner_content ◄━━━━┓ ━┛ ┃
              ┃  ┃    ┗━► t-tag           ━━► _compile_directive_tag               ━┫    ┃
              ┃  ┃    ┗━► t-call          ━━► _compile_directive_call              ━┫ ━━━┛
              ┃  ┃    ┗━► t-out           ━━► _compile_directive_out           ◄━┓ ━┫
              ┃  ┃    ┗━► t-field         ━━► _compile_directive_field          ━┛  ┃
              ┃  ┃                                                                  ┃
              ┗━━┻━► _compile_static_node                                          ━┛

Part-of: odoo/odoo#81024
2022-02-03 08:09:31 +00:00
Gorash 39ce6f02a6 [IMP] Profiler: more QWeb directive informations in speedscope graph
Part-of: odoo/odoo#81024
2022-02-03 08:09:31 +00:00
Rémy Voet (ryv) 27fe7bf3a3 [REM] core: remove check decorator
The `check` decorator in  `sql_db.py` was on a lot of `Cursor` methods.
It checks if the cursor is close before be using a the method.
Remove it because:
- It is completly redundant because `psycopg2` do already the job
to check the cursor before usage.
- It complicated the call stack and lead to a small overhead of
highly use methods (can be more than 1% of the execute call)
- Also the nature of
the Error isn't correct: raise `OperationalError`
(https://www.psycopg.org/docs/module.html#psycopg2.OperationalError)
instead of  `InterfaceError`
(https://www.psycopg.org/docs/module.html#psycopg2.InterfaceError).

Part-of: odoo/odoo#80961
2022-02-01 14:37:42 +00:00
Raphael Collet 75e6b645ac [IMP] core: install PG extension "pg_trgm" only if necessary
Task 2742526

closes odoo/odoo#83274

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-01-28 14:10:02 +00:00
Raphael Collet a1904aa6f6 [IMP] core: field index names
The possible index names have been renamed "btree", "btree_not_null"
(instead of "not null") and "trigram" (instead of "gin").

Task 2742526

Part-of: odoo/odoo#83274
2022-01-28 14:10:01 +00:00
Xavier Morel 722f0f9b65 [IMP] core: deduplicate @locked and @synchronized, use in Registry
It's not entirely clear whether `@synchronized` is even useful, but
keep it for now. `locked` is just the default instance of
`@synchronised`.

- rewrite `@synchronized` using `decorator`, don't fold everything
  into a single call as there's a potential for parametric conflict
- remove the independent `locked` in `sql_db.py`
- convert `lru` to `locked`
- move `Registry` over to `locked` where applicable

closes odoo/odoo#82718

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-01-26 08:48:43 +00:00
Fabien Pinckaers 5b72f01766 [IMP] cleanup of base_address_city & base_address_extension
Rely on country.address_view_id instead of hacking the view on the fly. The
address layout (street VS street_name street_number street_number2) depends now
on the country of the user, not on the installed localisation. (it's now
configurable per country)

Unified street format to "Chaussee de Namur 40 - Appt 12"; the
configurable ones where actually wrong by country.

Allow street split without base_address_extended; all EDIs can now be
used with or without base_address_extended.

Merged base_address_city into base_address_extension, to avoid creating bridge
modules for no reason. Uses city_id instead of city if the country of the user
defines it AND if enforce_cities is set on this country.

Improved post_init script for large databases.

Split of street removed on res.company. (not required by any l10n)

[IMP] l10n_cn_city,l10n_nl,l10n_cl,l10n_co,l10n_pe: base_address_extended improvements

l10n_cn_city defined cities but did not provide a means to edit the city_id, so enforce_cities
l10n_nl only requires street names and numbers and thus does not need to depend on base_address_extended
l10n_cl should not depend on base_address_extended as none of the features are used
l10n_pe improve the view
l10n_co remove base_address_city dependency - move to l10n_co_edi

closes odoo/odoo#81710

Related: odoo/enterprise#23108
Related: odoo/upgrade#3129
Signed-off-by: Laurent Smet <las@odoo.com>
2022-01-25 17:21:04 +00:00
Xavier-Do 292645573b [IMP] core: don't save config when setting admin password
When setting the password in the database manager the config is saved
automatically in a odoorc file.

This can be problematic, especially when testing locally, with
specific options.
Going in the database manager and creating a new database or changing
admin pasword will save all those options.

This commit proposes to only save the admin_passwd when modified.

Part-of: odoo/odoo#82874
2022-01-24 11:09:55 +00:00
Fabien Pinckaers 6b87526048 [IMP] Speed Imp: remove unnecessary base64 encode & decode
Avoid to base64 encode, then decode to process assets and images for a ~25% speed improvement.
Change image processing tool to work on images, rather than base64 encoded strings.

Performance is ~25% faster on assets & images:

  /web/assets/...frontend.min.css:    13ms to 7ms,  base64 enc/dec: 2 -> 0
  /web/image/XML_ID:                  10ms to 8ms,  base64 enc/dec: 3 -> 0
  /web/image/res.users/2/avatar_128:  40ms to 20ms, base64 enc/dec: 6 -> 2

closes odoo/odoo#82851

Related: odoo/enterprise#23537
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
2022-01-22 11:51:42 +00:00
Xavier Morel 62b731a3b8 [FIX] core: possibility of allocation error in native magic libraries
Both of the native `magic` libraries can trigger an allocation error
if the buffer being sniffed is large. This results in nothing useful
for the user and just triggers and exception when trying to create
large attachments (in the hundreds of MBs).

Fix by implementing the workaround suggested in
ahupp/python-magic#82, under the assumption that it should also work
fine for Debian's python-magic.

The pure-python trivial version of odoo should not have this issue, as
it does not normally copy the source buffer in full, it just seeks and
slices the buffer, and occasionally parses it as a zip (but does not
normally load everything in memory).

Task-2722110

closes odoo/odoo#83190

X-original-commit: 94fc66452f28a8c64b4077d6cc2dd17ea1f00aa6
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-01-21 15:58:59 +00:00
Fabien Pinckaers eedf37d6e2 [IMP] Better handling of indexes
Three supported types:
- btree (default for index=True)
- btree not null (when >90% of the data are null)
- gin trigram search (for char fields)

Review of indexes on all objects.

closes odoo/odoo#83015

Signed-off-by: Fabien Pinckaers <fp@odoo.com>
2022-01-19 16:52:23 +00:00
6367a89332 [FIX] core: import of o2m with recursive xml declaration
Let's imagine this data

```xml
<record model="my.model" id="1">
    <field name="name">parent</field>
    <field name="children_ids">
	<record model="my.model" id="1.1">
	    <field name="name">child</field>
	</record>
    </field>
</record>
```

Loading this data the first time, everything works fine.
But if we update it, this is what happens:
* load `my.model,1`: write on `name=parent` and `children_ids=None`
* load `my.model,1.1`: write on `name=child` and `parent_id=my.model,1`

The write on `children_ids=None` can unlink all the children if the
field is declared as `ondelete=cascade`
That will leade in all the records to be deleted and recreated, but
without keeping links with other documents.

For instance, when upgrading the `account` module, all the
`account.account.tag` are removed from `account.move.line` because when
the `account.tax.report.line` are updated, all the tags are deleted and
recreated.

closes odoo/odoo#82793

X-original-commit: de00316fa42173f3a5c394458bb4418e59e298cd
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: William André (wan) <wan@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
Co-authored-by: jva-odoo <jva@odoo.com>
Co-authored-by: william-andre <wan@odoo.com>
2022-01-14 16:37:50 +00:00
Xavier-Do 37227ad2cb [IMP] tests: avoid double logs on build page
Since auto-retry, real errors will lead to doubled errors on runbot.
This can be noisy and hard to understand.

This commit will help with this by using a lower log level on the first
execution. Log 25 are kepts.

closes odoo/odoo#80378

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2022-01-13 09:30:42 +00:00
Rémy Voet (ryv)andrco-odoo 504589efb2 [FIX] base: add __reversed__ in BaseModel for efficiency
Before when we do `reserved(records)`, it iterates in a reverse
order of `records`. Unfortunately, the `__reserved__` method don't
exist explicitly in BaseModel then Python fallback on
its own implementation using `__getitem__` and `__len__` (coming from Sequence): https://github.com/python/cpython/blob/3.10/Lib/_collections_abc.py#L1047-L1049
Because it uses __getitem__, it breaks the prefetch of the recordset.

Example:
-------------
```
partners = self.env['res.partner'].browse(1, 2, 3, 4, 5)
for partner in reversed(partners):
    partner.name
```
will generate 5 SQL requests to fetch data (one by record)

Then create our own `__reversed__` and handle the prefetch correctly
(like `__iter__`). Now in the example it will correctly generate only
1 SQL request because of the prefetch.

task-2687953

closes odoo/odoo#79622

Signed-off-by: Raphael Collet <rco@odoo.com>
Co-authored-by: rco-odoo <rco@odoo.com>
2022-01-12 10:17:36 +00:00
Fabien Pinckaers 8ae4d92a58 [IMP] base: remove postprocessing of rendered pages
First step to stream QWeb templates: removing the two post
processing operations applied on rendered templates. This
should slightly speed up the rendering of every page.

1/ Don't remove empty lines after rendering, but fix the root
cause of: view inheritancies and QWeb compilation that don't
add extra empty lines.

2/ handle page break in the two reports that uses it, rather
than processing every view produced.

closes odoo/odoo#82244

Related: odoo/enterprise#23328
Signed-off-by: Fabien Pinckaers <fp@odoo.com>
2022-01-10 19:28:29 +00:00
wan a77dee7b98 [FIX] core: typo in field type check
closes odoo/odoo#82238

X-original-commit: 5164739b835445fa11b2efc53e2ff839a4bad10a
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
Signed-off-by: William André (wan) <wan@odoo.com>
2022-01-04 20:16:47 +00:00
Xavier Morel 74241b3766 [FIX] test_lint: support fstrings in sql injection checker
Those were not accounted for, leading to fstrings passing through
unflagged.

Also update the SQL checker to be stricter but smarter:

The previous version would "fail open", unknown nodes would be allowed
through hence f-strings not being flagged when they started appearing
in arg0 position, should now fail-closed, anything that's not allowed
is forbidden.

This flags a few more cases, all of which seem acceptable upon review.

However the previous version would also only resolve arg0 (in case it
had a `NAME`, to see if that resolved to an acceptable form of
query-building). The new version performs resolution during
`_check_concatenation` and should thus allow e.g. format strings to be
separate variables (though not e.g. module-level constants, yet
anyway).

In resolution, replace the ad-hoc process by astroid's built-in
`lookup` which seems to provide the same information. Slightly more in
fact, as it yields every assignment in case of e.g. conditionals, but
making use of that would require a lot more changes in the checker so
leaving the behaviour as-is for now.

It's important to *not* use `ilookup` here, because ilookup is not
"iterable" but "inferring", and we don't want values, we want
expression ASTs for analysis.

NOTE: previous improvements as well as fixes to existing code were
only implemented in 14.0, hence this being merged in 14.0 not 13.0
despite 13.0 still being supported.

closes odoo/odoo#81721

X-original-commit: 376ccf0944dae1bc53ae9c5385977c4e6b23e083
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
2021-12-27 09:36:42 +00:00
Nicolas (vin) 08e9798d74 [FIX] [base, account_edi_facturx]: Fix pdf export
There is an issue when exporting pdf using edi documents created before
the PDF/A commit. With the subtype now included, the system would try
to use the subtype given by the ir.attachment which would not be formated
as expected by the pdf file format.

The attachment may get neutered by the ORM, so we may have to force
the mimetype when embedding it.

This fix in two parts will allow to force a subtype when adding an
attachment into a pdf, as well as parse the subtype of ir.attachment
to give them the right format.

xxx/xxx should become /xxx#2Fxxx

opw-2714040

closes odoo/odoo#81898

X-original-commit: 880d7a1c8474a3bee4fd05474788fbd4a63f4ae7
Signed-off-by: Laurent Smet <las@odoo.com>
2021-12-24 13:11:50 +00:00
Olivier Dony ea0e9ec0aa [FIX] tools.mail: ignore original email during encapsulation
When the system broadcasts an email response to document followers,
if the config parameters `mail.force.smtp.from` or
`mail.dynamic.smtp.from` are defined, it will rewrite the `From`
address to avoid spoofing the sender's domain.
**NOTE**: As of 15.0, this is based on the `from_filter` setting on the
corresponding ir.mail_server, rather than the abovementioned config
parameters, but the rest of the discussion stands.

For example, if the `mail.catchall.domain` is set to `example.com` and
an email response comes from:

   "John D" <john@doe.com>

it will rewrite it to:

   "John D (john@doe.com)" <notifications@example.com>

This will make sure the system never sends outgoing email for an external
domain, as it has no authority for doing so, and that could
break mail filtering/authentication rules (SPF, DMARC, etc.)

During this "encapsulation rewrite step", both the original Sender name
and their email are preserved, and put into the quoted "name" field of
the rewritten address. It seems sensible to preserve as much information
as possible about the original sender.

Unfortunately, the inclusion of the Sender email in the final name makes
it appear to some inbox providers as if the message is trying to
deceptively impersonate another person (as many phishing schemes would).
As of November 2021 GMail at least does this, and will hide the name in
the UI when it happens. It will keep only the rewritten email, which is not
very useful in the case of a notification (even though it's more
technically correct, of course).

This patch removes the original email from the rewritten notification,
keeping only the name, considering that the email is not the most
important part, and it's better to have one of the two than none.

So after the patch, the rewritten address is now:

   "John D" <notifications@example.com>

When there is no name in the original address, we keep only the local
part of the email, to avoid the same display issue. The recipient will
have to identify the sender based on the context / past messages.

closes odoo/odoo#81807

X-original-commit: 3c65ec5a8191a392980ceb0a8c584767eae405f1
Signed-off-by: Olivier Dony <odo@odoo.com>
2021-12-22 15:45:53 +00:00
Xavier Morel bdc9d9d369 [FIX] core; base: lots of docstrings
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
  fields) are out of scope for the lint but my editor catches

closes odoo/odoo#74604

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-12-09 14:36:58 +00:00
FrancoisGe 13d0741a47 [FIX] js_transpiler: exported hoisting function
After this commit, the js_transpiler will support exported hoisting
function. So it will be possible to define a hoisting function and
exported in an @odoo-module.

Example of exported hoisting function which wasn't supported before
this commit:
```
  hoisted();
  export function hoisted() {
    ...
  };
```

closes odoo/odoo#80965

X-original-commit: 40ef9f7aeac8048cce8f002c84181abc64bcbe87
Signed-off-by: Aaron Bohy (aab) <aab@odoo.com>
2021-12-07 14:41:46 +00:00
Nicolas (vin) 62954aaaf5 [IMP] [base, account_facturx]: Add PDF/A(-3B) support
Improve the factur-x export in two ways: make the exported PDF
PDF/A-3B compliant, and add the factur-x XMP metadata inside the file.

The added .ICC profile comes from https://www.color.org/srgbprofiles.xalter
License terms can be found here: https://www.color.org/profiles2.xalter#license

Task id # 2668919

closes odoo/odoo#80741

X-original-commit: 3a9a685fc360fee7fe10ed7fdb768414697673b2
Signed-off-by: Laurent Smet <las@odoo.com>
2021-12-04 21:45:04 +00:00
std-odoo 3b97f601be [FIX] base: avoid variable name collision with functions in QWeb
Bug
===
The arguments of the lambda function might be the same as other
variables in the QWeb. In that case the template rendering will crash.

To avoid this issue, we encapsulate the name of the argument to be sure
it's not the same as other variable in the QWeb engine (so the name of
the argument can no longer make the QWeb rendering crash)

Task-2674716

Closes #78392.

Forward-port of 96b09b127028e5c9a83c89acce86e1b6df904c6f
2021-12-01 17:30:51 +01:00
Martin Trigaux aa07f93ef7 [IMP] package: update lxml version
Update the version to benefit from the fix for
https://github.com/advisories/GHSA-pgww-xf46-h92r
https://nvd.nist.gov/vuln/detail/CVE-2020-27783

This vulnerability is reproducible in Odoo with
html_sanitize(..., sanitize_tags=False) which does NOT happen for
user-facing content.

Remove old compatibility check (lxml 3.1 was released in 2013)
and cleanup global variables only used once

closes odoo/odoo#64248

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
2021-12-01 14:07:47 +00:00
wan 2b80f3e5f1 [IMP] core: allow nested declaration of xml for one2many fields
Some data are formatted as a tree structure with a relation of parent.
We are adding meaning to the structure:

* This improves readability and allows to fold/unfold records in
  editors.
* This could be done before using `eval`, but this allows to have an
  xml_id for those records. It also allows to update the records without
  having to rewrite on the parent x2many field.

Part-of: odoo/odoo#76675
2021-11-25 00:31:16 +00:00
Thibault Francois ade64948ab [FIX] cloc: re introduce exclusion of design theme
Since
https://github.com/odoo/odoo/commit/ebf8d67485f7f71f6eb51762298cf50c5835ee08
design-themes repo is not excluded from the cloc count.
The module website_animate that was used as beacon to identify
the folder that contains the design-themes as been merged into website.

We need a new one, theme_common seems a reasonable candidate

closes odoo/odoo#79512

X-original-commit: f493b43462b7db14bb7f2b444c189f284acb1a1d
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2021-11-08 17:48:49 +00:00
Rémy Voet (ryv) 837e72fbfa [REM] base: remove unused code in translate.py
Remove unused code of translate.py file:
- remove unused trans_parse_rml method (since 3425752eac)
- remove unused in_modules method (since eb04aa1e39)
- remove unused ENGLISH_SMALL_WORDS set (since d85bf78976)
- remove unused node_pattern compiled regex (since 085e972e4a)

task-2678192

closes odoo/odoo#79011

Signed-off-by: Rémy Voet <ryv@odoo.com>
2021-11-03 11:06:18 +00:00
Jeremy Kersten 300954d8c4 [IMP] base: auto resize attachment to 1080p
After some analyze on a lot of customer databases, seems like most of the time
their are performance probleme, and big store, it is due to  a lot of big file
uploaded without reason. E.g. barcode, photo, ... a small one will be enough.

Now, we decided (in stable) to auto resize these pictures to 1920x1920px
by default and compress it with a quality of 80 when the source is bigger.

You can bypass this behaviour in your specific use case,
using a context key: 'image_no_postprocess' set to True.

You can disable the resize (and quality implicitely)
using an icp: 'base.image_autoresize_max_px' set to '0'.

You can change the default resize (1920x1920) format using an icp:
'base.image_autoresize_max_px' set to '<width>x<height>' (e.g. '1024x768')

You can change the default quality (80) using an icp:
'base.image_autoresize_quality' with a value between 0 and 100 where 0 skip it.

You can change the type of file that will be post process using icp:
'base.image_autoresize_extensions' (subtype of the mimetype comma separated).

Api of image has not be changed in this commit, only refactored to allow to
work with image directly without the need to encode/Decode in base64 the raw.

We decide to keep 1920x1920 by default instead of 1080p to avoid to resize
portrait picture in 1080px and stay consistent with field image_1920 that
return a 1920px image for width or height whatever the orientation.

+ fix some lint diff for ci style in master

closes odoo/odoo#78556

X-original-commit: d9ce0507960f247e1187baf7bd8399f90be237aa
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
2021-10-22 17:34:29 +00:00
Xavier-Do 4f3b1653de [FIX] profiling: avoid crash with thread pools
When using a profiler manually inside the code, and longpolling uses
this part of the code, an error will appear on runbot since gevent
server cannot be profiled properly.
This fix mitigate the issue by disabling the profiler automatically in
this case.

Part-of: odoo/odoo#78514
2021-10-22 17:34:17 +00:00
Julien Castiaux 86c27b7156 [FIX] core: lower_logging record.args not str
As per the logging documentation, it is fine to send arguments that are
not string. Logging takes care of stringifying and injecting the
arguments in the message template.

closes odoo/odoo#78172

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-10-11 15:08:44 +00:00
Nicolas Bayet 4813f42997 [IMP] mail,*: replace jinja with qweb
Jinja as a templating engine was problematic in differents respect:
- introduce external dependency to Odoo (less controll)
- add another templating mechanism in the stack
- specific feature in qweb cannot be reused
- difficulty in rendering easily editable templates
- more knowledge required with no betterment

By replacing jinja with qweb we can now build tools to edit a qweb
that will work with the previously jinja encoded document
(essentially `mail.template` records).

There is a catch however. Some email fields (eg. email_to) used jinja
syntax for rendering dynamic variables (ie. ${object.something} and
${object.something_that_should_not_be_escaped | safe}).

We still want user to use dynamic variables for some char fields (eg.
subject, from, to, ...). We made a new rendering engine called
"inline_template" that will render an expression enclosed by `{{` and
`}}`.

To be able to edit the templates from the backend interface, a
plugin to the Odoo editor has been made for seamlessly edit the
document.

This qweb plugin includes:
- make dynamic variables (eg. `<t t-out="variable"/>`) not editable
  (for preventing the user to shoot himself in the foot)
- group and hide related logical branching (ie. t-if, t-elif, and t-else)
  in order to see only one at once
- a floating select input to switch visibility of a particular logical
  branching

Task-27033

X-original-commit: odoo/odoo@68182baff4
Part-of: odoo/odoo#77377
2021-09-28 23:42:54 +00:00
Jérémy Kersten (jke) 0e293f1b7e [FIX] tools: properly guess image/svg+xml mimetype with magic lib
Magic return image/svg instead of image/svg+xml if file starts with <svg

+ refactor the import/else to if/else to avoid pylint error:
function already defined line 137 (E0102) at odoo/odoo/tools/mimetypes.py:181

That raises an error during CI/runbot

closes odoo/odoo#77193

X-original-commit: b76f13853a25a93307db2bcafb6a83ad75b92244
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com>
2021-09-27 07:11:58 +00:00
Xavier-Do 8a7990735d [IMP] tests: autoretry mechanism for staging
This pr proposes an auto retry mechanism for tests. This shouldn't
impact normal testing: tests are not supposed to fail, but the growing
number of tests and pull requests can lead to some bottleneck when a
staging fails because of a random error. This mechanism should help to
reduce splits/the need to retry a failed pr.

This branch have been tested with the nightly multi build, creating 40
identical build without test-tags to disable know random errors.
This multi build is used to detect test failing randomly, this is an
excellent candidate to detect the effect of the retry.
On average, with the current base of this pull request, there is between
 10 en 15 failures over 40 build.
With the auto retry mechanism, only 1 build failed over 40 builds since
the same error was triggered twice.
This is simply because with the retry mechanism, an error that has a
probability of p to fail randomly will still have a probability of p² to
 fail with the retry mechanism. A error that occurs 10% of the time
 should only appear 1% of the time with one retry.  In most of the case,
  the retry is sucessfull: https://runbot.odoo.com/runbot/build/10053257

The current solution to allow to enable this mechanism only in some
cases (staging) is to check an environment variable
"ODOO_TEST_FAILURE_RETRIES" that defines a number of retry.
This will allow to retry more than once if an error still occurs to ofen
 with the autoretry.

The mechanism will run multiple time the same test on the same
test_case, meaning that some modification on self may impact the second
execution. The following code is an example of how this could be
problematic, but also a good example to test the auto-retry mechanism.

```python

class TestRetry(HttpCase):
    def test_fail(self):
        self.t = getattr(self, 't', 0) + 1
        if True or self.t == 1:
            import logging
            _logger = logging.getLogger('test_a')

            with self.assertLogs(level="ERROR"):
                _logger.error("This shouldn't be log at all")

            with mute_logger('test_a'):
                _logger.error("This shouldn't be logged (mute)")

            _logger.error("This should be log")
```

As we can see here the error logs are also managed, and emit at a lower level the first time, butany log higher than 25 will make the test "failed" and the autoretry mechanism will be triggered. The second time, everything is logged normally. We also need to replace Traceback by _Traceback to avoid being catched by runbot Traceback detection regexes.

The inspiration here commes from the assertLogs, that replace all handlers. The mute_logger had to be adapted to use the same strategy, so that quite_logger won't detect logs catched by mute_logger or assertLogs.

closes odoo/odoo#76336

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-09-24 13:59:56 +00:00
John Wilson 63c00f700d [FIX] various: --test-file on Windows
closes odoo/odoo#75341

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-09-14 06:00:52 +00:00
Xavier ALT 68d950de45 [FIX] cloc: also exclude module's "upgrades" folder by default
Since commit bbb1a8f151, module's upgrade script can also be added
into special 'upgrades' folder as an alternative to the old 'migrations'
folder.

  As those upgrade scripts are not maintained from version to version, we
should not count them as maintenance.

OPW-2536046

closes odoo/odoo#76334

X-original-commit: 28a303a67935dc5566cc0d1fe2be157736691968
Signed-off-by: Xavier Alt (xal) <xal@odoo.com>
2021-09-10 13:42:06 +00:00
Gorash 4d0e111733 [IMP] profiling: Add profile qweb execution into the performance tools.
By activating the profiler debugger, the two qweb options is added. The
qweb templates that need to be rendered are compiled into a new function
to add instructions for saving data.

`Add qweb directive context`
It's a sub-option of "Record sql" or "Record traces", add some context on
thread at current call stack level.  This context stored by collector
beside stack and is used by  Speedscope to add a level to the stack with
this qweb directive information.
```
    directive=t-call='website.layout', xpath=/t/t
    t_call_content
    directive=t-foreach='5' t-as="'a', xpath=/t/t/div/div
    directive=t-esc='website.search([])', xpath=/t/t/div/div/t
    execute
```

`Record qweb`
Add profiling data used by ProfilingQwebView widget. In the `ir.profile`
form view, the widget display the duration and number of sql of every qweb
directives with the xml of templates. Every xml is recorded to be
consulted even if the user change the xml templates.
```xml
    <t t-call="website.layout">
       <div>
          <div t-foreach="5" t-as="a">
             <t t-esc="website.search([])"/> <!-- will display 5 separate requests -->
          </div>
       </div>
    </t>
```

closes odoo/odoo#74712

Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2021-09-08 10:21:42 +00:00