This commit adds tooling to profile performance and save execution by
saving stack traces and queries to a file/database in specific format.
----------
Collectors
----------
For now, three different profiling modes (aka Collectors) are available
even if a last once should be introduced by @Gorash to profile qweb
execution.
- SQLCollector (or 'sql'): Saves the current stack trace and the query
every time Cursor.execute() is called. Any query executed on the thread
will be collected, no matter the cursor.
- PeriodicCollector (or 'traces_async'): Saves the stack trace every
'interval' seconds using a parallel thread to profile the caller thread.
The python implementation was optimized to minimize impact on
performance while remaining portable and easy to enable/disable
inside a odoo execution. Higher the frequency (lower the interval),
more impactful the profiling will become on the execution and increase
memory usage. From last experiments, 1ms looks to be a good minimum for
short executions.
- SyncCollector (or 'traces_sync'): Saves the stack trace every function
call/return. This collector is obviously quite impactful on performance
and can quickly overload the memory for long executions, but this is
quite useful to understand the precise path followed by some short
executions. Any time related information will be almost irrelevant with this
collector.
A base Collector defining minimal collectors features can easily be
extended to create custom collectors if needed.
----------------
Profiler & Usage
----------------
Collectors are not supposed to be used by themselves, but should be
given to a Profiler. The Profiler will synchronize collectors starts and
stop, and manage saving them to a file of in a ir_profile in the
database.
Exemple of usage:
```
with Profiler():
do_stuff()
```
This simple example will use the default collectors (sql and
traces_async) and save them to the database. The database is defined
automatically from current_thread 'dbname' if available.
Example of usage:
```
with Profiler(collectors=['sql'], db=False, path=/home/user/logs/do_stuff_profile/{time}):
do_stuff()
```
This more complex example disable the default behavior consisting
to save to the database, gives a path where the profile will be saved
and specify to only use the 'sql' collector. Note that
collectors=[SQLCollector()] would have the same behavior since
Collectors can be either a Collector instance or a string describing the
desired collector. This allows to define custom params for the
collectors and use custom collectors if needed.
Note that it is always possible to get results after execution without
saving it since they are available on the profiler.
```
with Profiler(collectors=['sql'], db=False) as p:
do_stuff()
print(len([None for entry in p.collectors[0].entries if ...]))
```
Profiler will also save the stack below the profiler start point, and
collectors will only collect the part of the stack over this stack.
This is a good way to reduce collectors CPU and memory usage.
Collected entries will be saved as follows:
```
[{
'start': 2.0,
'context': {},
'stack': [
['path_to_file', lno, 'func_name', 'line_content'],
...
],
},
...
]
```
SQLCollector will add three additional keys on each entry:
- query (query without parameters)
- full_query (mogrified query with parameters)
- time (the 'exact' execution time of the query)
----------------
ExecutionContext
----------------
A last tool, ExecutionContext, allows to define some context on some block of code:
Example of usage:
```
def process_modules(modules)
for module in modules:
with ExecutionContext(module=module): # note the 'not linter frienldy but still convenient' 2 spaces indentation
do_stuff(module):
```
This context will automatically be added in the stack as a virtual frame between
process_modules and do_stuff in order to split do_stuff from one single frame to
one frame per module.
----------
Speedscope
----------
The saved data are in a simple json format easy to analyze, but can't be visualized in
speedscope as they are. A utility class `Speedscope` can be used to generate a format
readable by speedscope. The used format is actually the format defined by speedscope,
meaning that all features should be available using it.
The output format is evented, meaning that we need to transform a list of samples
(a list of stack) to a list of event (going in/out a frame).
This is the main task of the Speedscope, as well as combining samples from different
sources, to display SQLCollector and PeriodicCollector results mixed together.
When stored on an ir_profile, the default speedscope generation can easily be generated
with the speedscope computed field.
This class can be used as it is but will mainly be useful for the next commit.
Special thanks to @rco-odoo for the in depth review and @Gorash for support.
Description of the issue/feature this PR addresses:
It is currently quite difficult to differentiate users. Most of the time, people
don't take the time to upload an actual avatar so everybody looks the same. This
PR generates a custom avatar with the users initials and random color to
differentiate them. For res.users, res.partner and hr.employee, image fields now
hold the binary image and avatar are used to show the image or svg.
Current behavior before PR:
Avatar had only random colors and was being saved in database, being inefficient
Desired behavior after PR is merged:
A new mixin defines image fields and in case no image is set, it generates an
SVG image with the user's initials and random color.
closesodoo/odoo#69819
Task: 2404630
Related: odoo/enterprise#18199
Signed-off-by: Sébastien Theys (seb) <seb@odoo.com>
This commit changes the way assets are declared in Odoo modules.
Before: assets were declared in template files. Template bundles were
generated from primary templates, so technically any qweb template could
have been called as an asset bundle, with the 't-call-assets' directive.
Being standard qweb templates, they had access to standard HTML tags
(script, link, with or without raw scripts or style definition), qweb
directives (t-call, t-raw, etc.) and could be inherited by other
templates.
Now: assets are defined in the module's manifest and generated by the
't-call-assets' directive.
More information on the new system can be found on the updated user
documentation (see the "JavaScript Reference" section).
Task: 2352566
Co-authored-by: Bruno Boi <boi@odoo.com>
Co-authored-by: Julien Mougenot <jum@odoo.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
Co-authored-by: Mathieu Duckerts-Antoine <dam@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
Co-authored-by: Simon Genin <ges@odoo.com>
The onboarding modal for setting up the few base fields of a company
has now been moved to a wizard
It is accessible from the general settings, but also in the onboarding
section of sale and account modules.
The following company settings are editable with that wizard:
- Set report **layout**:
The user can chose the overall look of the report. The current choices
are : *Standard* (default), *Background*, *Boxed* and *Clean*.
- Set company **logo**:
Changes the company logo.
- Set report **colors**:
The user can set the primary and secondary colors of the report through
a newly added widget allowing to pick a custom color.
When changing the **logo**, colors are automatically set to its most dominant
colors.
> A "Reset colors" button also triggers the color calculation.
- Set report **font**:
Changes the overall font of the report. Only Google Fonts are used
for enhanced compatibility.
- Company **tagline**, also called "header"
- **Footer**
- **Paper format**
- Report **preview**:
A mockup of a final report
Automatically updates when changing **layout**, **logo**, **colors** or **font**
Co-authored by: Julien Mougenot <jum@odoo.com>
closesodoo/odoo#33863
Signed-off-by: VincentSchippefilt <VincentSchippefilt@users.noreply.github.com>
Co-authored-by: Lucas Perais <lpe@odoo.com>
As the decimal_precision module was removed, allow code like
from odoo.addons import decimal_precision as dp
fields.Float(digits=dp.get_precision('Foo')
to keep working
Co-authored-by: Olivier Dony <odo@odoo.com>
The decimal precision feature makes sense to be an ORM feature, no need to be
in a specific module
Previous syntax was
from odoo.addons import decimal_precision as dp
fields.Float(digits=dp.get_precision('Foo'))
and now is:
fields.Float(digits='Foo')
Remove the possibility to have a callable method on the digits attribute (it
was only used for precision anyway) and directly retrieve the digits on the
decimal.precision model
Rename the method digits to get_digits to avoid confusion between the field
attribute when declaring a field and the method to retrieve the precision
Task id: 48198
* = account, purchase, website_sale_comparison, website_sale_wishlist
Raw Image & Mixin
=================
Previously, the main image from a product was stored resized. This caused
inconsistencies with the size of the images on ecommerce: the main image was
resized, but the extra images were not.
Now, we always store all raw images, thanks to the new mixin. This way, product
images that were created before the installation of ecommerce will be ready to
be used in ecommerce without any other change.
We also store the resized versions instead of computing them on the fly: this
will take more disk space, but significantly increase the speed of the
pages when displaying images, as well as reduce server CPU load.
Images on variants
==================
Previously, it was only possible to set the extra product images on a template,
and not on a variant.
Now, we allow extra images also on the variants, this gives more flexibility to
the user.
Both extra images on template and variant have been given a sequence field to be
able to sort them easily.
On /shop we always display the image of the template for performance reasons.
On /product we never display the image of the template, unless the variant has
no image, then the template image is the fallback.
Carousel
========
The carousel view has been cleaned and improved:
- remove the duplicate XML that was created with the product configurator fix
- use a single loop from a single source to get all the images, instead of
merging the different sources in the view with complex conditions
- improve the HTML structure, reduce the CSS with appropriate classes
Preview Image
=============
Both in website and backend: make better use of `preview_image` to display
a small image whenever possible, to reduce download size for the end user.
task-34045
PR: #30656
Co-authored-by: Parth Gajjar <pga@odoo.com>
Co-authored-by: Sébastien Theys <seb@odoo.com>
Task 1856935
Currently, if installing a module's demo data fails (because the
module was uninstalled and some were left over and can't be
reinstalled, or because they're broken, or…) the installation of the
module fails and further installations are skipped (?).
Since demo data are non-essential (though required to run tests),
rather than fail everything if they fail just roll them back and
notify the user.
This change generates a warning log *and* shows a notification popup
to the end user.
Xavier-Do