Commit Graph
35 Commits
Author SHA1 Message Date
Raphael Collet 5468e41a4d [FIX] core: improve column ordering
This follows up e045e76e35.

Change the heuristics for ordering columns, as padding is not determined
by column size, but by column alignment inside a row.  Because in Odoo a
row always starts with a column of size 4, the following columns should
be the ones aligned on 4 bytes, then the ones aligned on 1 byte, then
the ones aligned on 8 bytes.

The analysis in the commit message of e045e76e was not correct, as it
did not take into account the fact that rows themselves are aligned on 8
bytes.  So we have: before each row used 40 bytes (+24b header)

       attname   |  typname  | typlen
    -------------+-----------+--------
     id          | int4      |      4
     create_uid  | int4      |      4
     create_date | timestamp |      8
     write_uid   | int4      |      4    -> 4 bytes padding
     write_date  | timestamp |      8
     active      | bool      |      1    -> 7 bytes padding

After each row uses 32 bytes (8 bytes saved per row):

       attname   |  typname  | typlen
    -------------+-----------+--------
     id          | int4      |      4
     create_uid  | int4      |      4
     write_uid   | int4      |      4
     active      | bool      |      1    -> 3 bytes padding
     create_date | timestamp |      8
     write_date  | timestamp |      8

Of course, when more columns are present, the space savings depend on
the alignment of the other columns.

closes odoo/odoo#88084

Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
2022-04-06 15:30:54 +02:00
Fabien Pinckaers e045e76e35 [IMP] base: reduce new DB size by ordering columns
Postgres is aligning columns to 4 or 8 bytes, depending on their type.
So, consecutive fixed-length columns of differing size will be padded
with empty bytes due to the alignment requirements.

Before this patch, columns where created in their definition order. Now,
they are ordered based on their size, in order to minimize the padding.

As an example, before each row uses 36 bytes (+24b header):

       attname   |  typname  | typlen
    -------------+-----------+--------
     id          | int4      |      4
     create_uid  | int4      |      4
     create_date | timestamp |      8
     write_uid   | int4      |      4    -> 4 bytes padding
     write_date  | timestamp |      8
     active      | bool      |      1    -> 3 bytes padding

After each row uses 32 bytes (4 bytes saved per row):

       attname   |  typname  | typlen
    -------------+-----------+--------
     id          | int4      |      4
     create_uid  | int4      |      4
     write_uid   | int4      |      4
     active      | bool      |      1    -> 3 bytes padding
     create_date | timestamp |      8
     write_date  | timestamp |      8

This saving scheme applies to all rows in all tables. We save between 4
and 8 bytes per row just on the usual create_uid, create_date,
write_uid, write_date.

closes odoo/odoo#87896

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-04-05 17:17:27 +02:00
Raphael Collet 75e6b645ac [IMP] core: install PG extension "pg_trgm" only if necessary
Task 2742526

closes odoo/odoo#83274

Signed-off-by: Raphael Collet <rco@odoo.com>
2022-01-28 14:10:02 +00:00
Raphael Collet a1904aa6f6 [IMP] core: field index names
The possible index names have been renamed "btree", "btree_not_null"
(instead of "not null") and "trigram" (instead of "gin").

Task 2742526

Part-of: odoo/odoo#83274
2022-01-28 14:10:01 +00:00
Fabien Pinckaers eedf37d6e2 [IMP] Better handling of indexes
Three supported types:
- btree (default for index=True)
- btree not null (when >90% of the data are null)
- gin trigram search (for char fields)

Review of indexes on all objects.

closes odoo/odoo#83015

Signed-off-by: Fabien Pinckaers <fp@odoo.com>
2022-01-19 16:52:23 +00:00
Xavier Morel bdc9d9d369 [FIX] core; base: lots of docstrings
* add configuration for `flake8[flake8-rst-docstring]`
* enable docstring-related checks
* fix invalid docstrings in odoo's core & `base`
* fix a few more bits (mostly missing or incorrect `:param:` info
  fields) are out of scope for the lint but my editor catches

closes odoo/odoo#74604

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2021-12-09 14:36:58 +00:00
Denis Ledoux b06e4454d5 [FIX] registry: check_foreign_keys, constraint names are limited to 63 chars
When computing the foreign key name,
`check_foreign_keys` didn't take into account the limit of 63 characters
for constraint names.

Because of this, some constraints were dropped and recreated
over and over while they were correct, during install and upgrades.

For instance, when installing `base`
when adding the foreign key for which the name was computed
`base_partner_merge_automatic_wizard_res_partner_rel_base_partner_merge_automatic_wizard_id_fkey`
Postgresql created the constraint under the name
`base_partner_merge_automatic__base_partner_merge_automatic_fkey`
and therefore, as the name did not match,
the constraint was dropped and re-created.

closes odoo/odoo#72234

X-original-commit: 43a4738ebf8a74a389b99f8f58330b3044beaa0c
Signed-off-by: Denis Ledoux (dle) <dle@odoo.com>
2021-06-16 13:15:26 +00:00
Stéphane Bidoul 13b40bff2c [FIX] base: allow using another postgres schema than public
Before 1721ec1363 and fdc4ef97c9, Odoo did work fine when the
user had another default schema than 'public'.
This commit restores this behaviour by searching
for existing objects in the user's current schema,
which is the first in the schema search path and
the one used when no schema is specified when
creating objects.

closes odoo/odoo#68144

X-original-commit: 223781b34afacd1c0c5674d395cece6d472b048c
Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2021-03-19 12:42:16 +00:00
Jeremy Kersten 6538e20bb6 [FIX] tools.sql, website_*: increment counter with skip lock
*blog, forum, slides

Based on https://github.com/odoo/odoo/pull/48552#discussion_r440061218 suggestion

Add a new method 'increment_skip_lock' in tools.sql to allow to easily
increment a specific field of 1 if the record is not locked.

The method return boolean if at least 1 record has been incremented.

closes odoo/odoo#58765

X-original-commit: d92e61e89f468c2db2fbd68b2f0d6b36c77f1065
Signed-off-by: Jérémy Kersten (jke) <jke@openerp.com>
2020-09-29 08:48:59 +00:00
Raphael Collet 3ecb6fdcb0 [IMP] core: optimize model._auto_init() on a new model
Create the table with all the columns from scratch, with the NOT NULL
constraint when required.  Also do not call `_check_removed_columns()`
on a new table.

This saves 0.5% of the total installation time.

X-original-commit: 0727cacf5194a143b15ab4cb9893f3035a67be1f
2020-04-03 11:40:28 +00:00
Adrian Torres f61262eb08 [FIX] core: delay constraint application in case of upgrade
closes odoo/odoo#44800

Co-authored-with: Xavier Dollé <xdo@odoo.com>
X-original-commit: bc2bb5e03c2b32d4ee1b0597ea5889c17d2b0e0e
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
2020-02-06 18:28:29 +00:00
Denis Vermylen cf0146934d [FIX] sql_db: ignore PG views when verifying table type
Collisions in table names of ORM models with built-in PG structures such
as "attributes", "domains", "routines", "parameters", ... could occur
and render the result of `table_kind` meaningless.

Based on what was done via https://github.com/odoo/odoo/pull/16651 more
than 2 years ago, it seems relying on our tables being in the 'public'
schema is safe, even though it's only a default from PG. We reuse that
same logic rather than the alternative of excluding
('information_schema', 'pg_catalog', ...), even though it looks safer at
first. If we did the latter we'd have to change the other comparison for
consistency, i.e. more risks.

closes odoo/odoo#42358

X-original-commit: d8e74eb14990c82f65a44ffe163aa84159d6ccc4
Signed-off-by: Denis Vermylen <Icallhimtest@users.noreply.github.com>
2019-12-24 19:08:11 +00:00
17c4f47b0a [FIX] base: allow custom model to use SQL (materialized) views instead of table
closes odoo/odoo#41267

X-original-commit: f17d389c4625e95f52c299c13a3f2983127c70bc
Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
Co-authored-by: Denis Ledoux <dle@odoo.com>
Co-authored-by: Raphael Collet <rco@odoo.com>
2019-12-03 07:45:25 +00:00
Moisés López a5251e1d40 [ADD] test_lint: Add sql-injection pylint check
closes odoo/odoo#36583

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
2019-10-02 14:16:38 +00:00
Raphael Collet c7f5c4afd2 [FIX] sql_db: add flush() in savepoint()
closes odoo/odoo#36060

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-08-26 13:39:16 +00:00
Raphael Collet 9920f20e4c [IMP] models: ORM speedup
This branch is the combination of several optimizations in the ORM:

* store field values once in the cache: the cache reflects more
faithfully the database, only fields that explicitly depend on the
context have an extra indirection in the cache;

* delay recomputations by default: use method `recompute` to explicitly
flush out pending recomputations;

* delay updates in method `write`: updates are stored in a data
structure that can be flushed efficiently to the database with method
`flush` (which also flush out recomputations);

* make method `modified` take advantage of inverse fields to inverse
dependencies;

* filter records by evaluating a domain on records in Python;

* a computed field with `readonly=False` behaves like a normal field
with an onchange method;

* computed fields are computed in superuser mode by default.

Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien
Pinckaers.

closes odoo/odoo#35659

Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
2019-08-20 12:43:59 +00:00
Stefan Rijnhart b3824aa6cb [RFR] Hide implementation of constraint definition storage, courtesy of Raphael Collet 2018-08-16 10:33:06 +02:00
Stefan Rijnhart 6327b8f97f [IMP] Store the original constraint definition as a comment on the constraint
and use it to determine if the constraint definition was changed. This prevents
endless readding of constraints that are reformatted by postgresql in an
unrecognizable way (in which e.g. "CHECK (credit*debit=0)" becomes
"CHECK ((credit * debit) = 0::numeric)")
2018-08-16 10:33:06 +02:00
Christophe Simonis c5f680200c [MERGE] forward port branch saas-17 up to d08b4407be 2017-09-12 12:09:11 +02:00
Christophe Simonis fdc4ef97c9 [FIX] core: only check table existance in public schema
This allow the use of tables defined in other schemas, like `domains`,
already defined in `information_schema`.

See #16651
2017-09-11 14:55:49 +02:00
Christophe Simonis d5382abeaa [MERGE] forward port branch saas-17 up to b8dd34fcbb 2017-09-06 17:40:59 +02:00
Christophe Simonis b8dd34fcbb [MERGE] forward port branch saas-16 up to 64d56995e0 2017-09-06 13:29:05 +02:00
Christophe Simonis 3ff4feafd2 [MERGE] forward port branch saas-15 up to 4d4d75709d 2017-09-04 18:12:00 +02:00
Christophe Simonis 4d4d75709d [FIX] core: also consider materialized views as existing
At the end of registry loading, a check is made on every model to
verify its table exists in the database. Materialized views weren't
considered during this check.

Note that we can't use `information_schema` views in this case because
it does include materialized views on purpose [1].

Forward-port of 4ca6945256

[1] http://www.postgresql-archive.org/Materialized-views-don-t-show-up-in-information-schema-tp5822643p5822644.html
2017-09-04 17:51:50 +02:00
Fabien Meghazi a867cce406 [IMP] sql: harvest less information from pg's information_schema (Fixes #18490)
Avoid `SELECT *` on `information_schema.columns` because specific access
right restrictions in the context of shared hosting (Heroku, OVH, ...)
might prevent a postgres user to read this field.
2017-08-07 12:59:34 +02:00
Raphael Collet faacacb45f [IMP] registry: check existence of tables with a single SQL query 2017-07-10 12:38:25 +02:00
Denis Ledoux ad67f7aff8 [FIX] sql: Do no consider a table exists if this is a schema table/view
Before this revision, this is not possible to create a model
with a `_table` set to `domains`, for instance.
2017-07-05 13:47:29 +02:00
Raphael Collet b59318ec12 [REF] registry: always perform registry/cache signaling at the end of request
Problem: the update of custom models/fields is not fully transactional, and may
potentially lead to an inconsistent database.  An other problem is creating two
custom fields by writing on a model: if the second one fails, the first one has
been committed without notice.  Retrying the request will give an unexpected
error (duplicate field name).

Solution: never commit in the middle of a request.  If the changes have an
impact on the registry, then mark it as invalid (with a new flag), and signal
registry invalidation after everything has been committed.  If the request
fails, reset the registry.  Both registry and cache invalidation are handled
the same way.
2017-05-03 15:41:05 +02:00
Christophe Simonis 9a769a8b47 [FIX] tools.sql: correct queries used to rename field 2017-04-28 18:59:14 +02:00
Raphael Collet 4deac93788 [FIX] tools/sql: column name in fix_foreign_key 2017-03-14 14:52:44 +01:00
Raphael Collet e78269664d [IMP] tools: use information_schema instead of pg-specific tables 2017-02-22 15:24:08 +01:00
Raphael Collet f8e573db23 [REF] models: refactoring of foreign keys 2017-02-22 15:24:08 +01:00
Raphael Collet d024c76021 [REF] tools: add functions for SQL schema manipulation
This helps factoring out a certain number of similar queries, and removing a
few methods from `BaseModel`.
2017-02-22 15:24:07 +01:00
Raphael Collet fa082019a0 [IMP] models: change API of _group_by_full
The dictionary `_group_by_full` is replaced by a field parameter `group_expand`
that is assigned to the method name.  The API of the method has been simplified
as well:

    @api.multi
    def _read_group_stage_ids(self, domain, read_group_order=None, access_rights_uid=None):
        # the stages are given by self.ids (wrong model);
        # read_group_order is the order given to read_group() on self;
        # return stages.name_get(), {stage.id: stage.fold)

    _group_by_full = {'stage_id': _read_group_stage_ids}

is now written:

    stage_id = fields.Many2one(..., group_expand='_read_group_stage_ids')

    @api.model
    def _read_group_stage_ids(self, stages, domain, order):
        # stages is a recordset;
        # order is the order to use on stages' model;
        # return a recordset which is a superset of stages
2016-09-16 17:35:24 +02:00
Raphael Collet 9e64f9f951 [REF] openerp: move openerp to odoo 2016-09-02 17:28:12 +02:00