Some options have been renammed a long time ago but there was no
mechanism to warn the user should those option be still present in its
configuration file.
Odoo versions up to Odoo 14 (excluded) used `osv_memory_time_limit` and
`geoip_database` in their configuration, those two options have been
renamed to `transient_age_limit` and `geoip_city_db` in 14.0 ab4000f and
saas-16.1 c59750d824 but no deprecation warning / automatic failover
were provided.
closesodoo/odoo#163193
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
The various keys and certificates used by the TestIrMailServerSMTPD test
suite were generated on-the-fly via a shell script present next to the
test. It is just easier to save the keys and certs in git rather than
re-generating them everytime.
Changed the private keys from RSA to ed25519 for the smaller files size,
changed the validity date to a thousand year.
task-3703209
opw-3640374
Part-of: odoo/odoo#151483
A previous commit broke the smtp authentication using a TLS certificate
and we only figured it out after that a client created a support ticket
several weeks later. It turns out that there are no tests that validate
the various ways outgoing mail servers can be configured.
In this work, we add a test suite where a local smtp server is started
and controlled during the test execution. This makes it possible to test
all the possible outgoing mail server configurations, including TLS.
This work revealed several problems that have been sorted in other PRs,
a problem that is left to solve is to verify those certificates as shown
by the `test_man_in_the_middle` test. This will be sorted in a future
work.
We chose [aiosmtpd] which is a pure-python lightweight SMTP server that
aims at providing a programming API that is well-suited to be used
inside unittests.
task-3703209
opw-3640374
[aiosmtpd]: https://aiosmtpd.readthedocs.io
Part-of: odoo/odoo#151483
Start a SMTPS server with client certificate authentication. In Odoo
configure an outgoing mail server with encryption="ssl/tls" and
authentication="certicifate". Load a valid client certificate and key to
use with the SMTPS server then test the connection.
The connection fails because the client certificate wasn't sent during
the TLS handshake.
If you're having trouble running a SMTPS server, I made a script here:
https://gist.github.com/Julien00859/5090d1cff6c02197e5854aabb67bf5ac
It uses aiosmtpd, a light pure python smtp server, install it with pip.
You'll need to copy your snakeoil ssl key + cert inside your /tmp
directory and to expose them to your current user:
# public cert
cp /etc/ssl/certs/ssl-cert-snakeoil.pem /tmp
# private key
sudo cp /etc/ssl/private/ssl-cert-snakeoil.key /tmp
sudo chmod 400 /tmp/ssl-cert-snakeoil.key
sudo chown $USER /tmp/ssl-cert-snakeoil.key
task-3703209
closesodoo/odoo#162297
X-original-commit: b3d7c1fc9c017a4354dc4a6f8abfbf590bc26a51
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
The conditionnal `isinstance(exc, NotFound)` is shadowed by the
conditionnal `isinstance(exc, HTTPException)` two lines above. Nobody
ever complained that the warning for NotFound error was gone. Since
werkzeug 1.0.0, the status code in the response log is colored, 404 is
colored yellow which should catch the eye. The explicit warning line
isn't really necessary.
closesodoo/odoo#159895
X-original-commit: 851b91f19b87446662421cb8d801a9472725bc72
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Upon calling `invalidate_recordset` the current recordset present is
flushed. That recordset can be active=True which override active=False
set `_process_job()`. Flushing the recordset *before* calling
`_process_job` makes sure that there is no dangling data to be flushed.
closesodoo/odoo#159262
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Create a cron with an `interval_number` of 0 and change its nextcall so
that it is called soon. When the cron gets executed, the cron worker
enters an infinite loop during the computation of the next nextcall.
The cron now gets disabled with an error message. On the form view,
users now get a warning when `interval_number` is invalid.
closesodoo/odoo#158519
X-original-commit: aaf498ff2acf39f73fe408eab74e82a8d58f6f1b
Signed-off-by: Rémy Voet (ryv) <ryv@odoo.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
It is possible for a cron to be executed twice at a same moment if the
cron is currently being executed by a cron worker and that a user click
on the "run manually" button from its form view.
closesodoo/odoo#157203
X-original-commit: a45f171eabdb571465d6245bf2a6bccecab53fa0
Signed-off-by: Olivier Dony (odo) <odo@odoo.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
The INFO "Starting job x" and "Job x done" logs are only logged for the
automatic executing of the cron by the cron worker. When running the
cron manually via its form view, no INFO was logged.
The technical support is reporting problems where a cron server action
is running twice at a same moment leading to problems such as
mass-mailing sending emails twice. There is a mutual exclusion mechanism
for cron workers but no exclusion mechanism seems in place for http
worker vs cron worker. Logging the "run manually" actions will help us
figuring out the problems.
X-original-commit: fcc2eabc671557e610688eac93bae333b0a2c119
Part-of: odoo/odoo#157203
Create an attachment with an URL to a static file that does not exists,
e.g. '/web/static/idontexist.png'. Inside your browser try to access
that file, open <localhost:8069/web/static/idontexist.png>. The browser
fails with a "Too Many Redirections" error.
When a path is not found, nor in the static files, nor in the
controllers, `_serve_fallback` kicks in and attempt to find a resource
outside of the router that matches the URL. In case it finds an
attachment with a matching URL, it'll deliver it.
In this specific case, it finds our attachment and return a redirection
to it's URL, which is the same URL as the request hence it loops back.
Don't deliver URL attachments via `_serve_fallback`, only deliver stored
files.
closesodoo/odoo#156323
X-original-commit: d2bea592db6a66d531b83437d42b273606f629f1
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Install mass_mailing with demo data, send a mailing using the "Thank
you for joining us" template. Inside your mail client, click on the
LOGIN button, this open your web browser on a link-tracker URL but the
page fails to load because "The page isn't redirecting properly".
Inside the template of that "Thank you for joining us" mail, the logging
button is basically defined as follow: `<a href="#">LOGIN</a>`, a URL
with a fragment that is empty, a redirection to the current page.
Upon rendering that template and send it to the recipients, all links
are wrapped inside a link-tracker for well tracking purpose, this
created a link `/r/xyz` targeting `#`. Upon accessing that `/r/xyz` URL
the client would be redirected to `#` which in that context is actually
`/r/xyz#`: the link-tracker itself. The browser detects that there is a
redirecting loop and show an error instead.
The problem is solved by saving an absolute link with the link-tracker
instead of a relative one.
Task-3603607
closesodoo/odoo#150678
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
It was technically possible to register a link that would loop on itself
by abusing the query and fragment parts of the URL. It was possible to
register a link targetting `#` and get the following HTTP exchange which
enter an infinite loop.
GET /r/bAd HTTP/1.0
HTTP/1.0 301 Redirect
Location: #
GET /r/bAd# HTTP/1.0
Prevent such links from being registered and highlight flaws inside the
`validate_url` method with unittests.
Part-of: odoo/odoo#150678
One of our customers is receiving emails with headers and attachments
encoded using the "iso-8859-8-i" charset instead of "iso-8859-8" which
is natively supported by Python. Both encoding are using the same
character set[^1] and only differ in the way the text is rendered on
screen[^2][^3] which is not revelant for Python.
Add an alias for iso-8859-8-i so that the emails that this customer
receive stop failing in Odoo. Note that there is a PR opened on
CPython for exactly that, see [bpo-18624].
opw-3653210
[bpo-18624]: https://bugs.python.org/issue18624
[^1]: https://encoding.spec.whatwg.org/#legacy-single-byte-encodings
[^2]: <data:text/html;charset=iso-8859-8,hello%20%E0%E1%E2%E3>
[^3]: <data:text/html;charset=iso-8859-8-i,hello%20%E0%E1%E2%E3>
closesodoo/odoo#150467
X-original-commit: 5a025467a605ca4fa963b79bae254c823518d54c
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Start a live session, on the manager side stay on the welcome page (the
one that counts how many attendees joined). Using several other private
browsing tabs join the live session. On the manager side, the number of
attendees never changes.
During a previous refactoring of legacy rpc => orm, an error slipped,
instead the records and fields as separated arguments, the two were
passed together as a list in a single argument.
We used the opportunity to increase the verbosity in case of errors and
to enrich our test cases.
Fine tunning of 7422eb6 ([IMP] *: remove legacy rpc)
Task-2834638
closesodoo/odoo#149427
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
This reverts commit 6c59eea421.
smtplib is expecting the Context of ssl found in the stdlib and the
Context object of the pyOpenSSL lib isn't a drop in replacement. The
various `TLS_METHOD` constants are not the same and the Context object
of pyOpenSSL lack a `wrap_socket`-like method.
opw-3640374
closesodoo/odoo#149566
X-original-commit: 028277129897c7d49942a148a0ac295386cd2d89
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
In #126914 a limit on the request size has been enforced, that limit is
by default 128MiB and can be configured via an ir.config_parameter. When
restoring a backup larger than 128MiB via the database manager, the
default limit was used and the request was cancelled with a Request
Entity Too Large (code 413) HTTP error.
It is now possible to define a default max content length per route,
that per-route limit takes over the `web.max_file_upload_size` ICP.
Moved the code from `get_http_params` to `pre_dispatch` to better align
with the httpocalypse new http stack.
Fixes: #144144
opw-3643475
closesodoo/odoo#147506
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Enable TOTP on your account and create yourself an API key. Connect in
xmlrpc using that API key. Traceback `request` is not bound.
Fixes: odoo/documentation#6919
See also: #147475closesodoo/odoo#146270
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Create a new live-session survey with a multi-choice question of 4
choices. In one of the choices write a word of 20+ characters. Start the
live-session, on the manager side the choices overlap on the screen.
In case a word is longer that the available place, it can overlap on the
labels of the other columns, making the text very hard to read. This
work wraps words that are too long for a single line over multiple ones.
Task-3530706
closesodoo/odoo#146554
X-original-commit: f085957855f86fb669c533be09fd13a14a6c874d
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
The Content-Security-Policy[^1] http header was only set on the response
generated by controllers but it was missing from the `/<module>/static/`
route.
It is not strictly necessary to set that header on the responses comming
from that routes as it is not possible to add new static files or edit
existing ones via the interface (not even as admin). Only the developers
and system administrator can access those files.
It is also worth mentionning that using the Odoo internal web server to
deliver static files is suboptimal. Outside of a dev environment, those
files will typically be delivered via a web server[^2] and sysadmins
should configure their web server to set the CSP header on static images.
[^1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
[^2]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachmentsclosesodoo/odoo#146591
X-original-commit: 55e09d504df9bd134afb6e0b38f03457b5c71e8e
Related: odoo/documentation#6953
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
For multi language website, when request http:/localhost/en/something,
Odoo reroutes from the requested path /en/something to the new path
/something with lang=en_US in context.
If the new path is a unicode string like http:/localhost/vi/xin-chào,
http:/localhost/ru/привет, a error should occur at
werkzeug._compat.wsgi_decoding_dance() because the path was not latin1
string.
The utf-8 encoding followed by a latin-1 decoding is required by the
WSGI specification[^1]. latin-1 is used as an encoding passthrought:
that encoding has a representation for all the 256 bytes, i.e. it is
impossible that decoding a text will raise a ValueError. The WSGI spec
uses this trick to save values until the actual charset (present in
the Content-Type header) in known.
[^1]: https://peps.python.org/pep-3333/#a-note-on-string-typesclosesodoo/odoo#143898
X-original-commit: 9b69b87c08b1d62b3581fe651bee692a4f217dff
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Because the class was imported in this file, unittest was discovering
it again and was running the TestHttpStatic cases twice: once because
of its inclusion in the test_static.py file, once more because of its
inclusion in the test_web_server.py file.
Changing the import solved the problem, since it is a python module
object that is now exposed and not test case classes, unittest doesn't
discover the classes.
closesodoo/odoo#143840
X-original-commit: 8067d3bf3d9f73b95595a16100096aafff5c2075
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Open a live session for the burger quiz, join as an attendee. Everytime
the host reveal the next question, the attendees get a traceback.
```js
TypeError: ... is not Iterable
```
A recent commit 7109f48 changed the controllers of survey, they used to
`return self._prepare_question_html(...)`, most of them now
`return {...}, self._prepare_question_html(...)`, with the notable
exception of the `survey_next_question` controller which continues on
returning only the `_prepare_question_html(...)`.
JS-side, the `self._prepare_question_html(...)` payload is retrieved via
`const [,result] = await nextScreenPromise;`, i.e. it excepts an array
of 2 elements and retrieve the second one.
As the other survey controllers have been modified inside 7109f48, the
`survey_next_question` should be updated too but was forgotten. Returns
an empty dict for the correct answers and the next page html.
Task-3374998
closesodoo/odoo#139204
Reference-to: 7109f48 ([IMP] survey: add scoring after each page)
Signed-off-by: Warnon Aurélien (awa) <awa@odoo.com>
Start the a live session, I used Burger Quizz, answer a question. On the
manager side, the current vote count for each choice should be visible
on top of each bar in the chart. That vote count is missing.
During the migration of chartjs from v2 to v4, the datalabel plugin was
updated from v0.7 to v2.2 in the same time. Using the new version of the
datalabel plugin, the plugin is not automatically loaded anymore, one
must explicitely loads it.
We also removed the datalabel plugin from the bundle to load it
explicitelly instead. The JS framework finds it cleaner this way.
Task-3537480
Reference-to: 7e3c1ecdb8 ([REF] *: Update Chart.js to V4.3)
Part-of: odoo/odoo#139204
Co-authored-by: Pierre Pulinckx <pipu@odoo.com>
Go to the form view of mailing.mailing, write "hello " as the subject,
insert an emoji, you end up with "hello:)" instead of "hello :)". The
problem is that when the input loose the focus, it is automatically
trim.
Reconfigure the `char_emojis` widget so that is doesn't trim.
task-id-3493168
closesodoo/odoo#136530
X-original-commit: 2adda1eb2760ccf970761f424105e245afc3de3f
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Users sometime want to rename existing controllers URL, e.g. /shop as
french /magasin. The feature is possible within Odoo thanks to 308 type
redirections that can be configured via a hidden ?debug=1-only website
menu.
Upon generating the routing-map (the structure that links URLs to
controllers), when a URL is found inside of the `website.rewrite` model,
two links (called Rules in werkzeug jargon) are registered: the new,
translated, URL is linked to the controller and the original URL is
linked to a redirection to the new URL.
For the context of this PR, it is important to note that the redirection
only applies to the very URL saved inside the `website.rewrite` model:
if a controller has multiple routes, e.g. `/shop` and `/shop/shop`, only
`/shop` is redirected to `/magasin`, `/shop/shop` is left as-is.
Without 308 redirection:
/shop -> def shop()
/shop/shop -> def shop()
With 308 redirection:
website.rewrite(from_url='/shop', to_url='/magasin')
/shop -> /magasin
/shop/shop -> def shop()
/magasin -> def shop()
The redirection is set on the routing dictionnary of the endpoint, this
is the dictionnary that collect the informations set via the `@route`
decorator (auth=, method=, type=, ...).
Prior to [HTTPocalypse], that dictionnary was duplicated so that the
redirection was applied on the single route endpoint that matched
the `website.rewrite` record. With [HTTPocalypse] that duplication has
been wrongly removed: all original routes redirected to the new
translated one.
Bug introduced in [HTTPocalypse]:
website.rewrite(from_url='/shop', to_url='/magasin')
/shop -> /magasin
/shop/shop -> /magasin <-- wrong
/magasin -> def shop()
This PR fixes the problem, it makes sure that the redirection is saved
*only* on the route that matched the website.rewrite record, not the
other routes.
[HTTPocalypse]: https://github.com/odoo/odoo/pull/78857closesodoo/odoo#133960closesodoo/odoo#134206
X-original-commit: 144a22c22c95004171860cdaecd1f7d7975dc468
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
The `test_override_signatures` linter is about validating the parameters
of overriding functions. It verifies that when a method overrides another
from a parent model, the new method has a signature that is compatible
with the method it overrides: same arguments, same default values, same
annotations.
Because it also verified annotations, when a parent method was
annotated, all the child methods had to be annotated too. We actually
only care about the annotations when the two methods are annotated, we
don't want to enforce annotations on non-annotated methods.
Note 1: the shortcut to skip `(self, *args, **kwargs)` is broken, the
code has been removed.
Note 2: the code about `parent_class` is dead code that survived a
previous refactor.
Note 3: xdo likes it when I respect flake8-errmsg (EM101-103)
Part-of: odoo/odoo#133049
The limit was only enforced by the front-end, meaning that anybody could
forge a request with a huge file and get it processed by Odoo. According
to the documentation of werkzeug[^1], such limit should be enforced by
the server server instead of the wsgi application. It is the case for
Odoo Online but on-premise customers might not configure their servers.
The `web.max_file_upload_size` system paramter is now enforced upon
parsing the content of the request. It defaults at 128 MiB which is
enough for most documents and images. We do not want to host large
files (e.g. videos) in the Odoo filestore.
[^1]: https://werkzeug.palletsprojects.com/en/2.0.x/request_data/Fixes: #124646
Part-of: odoo/odoo#126914
The `-i`/`--init` and `-u`/`--update` cli options behavior is only
defined when using a single database with `-d`/`--database`/`db_name`.
Using those two cli options along with multiple databases is undefined
and can have disastrous consequences[^1].
The server now crashes in this situation.
Fixes: #107188Fixes: #128273
[^1]: https://github.com/odoo/odoo/issues/107188#issuecomment-1627996425closesodoo/odoo#128306
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Web lacked a default route for /robots.txt meaning that if you didn't
installed website (which comes with a full fledged robots.txt) you would
get a 404 page not found.
closesodoo/odoo#127402
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Steps to reproduce, on SaaS only, not reproductible in standard:
1. Start a new SaaS Trial on saas-16.3
2. Install helpdesk, setup an incoming mail server
3. Send an email, one that should create a new helpdesk ticket
4. Traceback, request has not 'is_frontend` attribute.
This commit doesn't solve the root issue, it only makes it possible to
use helpdesk again. Using getattr/hasattr to access request.is_frontend
SHOULD NOT be necessary since [HTTPocalypse] BUT there are some rogue
controllers that bypass the normal flow of execution and fail to meet
the expectations of the new http stack.
This commit only makes the code robust to a missing attribute in this
very case as this is a recurring problem (unusable helpdesk). The root
problem is unlikely to be located nor in http_routing, nor in helpdesk.
THIS SOLUTIONS OF USING `getattr`/`hasattr` TO ACCESS `is_frontend` MUST
NOT BE REPLICATED ELSEWHERE WITHOUT PRIOR CONSULTATION WITH PEOPLE IN
CHARGE.
[HTTPocalypse]: https://github.com/odoo/odoo#78857
See-also: https://github.com/odoo/odoo/pull/99667
See-also: https://github.com/odoo/internal/pull/1902
opw-3359740
opw-3365843
closesodoo/odoo#126386
X-original-commit: c808719619167a582dd28420ff2979b57102d2ab
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
The Gevent worker has specifc needs in term of maximum concurrent
connections to the database and those needs are not compatible with the
default limit that is primerly set for http workers.
This PR makes it possible to supply a configuration dedicated to the
gevent worker.
task-id-3193565
task-id-2146565
closesodoo/odoo#125190
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Send a request to any json-rpc route with a HTTP header
`Content-Type: application/json-rpc` but send non-json or
non-jsonrpc data in the body. The application crashes with
a 500 Internal Server Error instead of a 400 Bad Request one
closes odoo/odoo#124571
Closes: #122048
X-original-commit: a1b83b07996f3fe4744474ff4023a3a2ce17ac7f
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Run `odoo-bin help`, you'll notice that some descriptions are missing,
this commit fixes that.
Run `odoo-bin shell --help`, you'll notice that the `usage:` line says
`odoo-bin [options]` instead of `odoo-bin shell [options]`. Other
commands that depend on the server cli are broken too. Fix those too.
closesodoo/odoo#121085
X-original-commit: 9c6bac741ff437564e7dbe4d0aa8dbd307b71f8d
Signed-off-by: Vincent Schippefilt (vsc) <vsc@odoo.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Install and then uninstall the utm module via the web client, you get a
traceback because the ir.http override of the utm module is still
present in the registry altought the module is not installed anymore.
The problem affects all modules that override the _post_dispatch method
of ir.http, it is not limited to UTM.
The problem is that, after the uninstallation, a new registry (without
the uninstalled modules) is created but the old registry was still used
by the HTTP stack.
closes odoo/odoo#122519
Closes: #121755
X-original-commit: 979844600a0bc5ea8b63cf4d58c7aba5133e82f3
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
This commit add a new `odoo.osv.expression.prettify_domain` function
that can be used to format a domain as a string with the correct
indentation.
Example:
['&', '|', ('name', 'like', 'Jack'), ('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
Becomes:
['&',
'|',
('name', 'like', 'Jack'),
('name', 'like', "O'Neill"),
('function', '=', 'Colonel')]
closesodoo/odoo#118067
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Rationale
---------
Given the following domain:
['|',
('company_id.name', 'like', 'BE'),
('company_id.city', 'like', 'Brussels')]
The ORM would generate the following SQL query:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."name" like 'BE'
) OR "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company"
WHERE "res_company"."email" like 'help@odoo.com'
);
Which is sub-optiomal as the WHERE clause of the two subqueries could be
regrouped inside of a single query like so:
SELECT "res_partner"."id"
FROM "res_partner"
WHERE "res_partner"."company_id" IN (
SELECT "res_company"."id"
FROM "res_company" WHERE (
"res_company"."name" like 'BE'
OR "res_company"."email" like 'help@odoo.com'
)
);
Postgres-wise, it is faster to execute the latter query than the former
one. This commit is about optimizing the ORM so that it generates
queries where the WHERE clause of compatible subqueries are grouped
together.
Technical solution
------------------
The solution explored by this work is to replace the regular relational
field accesses (over a dotted path) by a new `any` operator that look as
follow:
(relational_field, 'any', domain)
For instance, the above `('company_id.name', 'like', 'BE')` leaf becomes
`('company_id', 'any', [('name', 'like', 'BE')]` using `any`.
Having a domain as right-hand-side allow for the combination of the
domains of same compatible relations:
[('company_id', 'any', ['|',
('name', 'like', 'BE'),
('city', 'like', 'Brussels')])
Having combined domains allows for generating better SQL queries with
minimal changes to the expression parsing algorithm, which can only
translate a single leaf at a time to SQL. With the new `any` operator,
it is still a single leaf but the right-hand-side contains the merged
domain, thus the combined SQL WHERE clause is immediately generated.
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
This commit hads a few tests that report the existing behavior (before
merging subqueries WHERE clauses).
task-id-3234671
Part-of: odoo/odoo#118067
Co-authored-by: Raphaël Collet <rco@odoo.com>
All major systems (debian stable, ubuntu lts, windows) support py3.8
and all dependencies used by Odoo come with wheels for that version.
Most developers at Odoo SA uses 3.8 already and runbot is using ubuntu
jammy (which comes with py3.10) to test the current 16.0/master.
closesodoo/odoo#119492
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Disable performance tests from both tef and tcf.
Too many PR are blocked due to broken assertQueryCount, either there are
too many queries, either there are too few.
It is too much work to run all tests three times only to update a comment
with the final count (module alone + community + enterprise).
It is too hard to keep track of the hundreds queries to determine those
that moved, those that are missing and those that are new between two
branches. We have to apply tons of string-replace and sorts just to help
some diff tools (e.g. meld) into showing what changed.
Basically, except a few people, nobody care to do the investigation work
and just increase the query count (without changing the comments).
We tried for one year, now it is time to let those test go.
closesodoo/odoo#118614
X-original-commit: 353074fa1f362560737bb2905270ef4ae35e177e
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
A visitor could visit a web page having a lang in its context that is
not installed in the databased he is connected to. The problem is that
visitors are not logged-in thus it is not possible to determine their
lang via their `res.users` preferences. The lang used instead is the
lang set in the `Accept-Language` header of the incoming request, that
header is set by various browsers in accordance to the user system
preferences or browser settings.
The browser lang (`Request.best`) is only parsed according to the
`babel` database, it is a lang syntactically speaking but not necessary
a lang that is installed in the database.
At the moment the browser lang is set in the context (inside of
`Request._get_dbname_and_session`), it is not possible to verify it is
installed in the database as no connection to any database as been
established yet. Instead the lang is validated inside of
`ir.http._pre_dispatch` which is the method responsible to prepare/fix
various stuff on the request/session/context.
In regard to 93b684d3c7, we prefer to fallback on English, hence the
modification in `get_lang`.
closesodoo/odoo#116683
Reference-to: 93b684d3c7 ([FIX] base: lang should fallback on english instead of arab)
X-original-commit: 743e97667e44cdaab6db334d807cf3d409cea621
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
As the public user, browse the website where you usually should see some
profile pictures (e.g. inside the forum). All the images are wrongly
replaced by the grey avatar placeholder.
When using `ir.binary._find_record` it was checking the access rights
and raising `AccessError` early even if the record was
`website_published`.
closesodoo/odoo#113526
X-original-commit: 0611fb437b699588317919e72ccfa1c41f1245bc
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
In an edge case the content of the field has already been
prefetched as sudo during the read of another field as sudo,
therefore it doesn't try to read the field as the normal user
closesodoo/odoo#88134
Signed-off-by: Julien Castiaux <juc@odoo.com>
Install a database with many langs, arab, french, english, ... Keep
english as the default lang. Start a shell and validate a sale-order
using the superuser. On the web client, the sale order has been
validated in arab instead of in english.
In 16.0 the `context_get` method was changed to ensure there was always
a lang set in the returned context. It used the following fallback
order: context > request. The solution was partial because in case there
was no request to extract a lang from, no lang was set on the context.
In a recent 16.0 fix (f2523c4a), the mechanism was changed to fix the
previous problem. The fallback order became: context > request > first
installed lang. This solution is sub-optimal because the first installed
lang isn't always the best pick. e.g. when you have a mostly english
company but that arab is installed for some website pages, arab is
selected instead of english (the langs are alphabetically sorted)
In this work, the fallback order is changed once again:
1. The lang set on the user's profile if activated
2. The best lang extracted from the user's browser if activated
3. (new) The lang of the user's current company if activated
4. (new) English if activated
5. The first lang (ordered by ISO code) if any
6. English
The 3rd should cover most of ill-cases. For the 4th step, we assume that
english is prioritaty to other installed langs when no lang standout.
closesodoo/odoo#113186
X-original-commit: 03134bf7cb1e3d63f3be435fbc734e6198ca029b
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Fine tuning of 8eaac97, the `errors` attribute was added in py38[^1]
[^1]: python/cpython@ca7b504a4dclosesodoo/odoo#112588
X-original-commit: c6c19ff6a3093fe64035d9f970ab35522f8080f5
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Remove custom open introduced by bpo-26789 as we do not need it.
closesodoo/odoo#112453
X-original-commit: 8eaac9744b93e7132827edb2a59c28bb43732ec1
Related: odoo/enterprise#36978
Signed-off-by: Julien Castiaux (juc) <juc@odoo.com>
Co-authored-by: Martin Trigaux <mat@odoo.com>
The path matching logic got reimplemented in werkzeug 2.2[^1] and the
new router is no more compatible with regexp groups[^2]. Our custom
converter for slugged-records in urls (`'/partner/agrolait-5'` => `5`)
has been adapted to match the route using non-capturing groups. It still
extracts the slug/id pair using the groups-capturing regexp.
[^1]: https://github.com/pallets/werkzeug/pull/2433
[^2]: https://github.com/pallets/werkzeug/pull/2519
Part-of: odoo/odoo#112298
Based on RFC3462, a Content-Type text/rfc822-headers
exists and provide a mechanism to label and return
only the RFC 822 headers of a failed message (bounce)
These are only the headers and not the full message.
The Content-Type-Encoding should be either 7-bit(US-ASCII)
or Quoted-Printable (QP) as in the section 2 of the RFC.
Spawn the error:
After getting reported by a customer, I had to reproduce
by spamming wrong outlook addresses and add logging
in a sh database on the message_process of mail_thread.py
and logged the `message` variable.
After few retry, I got one of the part that was defined
as followed:
Content-Description: Undelivered Message Headers
Content-Type: text/rfc822-headers
Content-Transfer-Encoding: quoted-printable
The `get_payload()`function used was only assuming
that there is a full email on that part and that
it could only be encoded as an email, which was not
the case in this situation (quoted-printable:
76 characters per line, character `=` used
as the end of line character).
opw-3064589
task-3131561
closesodoo/odoo#110901
X-original-commit: f86f4b671696178f8fa42b81d8753d2591578431
Signed-off-by: Julien Castiaux <juc@odoo.com>
Co-authored-by: Baptiste Vergote <bve@odoo.com>
Well it is not a typo but a mini code improvement. The suggested diff
clarifies the intent of `_handle_control_frame` with regards to
`_get_messages`. Because it was `return self._handle...` we had to read
the method to determine if it was returning something in order to know
whether the message would be propagated to the rest of the websocket
routing by `_get_message`. Now it is clear that the control frames are
handled right away and that they are not propagated to the business.
closesodoo/odoo#110834
Signed-off-by: Julien Castiaux <juc@odoo.com>
Some cookies were left around even when the user logged out
The next user should log into the default company instead of the company
of the last user.
task-3077421
closesodoo/odoo#108998
Related: odoo/enterprise#35685
Signed-off-by: Julien Castiaux <juc@odoo.com>
Send an rich HTML email from Odoo, it lands in spam whereas it would
land in inbox in 13.0.
The problem is due to a missing "MIME-Version: 1.0" header on the email.
This header is correctly set on both the html and text alternatives of
the messages but it should be set on the enveloppe too.
The problem is present in the newer EmailMessage mail API of python that
is used since 14.0. Using the newer API, it doesn't set the header on
the enveloppe itself.
This reverts commit 8663f1e20727c315924bb20fc1de1accf3014c61.
opw-3098621
closesodoo/odoo#109212
X-original-commit: c480d2bf1de7b70892130e74bb7a8d4003a8a783
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
Start odoo-bin shell with `--geoip_country_db='/i/dont/exist'` and type
the following code:
from odoo.http import GeoIP
GeoIP('8.8.8.8').country_code
TypeError: The country method cannot be used with the GeoLite2-City
database
There are two "things" here: the raw country and city entries in the
databases and the rich python country and city objects returned when an
entry was found and parsed.
Then there are two "readers", a reader capable of reading and parsing
entries from the city database and **another** reader for the country.
The two readers are **only** compatible with their specific database:
the city reader cannot read or parse information from the country
database and vis-versa. That's the bug reported by the TypeError that
this commit fixes.
But, the two python instances created from parsing the entries of
respecting databases with respecting parsers **do** share a same API:
the city record class inherits from the country record class.
# Valid
city_db = geoip2.database.Reader(config['geoip_city_db'])
city_record = city_db.city('8.8.8.8')
city_record.country.name # works, a city record also
# holds country attributes
# Invalid
city_db = geoip2.database.Reader(config['geoip_city_db'])
country_record = city_db.country('8.8.8.8') # TypeError
closesodoo/odoo#109095
Signed-off-by: Julien Castiaux <juc@odoo.com>
The geoip resolver was moved from http_routing to code/http.py in #86015
and is always available since then. The `_geoip_resolver` global
variable is a leftover we forgot to remove.
closesodoo/odoo#91337
Related: odoo/documentation#2151
Related: odoo/enterprise#27399
Signed-off-by: Julien Castiaux <juc@odoo.com>
request.geoip is no more a dictionnary cached in the session. It is now
a full blown object with lazy and smart geolocalisation capabilities.
Among other things, the previous dictionnary API is now deprecated. The
changes are:
* `request.geoip['country_name']` -> `request.geoip.country_name`
* `request.geoip['country_code']` -> `request.geoip.country_code`
* `request.geoip['city']` -> `request.geoip.city.name`
* `request.geoip['latitude']` -> `request.geoip.location.latitude`
* `request.geoip['longitude']` -> `request.geoip.location.longitude`
* `request.geoip['region']` -> `(request.geoip.subdivisions[0].iso_code if request.geoip.subdivisions else None)`
* `request.geoip['time_zone']` -> `request.geoip.location.time_zone`
It is safe to access all the attributes. Doing `request.geoip.city.name`
when the geolocalization failed (missing db, invalid address, ...)
evaluates to None. It does not raise an AttributeError.
Task: 2848206
Part-of: odoo/odoo#91337
Maxmind offers multiple ip-geolocalization databases, historically we
have been using the City database which contains records on a
city-basis. Many years later it turns out we are primary using geoip to
know the country of the user. Geolocalization in the City database is
considered slow by our standard and we have been clever in order not to
geolocate each request by saving the info in the session.
On the other hand, the Country database that is offered by Maxmind is
much more lightweight and geoip using that country is considered a fast
operation by our standard.
In this work we make Odoo compatible with both the City and the Country
databases. Using multiple database at the same time, we can be smart and
only query each of the two on-demand. If a user ask for its country,
we'll use the fast Country db. If a user ask for its city/timezone we'll
use the slower City db.
By default it loads both database from the `/usr/share/GeoIP/` folder,
respectively the files `GeoLite2-City.mmdb` and `GeoLite2-Country.mmdb`,
you can provide alternative paths using the `--geoip-city-db` and
`--geoip-country-db` CLI options.
In the same mindset as #86015, geoip is still lazy. It is done on-demand
and the result is cached on the current request. The different with the
related PR is that as we know consider geoip to be fast, we no longer
cache the result in the session.
Task: 2848206
Part-of: odoo/odoo#91337
*: base_setup, hr_timesheet, mail, partner_autocomplete, web_tour
Start odoo without -d and with a --dbfilter that allows multiple
databases. Via JSON-RPC access the /web/session/authenticate route
providing a non-filtered database and valid credentials. Traceback,
`request.env` is None.
Since httpocalypse the initialization of the ORM (cursor, registry,
environment) is greedy. It means that the connection to the database is
established very early during the request routing or skip altogether in
case no dbname was known at that time. This contrast with prepocalypse
where the various ORM thingies were lazily setup the first time they
were accessed.
This changement has an important implication regarding authentication.
In prepocalypse, thanks to the lazy approache, a cursor/registry/env
would be setup on the database you just login upon using the
`request.env` for the first time. This was very nice in this regard but
had other problems.
Since httpocalypse such operation is no more possible. Devs must
initialize and use their own cursor/registry/env in case they
authenticate on another database than the one `request.cr` is (maybe)
connected to.
The `/web/session/authenticate` controller is an example of such case.
It crates its own cr/registry/environment after authentication. The
problem the controller uses `ir.http.session_info` and that not all
overrides were updated to use `self.env` (=the env created in the web
controller) instead of `request.env` (=the missing env of the request).
closesodoo/odoo#108063
X-original-commit: 7b9bd9d37731fae724dc5d91da656dab70aa9ad4
Related: odoo/enterprise#35012
Signed-off-by: Julien Castiaux <juc@odoo.com>
*: base, account, crm, hr, hr_attendance, test_access_rights
The various public methods of the ORM can be override in other models,
those overrides sometime don't implement the exact same signature as the
original method in the ORM. In this work we sanitize all the overrides
to ensure a better compatibility. The background objective is to make it
possible to call any public method using kwarg: `search(domain=[...])`.
* `search`, the first parameter was renamed from `args` to `domain` in
0e9adf7 but the overrides were not updated.
* `invalidate_models` and `invalidate_recordset`, a new `flush=True`
parameter was introduced in 9c3b9a4 but the overrides were not
updated.
* `update`, there is a clash between the `update` method responsible for
writing on a record and `update` in bus responsible to update the user
presence. The bus method has been renamed so it doesn't clash with the
ORM.
This sanitization comes with a new linter that verifies that all
overrides of BaseModel public methods share a compatible signature. The
linter has been disabled for `create`, `write` and `default_get` as too
many overrides don't respect the signature of BaseModel.
closesodoo/odoo#106999
Related: odoo/enterprise#34991
Signed-off-by: Julien Castiaux <juc@odoo.com>
Start a server without a -d and with a --dbfilter that allows for
multiple database. Make sure one of the database has the
`base_import_module` addon installed. Create an empty module using
scaffold and deploy it to the server, make sure to provide the `--db`
argument to the deploy command.
It zips the file and attempt to upload it but it fails for a 404 page
not found error.
The problem is that the controllers of base_import_module are only
accessible when the client is connected to a database. It must first
connect to a database (to have a db in his session) and then access the
controller.
The /web/login route is an example of a rather cheap route to get that
is both accessible without being connected to a database and that takes
a `?db=` argument to connect to one. Using that route, we can ensure
that we are connected to a database prior to uploading a module.
Closes#104589closesodoo/odoo#107906
X-original-commit: 8feb5f366255c8cb7927842acea3ec76bbe11df2
Signed-off-by: Julien Castiaux <juc@odoo.com>
The routing-map is a mapping that maps HTTP verbs and paths to python
controller methods (endpoints), e.g. it maps `GET /web/health/` to
`/web:Home.health`. The `_generate_routing_map` function is the function
responsible to generating the werkzeug routing-map in regard to the
controller inheritance mechanism. The mechanism makes it possible to
override an endpoint is various odoo modules to enrich it with new
features, e.g. `/web/login` is overriden in website to change the visual
of the page.
Implementation-wise, the informations from each `@route` decorator must
be merged with the other `@route` info for each endpoint override. When
a method is not overriden in a controller, there is not new info and
that controller should be skipped for that method.
The previous implementation attempted to skip such method using the
following idiom:
if not hasattr(controller, method_name):
continue
That idiom doesn't work as `hasattr` will perform a lookup on the full
controller's MRO instead of a lookup only on controller's own methods
and attributes. The controller's own methods and attributes are
actually found in `controller.__dict__`.
closesodoo/odoo#107064
X-original-commit: 42f52d26b6a81a68f1fe28e28971f0e7f4c97d11
Signed-off-by: Julien Castiaux <juc@odoo.com>
Define the following controller:
@route('/get-context', auth='public', type='json', website=True):
return request.env.context
Access that route via JSON-RPC providing a context:
this._rpc({
route: '/get-context',
params: {
context: {"a": 1}
}
})
Since httpocalypse, it replaces the context entirely instead of updating
it. i.e. before you would get along those lines:
{"a": 1, "lang": "en_US", "tz": "Europe/Brussels", "uid": 2}
but since you get instead (and that's wrong):
{"a": 1}
Note that after this commit, this merging-context behavior will be the
same whether or not website=True is defined (while it was only there for
website=True before).
Discovered via opw-2885948
X-original-commit: 7887cfc3f177d9e64f008aabb8bd710f54024dea
Part-of: odoo/odoo#107179
The wsgi application entrypoint moved during the [httpocalypse]. Some
clients don't use the odoo builtin wsgi server and have troubles
upgrading from 15.0 to 16.0 because the `odoo.service.wsgi_server`
module doesn't exist anymore.
[httpocalypse]: https://github.com/odoo/odoo/pull/78857closesodoo/odoo#106187
X-original-commit: 4e5d7d2e93fcd082bd1b3a9e76cedab85b46f456
Signed-off-by: Julien Castiaux <juc@odoo.com>
Create a 15.0 database with website, access the home page via your
browser. Stop the server and migrate the database to 16.0. Restart the
server with a `--dbfilter` that rejects the database you created and
refresh your browser. 500 Internal server error, attribute error:
the `request` object as no `session`.
An error could occurs after a migration to 16.0 due to the presence of
the `geoip` key in the session. `request.session.geoip` has been made a
deprecated alias to `request.geoip` between 15.0 and 16.0, see 04e9726.
Because the session was created before 16.0, the session dict does
contain a `geoip` key. Upon logging the session out, the session dict
is cleared. The default implementation of `clear()`[^1] inside of
`collections.abc.MutableMapping` can be summarized for our usecase to:
for key in self:
value = self[key]
del self[key]
There is an extra `__getitem__` call due to `value = self[key]`, in the
case of the `geoip` key, it would access the alias. It is not possible
to accessing that alias inside of the `_get_dbname_and_session` method
of request as the session has not been set on `self` (the request) yet.
Yet inside of that method, we do `session.logout()` which `clear()` the
session which (wrongly) access the alias because `geoip` exists in the
internal dict (`'geoip' in self.keys() # True`).
The solution has been to implement the `clear()` function ourself
instead of using the mixin of `MutableMapping`.
[^1]: https://github.com/python/cpython/blob/b43496c01a554cf41ae654a0379efae18609ad39/Lib/_collections_abc.py#L925-L931closesodoo/odoo#105763
X-original-commit: b66e1ffa8e348eedf2de735babbc398290a8bffb
Signed-off-by: Julien Castiaux <juc@odoo.com>
Start odoo on a specific database, e.g. 'db-example'. Drop it via JSON
or XML RPC. The database is successfully dropped but the RPC fails with
a traceback because it attempts to commit on a database that doesn't
exist anymore.
import requests
admin_passwd = ...
requests.post(
'http://127.0.0.1:8069/jsonrpc',
json={'params': {
'service': 'db',
'method': 'drop',
'args': [
admin_passwd, 'db-example'
]
}}
)
Closes odoo#104527
closesodoo/odoo#105710
X-original-commit: b6e195ccb3a6c37b0d980af159e546bdc67b1e42
Signed-off-by: Julien Castiaux <juc@odoo.com>
The new `borrow_request()` function has been introduced to properly
separate the HTTP layer from the RPC layer. We forgot to protect some
RPC endpoints, mainly inside of `odoo.addons.web.controllers.database`.
This commit moves `odoo.service.dispatch_rpc` to `odoo.http` as we only
permorm RPC from the controllers and that we don't want to import
`borrow_request` inside of `odoo.service` (circular import).
X-original-commit: d0ee8615d8820e02232989c50a6e6ffbc66a9266
Part-of: odoo/odoo#105710
Define a route that is website but not multilang, e.g.
@route('/example', website=True, multilang=False)
Login to the frontend, change the website lang to another (non-default)
lang (e.g. install french, keep english as default lang, log in the
french website) then access the '/example' controller by typing it
directly in your address bar.
You are being redirected to '/fr/example', you should not.
This commit restore the behavior pre-httpocalypse, that is the address
is kept as-is.
Note: in the comment, the 4th and 5th cases were inverted, we use this
commit as an opportunity to reorder the two.
closesodoo/odoo#105686
X-original-commit: be7a02917a66136a8d3b601d61a898b0419ff79d
Signed-off-by: Julien Castiaux <juc@odoo.com>
We introduced `odoo.http.Stream` and its `ir.binary` companion model
to refactor all the "stream data over http" bits into a single unified
API. The API takes care of creating a cache-aware, proxy-accelerated,
headers-rich HTTP response out of any path, attachment or binary-field.
closesodoo/odoo#99658
Signed-off-by: Julien Castiaux <juc@odoo.com>
The cron subsystem is the system responsible of running background task
at regular interval, it runs in multiple dedicated threads or workers
that are independent of the regular HTTP threads/workers.
There was a major overhaul of the system in v15 (4b28f1162a) to
introduce cron triggers, a way to run a task at a given moment in
addition to the regular configured interval. Although the cron system
was not extensively tested before that v15 refactor, no new test were
introduced with that refactor leaving the system mostly untested.
Since then we had to fix multiple subtle concurrency bugs such as
b940d1c25f and c06cee44fe. Due to the lack of an existing test suite,
no regression tests were added next to those fixes.
With this commit we introduce the missing cron test suite. The test
suite is separated in two different test cases:
- A standard pre-install TransactionCase to test everything that can be
tested with a single cursor. This case can be run the usual way with
`--test-tags :TestIrCron`.
- A non-standard post-install **database breaking** test case to run
concurrency tests that often require multiple SQL transactions. This
case requires the special `--test-tags database_breaking` to be
executed. You MUST backup your current database before running that
test or you'll loose data.
closesodoo/odoo#97087
Signed-off-by: Julien Castiaux <juc@odoo.com>
This fix allows generating the sitemap (via `website.search_pages`) via
RPC. The changes are necessary as since #99667 the `request` object is
no more available to RPC-executed functions.
See also: the `test_search` test case of `website.tests.test_page`.
closesodoo/odoo#104567
X-original-commit: 8f4e214417cfb1aed7e595b8d6ab71f33a347b29
Signed-off-by: Julien Castiaux <juc@odoo.com>
Install website then render a qweb template via xmlrpc, attribute error:
`request` has no `is_frontend` attribute.
Inside the qweb's `_prepare_environment` override of the http_routing
module (a dependency of website) is the following code snippet:
if (not irQweb.env.context.get('minimal_qcontext') and
request and request.is_frontend):
return irQweb._prepare_frontend_environment(values)
The conditionnal is about injecting extra informations in the qweb's
context in case we are serving a frontend request. By default, there is
no `is_frontend` attribute on the request object. That attribute is set
by the ir.http's `_match` override of the http_routing module (a website
dependency): it is set True when we don't match any endpoint or that we
match an endpoint that is `website=True`, it is set False otherwise.
The ir.http's `_match` method is called whenever we are serving a http
request whoose session is bound to a specific database. i.e. when there
is a valid database saved in the request's session. When there is not
database in the request's session (or that it is invalid) the matched
endpoint is directly called without going throught ir.http. Most
endpoints are only accessible via ir.http.
The two `/xmlrpc` and `/jsonrpc` endpoints are examples of endpoint that
do not require an established database connection to work. They perform
the request authentication and database connection themselves. It is
possible to call those two endpoints with no database saved in the
session, thus it is possible to call those two endpoints without going
throught ir.http. This is expected.
The two endpoints's duty is to execute public model methods and return
the xml/json serialized result. To do so, a registry is loaded on the
database with all the installed modules, including http_routing.
We fall in a situation where (1) there is a request, (2) we are using a
registry where http_routing is loaded, (3) there is no `is_frontend`
attribute on `request` as we didn't serve the endpoint via ir.http. This
situation is illegale.
To solve the problem, instead of working on the `if request.is_frontend`
bit of the above conditional, we decided to work on the `if request`
bit. ISO-model wise, RPC is an extra 8th layer built on top of HTTP.
HTTP is merely a transparent transport between a RPC client and a RPC
server, any other request-response capable procotol could fit. The
method executed via RPC must be independant from the usage of HTTP as
mean of transportation thus it should not be capable of using the
current request.
The proposed change is to temporary un-expose the current request from
the local-stack during the execution of the RPC method. This fixes the
problem as the code now run like it was executed from the shell or from
a cron. The other benefit is that the pattern used inside the condition:
`if request and request.is_frontend` doesn't need to change.
X-original-commit: f28863bfdaac3f644fcb274c856e0770783ed75c
Part-of: odoo/odoo#104567
Install website_links, create a link e.g. to http://example.com, install
the russian language and translate the default website. Access the short
link you created before-hand, 404 website page not found.
Accessing a website starting with /r is ambiguous, is /r the
link-tracker controller or is /r a russian lang alias (nearest lang
algorithm)? The controller should be prioritary to the lang alias.
This restore the behavior as it was before the httpocalyse.
closesodoo/odoo#99555
X-original-commit: e8a1b4c0cdffb38e48ca980205a61c75c564dee8
Signed-off-by: Jérémy Kersten <jke@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
Commit c06cee44fe corrected a nasty concurrency error in crons but the
serialization error was still logged.
closesodoo/odoo#98741
X-original-commit: 81c70a594a785fc0e32c9ff47c28c92b0f837923
Signed-off-by: Julien Castiaux <juc@odoo.com>
Prior to saas-15.4, access /web/content/res.users/4/image_128 without
being logged in, there was an internal server error because you cannot
access the public user (id=4) image.
The saas-15.4 branch is unaffected by the bug but it is a good idea to
forward-port the test that was next to the fix merged in prior versions.
Closes#94258closesodoo/odoo#98201
X-original-commit: 4deb7e373e046844d997ddc0a11999cc12b765d8
Signed-off-by: Julien Castiaux <juc@odoo.com>
Co-authored-by: ypn <ypnwebdev@gmail.com>
An HTTP Partial Request is a regular GET request with a Range header
that indicate what chunk of the data the browser wishes to download. It
is useful to peek in a video stream or to resume an interrupted
download. The server can respond with a 206 - Partial Content response,
set the appropriate Content-Length and Content-Range headers and only
send a chunk of the data in the body.
Werkzeug's `send_file`, the library function we use to stream files over
http, is fully compliant with partial requests and responses. It
understands the Range request header and create the corresponding
response.
NGINX does also comply with the Range header even via X-Accel-Redirect.
closesodoo/odoo#97422
Signed-off-by: Julien Castiaux <juc@odoo.com>
`_post_dispatch` is a late addition to the httpocalypse. It can be used
to alter the response object e.g. to inject headers or cookies. The
function is automatically called for regular, fallback and error
responses. On the other hand, `_dispatch` is only called when an
endpoint is matched and might not return a response in case of error.
task-2839031
closesodoo/odoo#96651
Related: odoo/upgrade#3710
Signed-off-by: Julien Castiaux <juc@odoo.com>
Create an empty image attachment and load it via an `<img>` html tag in
a document. Upon rendering the image is replaced by the default browser
placeholder instead of the pretty Odoo one.
closesodoo/odoo#95702
Task: 2886028
X-original-commit: 981d56f131f85d33e2415edaef36cfa501d86f0e
Signed-off-by: Julien Castiaux <juc@odoo.com>
Install multiple langages, each time translating the website. In a
private browsing session access /contactus, show the page source, the
multiple alternate URL (`<link rel="alternative">` in the `<head>`) are
all pointing the canonical URL instead of the alternative.
closesodoo/odoo#95683
X-original-commit: 3d4b4d3dcff864613a9e7038137e21674425ed08
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
Each cookie binds to a domain name, multiple cookies can be set for the
same name if they are for different domain name. In this case, two
`session_id` cookies were set: (1) the first set right on the opener at
`opener.cookies[...] = ...`, (2) the second set upon inside of
`http.Request._save_session` because the session was rotated upon login.
The problem is that the former cookie (the one set on the opener, the
one *not* rotated) was used instead of the second cookie (the one
holding the registered user) in the subsequent queries. There is a long
comment explaining the same problem inside of
`odoo.tests.common.HttpCase.authenticate`, we used the same solution as
they did inside of `authenticate`: we diched the previous opener.
closesodoo/odoo#94773
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
Install website and website_hr_recruitment, open /web with ?debug=1, go
to website > configuration > redirect, create a temporary (302)
redirection from `/jobs/detail/experienced-developer-4` to `/404`. Open
the `/jobs/detail/experienced-developer-4` as admin and unpublish the
page. Open the same URL via private browsing (so that you are not
connected), you get the default 403 - Forbidden page, you were not
redirected to the 404 - Not Found page.
Custom 301 (permanent) and 302 (temporary) redirections are fallback
redirections when the requested page does not exist or is not accessible
to the current user. The HTTPocalypse broke the later case, it was not
checking for existing redirection upon access error.
The use case is the one supported with [1] where people want/need to
display something better than a 403 when they unpublish a record like a
job position for instance (most of the requested cases on opw).
Indeed:
- People have link to that record/job everywhere on the internet
- The job position / record is no more relevant, and people need to
unpublish it
- People don't want to delete it (or can't sometimes due to record
relations)
- People don't want visitors to land on a 403, mainly because it is a
non customizable advanced/technical page (it displays a technical
message including the record name etc)
- Their need is to either land a their customizable friendly 404 or
sometimes on another record to promote it.
[1]: https://github.com/odoo/odoo/commit/3b9cd536607b1631dd375ab2e5cc94eb814a6e9bclosesodoo/odoo#93981
X-original-commit: eb7eecec976570ae3301c17a04adc9c110d5b14a
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
Co-authored-by: Romain Derie <rde@odoo.com>
Create an empty database and start 2 http workers, go on the web app
menu and install website (don't install website via -i). Once website is
installed, you are redirected on `/website/configurator` but the route
does not exist and it fails with a 500 internal server error.
The problem is due to an invalid registry manipulation introduced in the
saas-15.3's httpocalypse. When new modules are installed the registry
must be reloaded in all workers. The function that determine if the
registry must be reloaded and reloads it is `check_signaling`.
When the current registry is up-to-date, it is returned as-is by
`check_signaling`. When it is outdated, `check_signaling` creates and
returns a new fresh registry; it does not nor discard nor change
in-place the previous (outdated) registry, it is up to the callee to
discard the previous registry itself.
closesodoo/odoo#93579
X-original-commit: 23bdcc3fd31e6908ee6737cb74787aefa5d057a5
Signed-off-by: Raphael Collet <rco@odoo.com>
Signed-off-by: Julien Castiaux <juc@odoo.com>
Fine tuning of da8def8e41closesodoo/odoo#93407
X-original-commit: 7efa8743b1bbe9efc4b81a10331f096e8de1e52d
Signed-off-by: Julien Castiaux <juc@odoo.com>
Access `/web/image/82303?height=16`, traceback because the placeholder
image cannot be resized to `"16"`.
closesodoo/odoo#92891
Signed-off-by: Julien Castiaux <juc@odoo.com>
Rationnals
----------
Web servers can serve some resources (e.g. static files) right away
without any interaction with the web application. The network model of
most web servers makes them capable of handling thousands of
simultaneous requests when it comes to intensive IO operations such as
streaming data from a file. The network model of Odoo is different: it
is capable of a lot of processing power but can only serve a handful of
requests at a time, i.e. Odoo (with some help from postgres) is
optimized for CPU operations, not IO.
Some users don't configure their web server, they use a basic
configuration that relay all requests to Odoo. The result is that many
Odoo HTTP Workers can be busy streaming static files instead of
processing other requests. This can lead to a worker starvation, i.e.
all workers are busy streaming files and cannot process new requests.
X-Sendfile
----------
In this work, we add the support for the [X-Sendfile] header family,
they are multiples http headers that can be used by the web application
to communicate with the web server in order to delegate the delivery of
files stored on the file system. Odoo still receives the request but it
does no more stream the file content from within its HTTP worker,
instead it skips the response body altogether and sets the `X-Sendfile`
special header with the path of the file on the filesystem. The web
server intercepts that special header, open the file and stream it.
Using those headers, we can use the best of both the web application and
the web server. The web application is still responsible to locate the
resource and verify the access rights, the web server is still
responsible of streaming the content.
Using X-Sendfile is opt-in via the `--x-sendfile` CLI flag. We set both
`X-Sendfile` (apache) and `X-Accel-Redirect` (nginx). If you are using
apache, make sure `mod_xsendfile` is enabled. If you are using NGINX
you have to add the following location block:
location /web/filestore { # custom path, hardcoded within Odoo
# Prevent access from the outside world, i.e. makes this
# route only accessible via X-Accel. MANDATORY!!!
internal;
# Give access to the filestore using this server's
# permissions. Odoo is in charge of verifying the access
# rights.
alias /path/to/odoo/data-dir/filestore;
}
The Odoo [deployment documentation] has been updated accordingly.
[X-Sendfile]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
[deployment documentation]: https://www.odoo.com/documentation/master/administration/install/deploy.html#serving-static-files-and-attachments
Changes to the API
------------------
To benefit most from X-Sendfile, all APIs related to streaming content
over HTTP has to be adapted. They are: (1) `request._serve_static`,
(2) `ir.http._serve_fallback`, (3) `/web/content` and (4) `/web/image`.
Each used it own way to deliver content: (1) `_serve_static` was using
`send_file` (flask's send_file that as been vendored with odoo 10
years ago and not maintenained since then), (2) _serve_fallback was
handcrafting a `werkzeug.wrappers.Response`, (3) /web/content-image were
using the "binary server" `ir.http.binary_content` API.
I has been decided to remove all 3 APIs and to merge the code inside of
the new `http.Stream` object and the `ir.binary` helper model.
A Stream wraps what is going to be sent to the browser, it can be a path
to a file on the locale filesystem, a blob of raw data or an URL to an
external resource. The Stream also holds various metadata that are
mainly used for caching. The preferred way to create a Stream is via one
of its three factories so that all the metadata are set. The factories
are: `from_path`, `from_attachment` and `from_binary_field`. A stream
instance exposes a single method `get_response()` used to create the
corresponding HTTP response object out of the stream.
Inside of `ir.http` were a few methods that were not related to the http
routing and formed what was called the "binary server". All those
methods have been removed and the feature have been refactored inside of
the new `ir.binary` model. The removed methods are:
- `_xmlid_to_obj`
- `_get_record_and_check`
- `_binary_ir_attachment_redirect_content`
- `_binary_record_content`
- `_binary_set_headers`
- `binary_content`
- `_response_by_status`
- `_get_content_common`
- `_content_image`
- `_content_image_get_response`
- `_placeholder_image_get_response`
The new `ir.binary` abstract model exposes the following utilities:
**`_find_record`**
Find an attachment or a record with a binary-field out of an xmlid or
out of a pair record-model/record-id. Check the access rights and the
access token.
**`_get_stream_from`**
Create a Stream from an attachment or a record with a binary-field.
**`_get_image_stream_from`**
Same as `_get_stream_from` but adapted for images. It sets a sensible
ETag on the stream and has image resizing support.
**`_placeholder`**
Get the image placeholder blob.
Testing
-------
It is possible to test the web server configuration using the
`test_http` module. Install the module then run the unittest using the
`webserver` test-tag. By default it attempts to connect to a web-server
running on `http://localhost:80`, you can change this URL by setting the
`WEB_SERVER_URL` environment variable.
odoo-bin -i test_http --stop-after-init
WEB_SERVER_URL='http://localhost:80' odoo-bin --test-tags webserver --stop-after-init
closesodoo/odoo#88134
Task: 2801675
Related: odoo/documentation#2083
Related: odoo/enterprise#26191
Signed-off-by: Julien Castiaux <juc@odoo.com>
We introduce a new decorator/context-manager to catch some exceptions
and re-raise them as another error. This utility main's purpose is to
hide a route that a user has no access to behind a fake HTTP 404 Page
not Found error.
The utility is at `odoo.tools.misc.replace_exceptions(*exceptions, by)`.
Its usage is as follow:
@route('/some/route', auth='public')
@replace_exceptions(AccessError, AccessDenied, by=NotFound())
def some_route(self):
if not request.session.uid:
raise AccessError("Must be connected to see this route")
...
Or as a context-manager if you don't want to except an entire function:
@route('/some/route', auth='public')
def some_route(self):
with replace_exceptions(AccessError, AccessDenied, by=NotFound()):
if not request.session.uid:
raise AccessError("Must be connected to see this route")
...
Task: 2800772
Close: #90433
Part-of: odoo/odoo#88134
When the homepage was set to a different page than /, the resulting page
was blank.
The problem is related to a change of the rerouting algorithm, before
the httpocalypse it was re-dispatching the request itself, now it is up
to the called to do so. Here it is what `serve_path` does.
closesodoo/odoo#91034
X-original-commit: b00fefec2a26eb36be8a13cb845ee4f12f87d4eb
Signed-off-by: Julien Castiaux <juc@odoo.com>
Signed-off-by: Romain Derie (rde) <rde@odoo.com>
A reroute in an internal redirection, the URL on the current request is
changed. The actual changes occurs on the "WSGI environment", a
dictionnary that is populated with the request information by the WSGI
server (werkzeug in our case). Among all the properties there are
PATH_INFO, RAW_URI and REQUEST_URI.
* PATH_INFO is the standard value, it contains a RFC-3986 path.
`request.httprequest.path` is based on that value.
* RAW_URI and REQUEST_PATH are two custom values set by werkzeug to
mimic other popular wsgi servers (gunicorn, uwsgi, mod_wsgi), they
also only contain a RFC-3986 path.
When rerouting only PATH_INFO and RAW_URI are updated. REQUEST_URI is
left as-is in order to keep the original path somewhere. When building
the cannonical path, one must use the original path instead of the
rerouted one.
X-original-commit: 779486c4dff23b946b4b14e3dca164ecca4f85a8
Part-of: odoo/odoo#91034
When started with the CLI option --dev=werkzeug, errors in controllers
are caught by a friendly web debugger. This debugger should not be
started in case of error in a JSON-RPC controller.
closesodoo/odoo#90420
Task: 2837457
X-original-commit: 711a29e76f55e87d73dad89ddbef889dd5c7af10
Signed-off-by: Julien Castiaux <juc@odoo.com>
With this work we relax the http controller so that it accepts all
requests, including `Content-Type: application/json`. We also enrich
the framework with two new helper methods dedicated to serialaze json
requests and responses:
- `request.get_json_body()`, loads the json content from the request's
body and returns the corresponding python object (usually a dict).
- `request.make_json_response(data)`, dumps `data` to json and makes an
http response out of it.
closesodoo/odoo#86300
Task: 2779837
Signed-off-by: Julien Castiaux <juc@odoo.com>
Commit "[IMP] core: don't save visitor default session" moved the geoip
from the session to the request with a deprecation warning. This commit
adapts the remaining modules to use `request.geoip` instead of
`request.session.geoip`.
Geoip is always set on the request but it can be an empty dictionnary in
case the geolocalization failed.
closesodoo/odoo#86015
Task: 2789035
Related: odoo/enterprise#25192
Signed-off-by: Julien Castiaux <juc@odoo.com>
Julien Castiaux