pengyuthon/odoo_source
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e5361e93be3edbc7465719053daf4f5ccfdffcb0
odoo_source/odoo/tools/test_reports.py
T
Denis Ledoux b03c227e88 [REF] models: refactor fields_view_get, load_views 
Refactor the `load_views` API so it no longer sends multiple times the same
fields description.

e.g.
When `load_views` is called to get the kanban, tree and form views,
the list of fields of the model was sent 4 times:
- Once for each view, with only the fields used in the view,
  in `['fields_views']['kanban']['fields']` for instance
- Once globally, with all the fields of the model, in `['fields']`

The goal of this revision is to change that so it sends the list of all fields
only once.

In addition, if a view contains x2many fields,
the fields description of the comodel is also sent.
It was sent in the `views` key of the view fields dict.
e.g.
When calling `load_views` of `res.partner` to get the kanban,
tree and form views,
the `res.partner` fields description was actually sent 6 times:
- Once for each view
- Once globally
- Once for each view of the many2many field `child_ids` of the form view, in
  - `['fields_views']['form']['fields']['child_ids']['views']['kanban']['fields']`
  - `['fields_views']['form']['fields']['child_ids']['views']['form']['fields']`

The change suggested in this revision is to:
- Remove the fields description for each view in `['fields_views']`.
  As it no longer contains the fields,
  the key becomes `['views']` instead of `['fields_views']`.
- Replace the dict key `['fields']` by `['models']`,
  which is a dict with as key the model name and as values
  the model fields description. It contains the fields description
  for all models implied in the view:
  the model of the main view and the model of all one2many and many2many fields.

With this change, the fields description will only be sent once by model
implied in the view.

In addition, the web client was getting the information about the fields
sometimes in the global fields description list (e.g. `['fields']`),
sometimes in the fields description list of the view type
(e.g. `['fields_views']['form']['fields']`),
making it a pain to try to make changes / performance gain
in these field description dictionaries, because you never knew in which dict
the web client was getting its info.
Now, as there is only one place to get the fields description from,
it's clearer and cleaner.

- one2many and many2many fields views are passed directly in the main view
  architecture rather than being put in the `views` key
  of the field description.
  This is actually easier to treat by the web client,
  and this will allow in a future work to cache an entire view in one block
  of text rather than having to combine multiple cached blocks of text
  to return one view.
- one2many and many2many fields which do not have directly embedded views
  have their views directly injected in the architecture,
  so the web client doesn't have to do RPC calls to `load_views`
  for each one2many and many2many fields not having embedded views.
  For instance, this allow to reduce the number of RPC calls to `load_views`
  from 8 to 1 when loading the form of `product.product`.
  Currently, this behavior is limited to 1 level deep but we consider making it
  go all the way down in future works. We did not do it for the moment because
  in certain cases it rises the processing time and the size (bytes) too much.
  e.g. the sale.order view can be 5 levels deep,
  meaning you can reach 4 dialogs on top the main view.
  ```
  sale.order form > order_line > sale.order.line form > invoice_lines >
  account.move.line form > asset_ids > account.asset form >
  depreciation_move_ids > account.move form.
  ```
  This will also benefit in future works to cache an entire view in one block
  of text rather to having to combine multiple cached block of text
  to get one view.
- `fields_view_get` becomes `get_view`.
  As it no longer returns the fields description,
  keeping the `fields` in the name `fields_view_get` no longer makes sense.
  Hence removing `fields` from the method name, it becomes `view_get`.
  As it gets renamed anyway, we take the opportunity to rename it `get_view`,
  which is more in line with the general getter/setter guidelines
  in the model object world.
- `_fields_view_get` becomes `_get_view`. For the same reasons than above.
- `load_views` becomes `get_views`.
  This is not mandatory, there is no technical reason to rename `load_views` as
  it practically sends the same info as before,
  the view architectures and their fields description. Just in another way.
  We just take the opportunity of this pull request to suggest a cleaner API:
  `_get_view`, `get_view` and `get_views`.
- Arguments `toolbar=False, submenu=False` fo the methods
  `_fields_view_get` and `fields_view_get` are converted to a kwargs `**options`
  in `_get_view` and `get_view`.
  The rationale is that submenu was already no longer used (deprecated)
  and the mobile options is introduced.
  The mobile options is necessary to tell the server to send the mobile views
  for x2many fields (kanban instead of tree).
  Instead of adding a new argument each time we add a new option to
  `fields_view_get`, it seems wiser to have a kwargs `**options` to avoid
  to re-write all overrides each time a new option is introduced.
- `_fields_view_get` returned a dict containing the arch in text and some of the
  view information. Now, `get_view` returns a tuple with the view architecture
  as an `etree` node, and the view as a browse record. The rationale is that all
  overrides of `_fields_view_get` were about modifying the arch only
  (e.g. changing the address format/re-organizing the address related field
  nodes of the partner according to the company country).
  To do so, all these overrides were doing `etree.fromstring` to parse the arch
  which was sent in text to convert it to an `etree`,
  then operations were done on the `etree`,
  and then `etree.tostring` was called to convert back the arch to string.
  With this change of signature to send the arch as an `etree`,
  all these back and forth `etree.fromstring` -> `etree.tostring` are avoided,
  allowing some performance gain and less code in the end.
- A cleanup of the keys returned in the dict of `fields_view_get`
  has been performed in `get_view`:
  - `fields` is removed, as explained above,
  - `view_id` is renamed `id`,
  - `name` is removed, it was unused by the web client,
  - `type` is removed, it was unused by the web client,
  - `field_parent` is removed, it was unused by the web client,
  - `base_model` is removed, it was unused by the web client.
- `filters` is moved from the global dict returned by `load_views`
  (now `get_views`) to the dict returned by `fields_view_get` (now `get_view`)
  as it applies only to the `search` view type.
- Retro-compatible methods for the 3 methods
  `fields_view_get`, `_fields_view_get` and `load_views` are provided,
  with deprecation warnings in them.

- The web client could cache the model fields description
  (as it already caches the views),
  so it doesn't need to fetch them again if it asks for another view of a model
  for which he already has the fields description.
  If we do so, `get_views` could return only the list of models used by
  the views, without the fields description as of now,
  and the web client would then call `fields_get` independently only for
  the models for which it doesn't have yet the fields description.
  This would avoid the server to return the fields description
  and to call `fields_get`, which is costly, for each `get_views`,
  therefore gaining performances.
- Inject the views of the one2many and many2many fields all the way down,
  unlimited depth level, as explained above.
- Cache with `ormcache` the architecture of back-end views.
  This is already done for qweb views, it's not done for back-end views.
  Therefore the postprocessing of the views is performed for each `get_views`,
  which is costly, while the view architecture doesn't change for users
  belonging to the same groups, according to the groups implied by the view.

This pull request is co-authored by
Aaron Bohy (aab) for the web client part and
Denis Ledoux (dle) for the server part.

Part-of: odoo/odoo#87522
2022-04-29 09:57:44 +02:00

300 lines
12 KiB
Python
Raw Blame History

# -*- coding: utf-8 -*-
# Part of Odoo. See LICENSE file for full copyright and licensing details.
""" Helper functions for reports testing.
Please /do not/ import this file by default, but only explicitly call it
through the code of python tests.
"""
import logging
import os
import tempfile
from lxml import etree
from subprocess import Popen, PIPE
from .. import api
from . import ustr, config
from .safe_eval import safe_eval
_logger = logging.getLogger(__name__)
_test_logger = logging.getLogger('odoo.tests')
def try_report(cr, uid, rname, ids, data=None, context=None, our_module=None, report_type=None):
""" Try to render a report <rname> with contents of ids
This function should also check for common pitfalls of reports.
"""
if context is None:
context = {}
_test_logger.info(" - Trying %s.create(%r)", rname, ids)
env = api.Environment(cr, uid, context)
report_id = env['ir.actions.report'].search([('report_name', '=', rname)], limit=1)
if not report_id:
raise Exception("Required report does not exist: %s" % rname)
res_data, res_format = report_id._render(ids, data=data)
if not res_data:
raise ValueError("Report %s produced an empty result!" % rname)
_logger.debug("Have a %s report for %s, will examine it", res_format, rname)
if res_format == 'pdf':
if res_data[:5] != b'%PDF-':
raise ValueError("Report %s produced a non-pdf header, %r" % (rname, res_data[:10]))
res_text = False
try:
fd, rfname = tempfile.mkstemp(suffix=res_format)
os.write(fd, res_data)
os.close(fd)
proc = Popen(['pdftotext', '-enc', 'UTF-8', '-nopgbrk', rfname, '-'], shell=False, stdout=PIPE)
stdout, stderr = proc.communicate()
res_text = ustr(stdout)
os.unlink(rfname)
except Exception:
_logger.debug("Unable to parse PDF report: install pdftotext to perform automated tests.")
if res_text is not False:
for line in res_text.split('\n'):
if ('[[' in line) or ('[ [' in line):
_logger.error("Report %s may have bad expression near: \"%s\".", rname, line[80:])
# TODO more checks, what else can be a sign of a faulty report?
elif res_format == 'html':
pass
else:
_logger.warning("Report %s produced a \"%s\" chunk, cannot examine it", rname, res_format)
return False
_test_logger.info(" + Report %s produced correctly.", rname)
return True
def try_report_action(cr, uid, action_id, active_model=None, active_ids=None,
wiz_data=None, wiz_buttons=None,
context=None, our_module=None):
"""Take an ir.actions.act_window and follow it until a report is produced
:param cr:
:param uid:
:param action_id: the integer id of an action, or a reference to xml id
of the act_window (can search [our_module.]+xml_id
:param active_model:
:param active_ids: call the action as if it had been launched
from that model+ids (tree/form view action)
:param wiz_data: a dictionary of values to use in the wizard, if needed.
They will override (or complete) the default values of the
wizard form.
:param wiz_buttons: a list of button names, or button icon strings, which
should be preferred to press during the wizard.
Eg. 'OK' or 'fa-print'
:param context:
:param our_module: the name of the calling module (string), like 'account'
"""
if not our_module and isinstance(action_id, str):
if '.' in action_id:
our_module = action_id.split('.', 1)[0]
context = dict(context or {})
# TODO context fill-up
env = api.Environment(cr, uid, context)
def log_test(msg, *args):
_test_logger.info(" - " + msg, *args)
datas = {}
if active_model:
datas['model'] = active_model
if active_ids:
datas['ids'] = active_ids
if not wiz_buttons:
wiz_buttons = []
if isinstance(action_id, str):
if '.' in action_id:
_, act_xmlid = action_id.split('.', 1)
else:
if not our_module:
raise ValueError('You cannot only specify action_id "%s" without a module name' % action_id)
act_xmlid = action_id
action_id = '%s.%s' % (our_module, action_id)
action = env.ref(action_id)
act_model, act_id = action._name, action.id
else:
assert isinstance(action_id, int)
act_model = 'ir.actions.act_window' # assume that
act_id = action_id
act_xmlid = '<%s>' % act_id
def _exec_action(action, datas, env):
# taken from client/modules/action/main.py:84 _exec_action()
if isinstance(action, bool) or 'type' not in action:
return
# Updating the context : Adding the context of action in order to use it on Views called from buttons
context = dict(env.context)
if datas.get('id',False):
context.update( {'active_id': datas.get('id',False), 'active_ids': datas.get('ids',[]), 'active_model': datas.get('model',False)})
context1 = action.get('context', {})
if isinstance(context1, str):
context1 = safe_eval(context1, dict(context))
context.update(context1)
env = env(context=context)
if action['type'] in ['ir.actions.act_window', 'ir.actions.submenu']:
for key in ('res_id', 'res_model', 'view_mode',
'limit', 'search_view', 'search_view_id'):
datas[key] = action.get(key, datas.get(key, None))
view_id = False
view_type = None
if action.get('views', []):
if isinstance(action['views'],list):
view_id, view_type = action['views'][0]
datas['view_mode']= view_type
else:
if action.get('view_id', False):
view_id = action['view_id'][0]
elif action.get('view_id', False):
view_id = action['view_id'][0]
if view_type is None:
if view_id:
view_type = env['ir.ui.view'].browse(view_id).type
else:
view_type = action['view_mode'].split(',')[0]
assert datas['res_model'], "Cannot use the view without a model"
# Here, we have a view that we need to emulate
log_test("will emulate a %s view: %s#%s",
view_type, datas['res_model'], view_id or '?')
model = env[datas['res_model']]
view_res = model.get_view(view_id, view_type)
assert view_res and view_res.get('arch'), "Did not return any arch for the view"
view_data = {}
arch = etree.fromstring(view_res['arch'])
fields = [el.get('name') for el in arch.xpath('//field[not(ancestor::field)]')]
if fields:
view_data = model.default_get(fields)
if datas.get('form'):
view_data.update(datas.get('form'))
if wiz_data:
view_data.update(wiz_data)
_logger.debug("View data is: %r", view_data)
for fk in fields:
# Default fields returns list of int, while at create()
# we need to send a [(6,0,[int,..])]
if model._fields[fk].type in ('one2many', 'many2many') \
and view_data.get(fk, False) \
and isinstance(view_data[fk], list) \
and not isinstance(view_data[fk][0], tuple) :
view_data[fk] = [(6, 0, view_data[fk])]
action_name = action.get('name')
try:
from xml.dom import minidom
cancel_found = False
buttons = []
dom_doc = minidom.parseString(view_res['arch'])
if not action_name:
action_name = dom_doc.documentElement.getAttribute('name')
for button in dom_doc.getElementsByTagName('button'):
button_weight = 0
if button.getAttribute('special') == 'cancel':
cancel_found = True
continue
if button.getAttribute('icon') == 'fa-times-circle':
cancel_found = True
continue
if button.getAttribute('default_focus') == '1':
button_weight += 20
if button.getAttribute('string') in wiz_buttons:
button_weight += 30
elif button.getAttribute('icon') in wiz_buttons:
button_weight += 10
string = button.getAttribute('string') or '?%s' % len(buttons)
buttons.append({
'name': button.getAttribute('name'),
'string': string,
'type': button.getAttribute('type'),
'weight': button_weight,
})
except Exception as e:
_logger.warning("Cannot resolve the view arch and locate the buttons!", exc_info=True)
raise AssertionError(e.args[0])
if not datas['res_id']:
# it is probably an orm_memory object, we need to create
# an instance
datas['res_id'] = env[datas['res_model']].create(view_data).id
if not buttons:
raise AssertionError("view form doesn't have any buttons to press!")
buttons.sort(key=lambda b: b['weight'])
_logger.debug('Buttons are: %s', ', '.join([ '%s: %d' % (b['string'], b['weight']) for b in buttons]))
res = None
while buttons and not res:
b = buttons.pop()
log_test("in the \"%s\" form, I will press the \"%s\" button.", action_name, b['string'])
if not b['type']:
log_test("the \"%s\" button has no type, cannot use it", b['string'])
continue
if b['type'] == 'object':
#there we are! press the button!
rec = env[datas['res_model']].browse(datas['res_id'])
func = getattr(rec, b['name'], None)
if not func:
_logger.error("The %s model doesn't have a %s attribute!", datas['res_model'], b['name'])
continue
res = func()
break
else:
_logger.warning("in the \"%s\" form, the \"%s\" button has unknown type %s",
action_name, b['string'], b['type'])
return res
elif action['type']=='ir.actions.report':
if 'window' in datas:
del datas['window']
if not datas:
datas = action.get('datas')
if not datas:
datas = action.get('data')
datas = datas.copy()
ids = datas.get('ids')
if 'ids' in datas:
del datas['ids']
res = try_report(cr, uid, action['report_name'], ids, datas, context, our_module=our_module)
return res
else:
raise Exception("Cannot handle action of type %s" % act_model)
log_test("will be using %s action %s #%d", act_model, act_xmlid, act_id)
action = env[act_model].browse(act_id).read()[0]
assert action, "Could not read action %s[%s]" % (act_model, act_id)
loop = 0
while action:
loop += 1
# This part tries to emulate the loop of the Gtk client
if loop > 100:
_logger.info("Passed %d loops, giving up", loop)
raise Exception("Too many loops at action")
log_test("it is an %s action at loop #%d", action.get('type', 'unknown'), loop)
result = _exec_action(action, datas, env)
if not isinstance(result, dict):
break
datas = result.get('datas', {})
if datas:
del result['datas']
action = result
return True
Reference in New Issue View Git Blame Copy Permalink