Commit Graph
23 Commits
Author SHA1 Message Date
Hitesh Trivedi 53fe4007f4 [MIG] base_import: migrate tests
Some changes were required since in new API, fields label
are not manage as before (defaulf value is field name, and
not 'unknown' anymore).
2016-07-07 14:44:10 +02:00
Hitesh Trivedi 46df9da439 [MIG] base_import : migrate to new api 2016-07-07 14:44:10 +02:00
Hitesh Trivedi cf3e8324f1 [MOV] base_import: module directory organization
Note: move test models file in models to not import tests
directory, which breaks the mute_logger of test cases
2016-07-07 14:44:10 +02:00
Raphael Collet 8da9116f5d [REM] models: remove deprecated method BaseModel.import_data() 2016-06-16 16:50:46 +02:00
Cedric Snauwaert 841f208b22 [REF] base_import: simplify import by reducing choice for the columns based on heuristic
automatically try to detect type of columns from the first 10 rows of file:
If column only contains integer, only show, int, float, monetary, m2o, o2m, m2m fields in wizard
If column only contains true/false values, only show boolean fields in wizard
etc

add an advanced mode which is the same as previous import (show all fields)

Support different date format and float value with currency as well as float value with parenthesis to reprensent negative value.
2016-04-07 09:42:23 +02:00
Christophe Simonis c169c59e69 [FIX] base_import: adapt forward-ported test to v9 2016-02-24 18:45:51 +01:00
Christophe Simonis dd875ed1a1 [MERGE] forward port of branch saas-6 up to f5ed710 2016-02-24 18:02:53 +01:00
Nicolas Lempereur d4e5019ec7 [FIX] web: csv export newline for text/html field
CSV export was added in f146e2a and since then newline were not
exported.

But new line should be allowed if the string is quoted by " characters
which is done in Odoo.

closes #11005
opw-667853
2016-02-24 11:26:16 +01:00
Xavier Morel 97d2b758fe [FIX] catch exception on incorrect mime type
If the user provides a known mimetype which doesn't actually match the
file as far as we're concerned, warn and fallback on extension rather
than blow up.

This comes up when a user installs Excel on Windows: it associates CSV
files with itself and sets their mime type to application/vnd.ms-excel,
which is not correct and leads to us trying to load a CSV with xlrd (an
excel-parsing library).

Fixes #9779
2016-02-17 15:45:11 +01:00
Xavier Morel f5a1693f06 [ADD] base_import: support for XLS and ODS files
* added intermediate ``_read_file`` step dispatching between CSV, ODS
  and XLS(X) parsing
  - primary dispatch on mime type, secondary on file extension:
    + OS may not provide a relevant mime type if no software locally
      installed for the filetype (e.g. Windows sends excel files as
      application/octet-stream if excel is not installed, and ODS files as
      zip if OpenOffice/LibreOffice isn't installed)
    + applications may re-register extensions to non-standard mimetypes
      breaking the dispatcher

    So if the mimetype is found trust it, otherwise try with the
    filename's extension (if any)
  - all readers skip lines with only empty cells in their output
  - ODS and XLS content are "CSVified", rows are converted to arrays of
    unicode strings
* UI altered to not assume CSV files everywhere, and avoid returning
  garbage preview data for non-CSV imports

Various:
* added a ``can_import`` utility function to tests.common, can be used
  to skip tests if an optional Python dependency is not installed (but
  one would like tests to run if the dependency is available)
* fixed datetime issue in ir_fields
* added converter for monetary fields (iso float)
* spreadsheet don't have integers, twiddling required to ensure integral
  values won't be serialized as floats (breaking conversion back to
  Python)
* improved some tests by asserting no error is generated (bonus: logs
  the error message if there is one, rather than just saying the result
  is blown)
* because some systems only provide elderly versions of
  XLRD (e.g. current debian stable provides 0.9.2 from April 2013)
  - keep using xldate_as_tuple instead of 0.9.3's xldate_as_datetime,
    workaround is simple
  - check for xlrd.xlsx in case of pre-0.8 XLRD

Authorship:
Ronak Baxi <rba@odoo.com>
Mohammed Shekha <msh@openerp.com>

Task 10792
Closes #7285
2015-08-26 16:50:09 +02:00
Christophe Simonis f18b46fcde [FIX] base_import: ignore stripped empty lines
Better fix for commit dab37df04d
2015-04-01 13:22:11 +02:00
Xavier Morel 68f14c6870 [FIX] handling of big fields in CSV
fixes #2742
2015-02-27 14:11:21 +01:00
Xavier Morel 65cd4a2a33 [FIX] remove deprecated checks/fast_suite test attributes from standard modules 2015-01-15 14:31:40 +01:00
Martin Trigaux f138aa2608 [FIX] models: display_name and name_get mismatch
- display_name uses name_get and not the other way around:
name_get should not call _compute_display_name, _compute_display_name should call name_get.
The previous behaviour was not backward-compatible with the old api.
All the models redefining name_get would have 2 different behaviors between name_get and display_name.

- Do not set an inverse function to display_name:
In most cases, writing on display_name writes on _rec_name (if any, not mandatory).
If the display_name computation is redefined, we need to redefine as well the inverse method to avoid unexpected behaviour
This required to also modify tests in base_import as readonly fields are avoided.

- Remove search method on display_name:
For the same reason as for the first point, it could be good that searching on display_name use name_search (and not the other way around).
However doing this would be very inefficiant (need to do the search, without limit, extract the ids of the name_get result just to generate
a subdomain ('id', 'in', [...]). As in most cases it would anyway mean to search on the _rec_name it's better to directly do so.

- Changing label to avoid mismatch:
In view displaying the list of fields or when a match is made on the label of a field (e.g. when importing csv file,
matching is made on both label and technical name), the fact that display_name field has '
Calling it 'Display Name' will avoid most errors.

- remove display_name definition from website_forum_doc,ir_model:
These fields are doing the same thing as the display_name of the new api, we can remove them.
We need to keep the one for res.partner as it's a stored field.
2014-07-25 13:58:59 +02:00
Raphael Collet cbe2dbb672 [MERGE] new v8 api by rco
A squashed merge is required as the conversion of the apiculture branch from
bzr to git was not correctly done. The git history contains irrelevant blobs
and commits. This branch brings a lot of changes and fixes, too many to list
exhaustively.

- New orm api, objects are now used instead of ids
- Environements to encapsulates cr uid context while maintaining backward compatibility
- Field compute attribute is a new object oriented way to define function fields
- Shared browse record cache
- New onchange protocol
- Optional copy flag on fields
- Documentation update
- Dead code cleanup
- Lots of fixes
2014-07-06 17:05:41 +02:00
Xavier Morel 9b0f10af12 [FIX] base_import: just loading module should not require unittest2. Reported by M.Taylor
bzr revid: xmo@openerp.com-20120911132446-u1yd48tsfd7km6hn
2012-09-11 15:24:46 +02:00
Xavier Morel cdb6869809 [IMP] add encoding csv option
and generify handling of options a bit, imprive styling, and rename 'quote' to 'quoting'

bzr revid: xmo@openerp.com-20120830141226-exli9s1hh4vij6sd
2012-08-30 16:12:26 +02:00
Xavier Morel 7006b17af3 [IMP] un-b64 the import file field, use raw binary
bzr revid: xmo@openerp.com-20120830103215-trv5cjg9jvj8ano1
2012-08-30 12:32:15 +02:00
Xavier Morel 34f9664377 [FIX] no field selected for import by the user
bzr revid: xmo@openerp.com-20120814142933-myng67f2cily6vfg
2012-08-14 16:29:33 +02:00
Xavier Morel 84ea769476 [IMP] add information to _match_headers
bzr revid: xmo@openerp.com-20120814120737-0zuxqyjhnnx1aye3
2012-08-14 14:07:37 +02:00
Xavier Morel de09433413 [ADD] feeding of import data into import_data
bzr revid: xmo@openerp.com-20120814093813-ql1uq1kb2766db0e
2012-08-14 11:38:13 +02:00
Xavier Morel 45d5003dd6 [ADD] matching and parsing of CSV, preview production
bzr revid: xmo@openerp.com-20120813134219-ijo7d2kwn82qs4tm
2012-08-13 15:42:19 +02:00
Xavier Morel b34767381b [ADD] fields fetching and filtering method for import
bzr revid: xmo@openerp.com-20120810073513-zidmkuw6yjhtuwpj
2012-08-10 09:35:13 +02:00