Commit Graph
3 Commits
Author SHA1 Message Date
Xavier Morel be96188f2f [IMP] base_import: date-parsing patterns
* expand auto-detected date and time patterns (e.g. %b, %I, ...)
* try to make date-pattern-detection clearer
* add a select2 dropdown for date patterns (with a bunch of
  preselected patterns) rather than just an input
* also try to improve other column-matching bits (e.g. less reliance
  on exceptions, attempts to avoid redundant work)
2018-08-09 10:58:52 +02:00
Xavier Morel 36ee8a78ec [IMP] base_import: attempt to automatically guess the separator
It would probably be even better to iterate the file content and get
the non-quoted non-alphanumeric characters as separator
candidates (instead of a hard-coded list) however Python does not seem
to have a decoding iterator (taking bytes and yielding an iterator of
codepoints or even grapheme clusters) — incidentally uniseg seems to
require up-front decoding as well — so that's not really convenient as
we may be dealing with large-ish files and not want to load it
entirely in memory.

An alternative would be to use TextIOWrapper and iterate the file by
buffers of a few ks, and classify that based on either codepoints or
grapheme clusters.
2018-08-07 16:46:29 +02:00
Xavier Morel 5aba5ac99e [IMP] base_import: reintroduce encoding selector & override guess
* if an encoding is explicitly specified, use it and don't guess
* otherwise guess and return the guessed encoding so it can be
  displayed in the configuration UI
* fix less-than-stellar configuration & behaviour of select2 inputs
  to properly reflect underlying values as they get modified, to
  correctly handle future configuration guesses
2018-08-07 13:07:05 +02:00