Character Reference Support

[cheebusjeebus]

I like write CSV and sometimes i like store complex datas inside. Example i want to put bytes inside or a formatted paragraph however they are written legibly.

Proposed schema.ini:

[mydata.csv]
Format=Delimited(;)
Col1=EntryId Integer Width 8
Col2=SecurityKey Text Width 17
;Col2=SecurityKey Text CharReferences CStrings
Col3=Crc32 Text Width 8
;Col3=Crc32 Hex Uppercase
Col4=Md5 Text Width 32
;Col4=Md5 Hex Lowercase
Col5=Description Text Width 150
;Col5=Description Text CharReferences HtmlXml

mydata.csv test:

"EntryId"; "SecurityKey";             "Crc32";    "Md5",                              "Description"
"1";       "\0\0a3MKWOEL{\x88\rCyu_"; "EA041CBC"; "1919e30750a84780f3e1db0a069ff4b8"; "This key & all other datas require backups!
Kindly retrieve a disk for him."

Sometimes i'm write Hello\nworld! but CSVLint lacking a ability to disambiguate this behaviour. I'm satisfy to know his datatypes. Kindly add him to your project?

[BdR76]

I assume you mainly ask this for validating the columns, is that correct?

It's an interesting request but it would require adding a lot of different data types. I think this can also be achieved by adding support for Regular Expressions. That way you can define a regular expression for a column and the CSV Lint plug-in could validate and check for any non-matching values in that column.

You already requested this earlier in #105 and I think adding support for regex would address a more wider use-case for more users, because this would also allow you to validate columns with zipcodes, telephone numbers, e-mail adresses etc. So it the schema.ini would something like this:

Col3=Crc32 Text Width 8
;Col3=Crc32 Regex ^[A-F0-9]{8}$
Col4=Md5 Text Width 32
;Col4=Md5 Regex ^[a-f0-9]{32}$

You could use it to match all kind of things, although I don't know if there is a regular expression for a CStrings or HtmlXml:

Postal codes     : ^\d{5}$
Dutch postcodes  : ^[1-9][0-9]{3}\s?[A-Z]{2}$
E-mail address   : ^[^\s@]+@[^\s@]+\.[^\s@]+$
Telephone number : ^\+?[0-9\s().-]{7,20}$
ATC code         : ^[A-Z][0-9]{2}(?:[A-Z](?:[A-Z](?:[0-9]{2})?)?)?$
ICD 10 code      : ^[A-Z][0-9]{2}(?:\.[A-Z0-9]{1,4})?$
Crc32 hex 8      : ^[A-Fa-f0-9]{8}$
MD5 hex code     : ^[A-Fa-f0-9]{32}$
etc.