Sorting through data in .csv file

General
41

Hmm it definitely is written assuming single byte encodings (that was long ago & far away)

I’m pretty sure text input stream IS indeed byte count baed on the Notes
It will read up to Count BYTES and convert that into a string in the encoding but may require fewer bytes than you read
You could ask it to read 5 but only 3 are used to return a valid string in the encoding provided

But cant you grab the string from the field & then define the encoding as UTF-8 for that field to ungarble it ?

If you have a sample file I can take a look
Mail me offline

42

I think you are right about using DefineEncoding – for CSV, the significant characters (comma, quote, and newline) are all ASCII. In UTF8, bytes 2-N of multi-byte characters are all non-ASCII. So CSVParser should leave them alone. My garbling is therefore probably not occurring in CSVParser. I’ll keep investigating and email you if I can’t get past it, thanks. Meanwhile, back to 2015…

43

It could certainly be occurring there but the 2-N bytes should NOT be any of the relevant bytes that the parser IS looking for to delimit the CSV

I am wondering IF when you grab a value from one of the fields if you then just do Define Encoding as UTF-8 if that will suffice ?

Much like you’d do for grabbing data from a database field

44

Yup DefineEncoding to UTF8 does the trick. Thanks for that nudge. CSVParser wins again!

BTW I generalized it slightly to handle multiple header lines, which my file happens to have. I changed the “treatFirstLineAsHeaders” boolean property to an integer “headerLineCount” (default = 1), then made this change in handleLine:

  • old: if treatFirstLineAsHeaders and lineNumber = 1 then
  • new: if lineNumber <= HeaderLineCount then
45

Bah no sane person has multiple header lines :stuck_out_tongue:

46

[quote=179133:@Axel Schneider]try this

Test[/quote]
How did I Use it? It sounds like a Class, but how do you use it?.
Regards

47

The Example that you posted have a broken link :(, can you re-post it?

Thanks

48

[quote=189004:@Gerardo García]How did I Use it? It sounds like a Class, but how do you use it?.
Regards[/quote]

It is no Class, it is a Xojo Project File (xojo_xml_project)
You can open it with File - Open in Xojo

49

[quote=189009:@Axel Schneider]It is no Class, it is a Xojo Project File (xojo_xml_project)
You can open it with File - Open in Xojo[/quote]
Ok I got it, I can see that inside the project has a General Usage, thanks

50

[quote=189009:@Axel Schneider]It is no Class, it is a Xojo Project File (xojo_xml_project)
You can open it with File - Open in Xojo[/quote]
I saw this “General Usage” inside the sbCSVin:

Sub Action()

’ Set up and called like this:

dim f As FolderItem = GetOpenText(“csv”)
if f = nil then exit sub

dim c As new sbCSV
c.FileName = f

’ Accessing the filename will automatically load the CSV file.
’ The defaults are:
’ Delimiter = ,
’ TextDelimiter = "
’ Decimal = .
’
’ If you want to change these then do so BEFORE setting the folderitem. Like this:

dim c As new sbCSV
c.Delimiter = “;”
c.TextDelimiter = “’”
c.FileName = f

End Sub

So I put this code on a Button on Xojo 2015, Release 2.
And this error happens:

This item does not exist:
dim f as FolderItem = GetOpenText(“csv”)
Line 4

Can’t find a type with this name
dim c As new sbCSV
Line 7

Can’t find a type with this name
dim c As new sbCSV
Line 18.

Am I wrong?

51

this is not in the file from my link.

52

It should be

dim c as new sbCSVIn
53

[quote=188763:@Lars Jensen]Yup DefineEncoding to UTF8 does the trick. Thanks for that nudge. CSVParser wins again!

BTW I generalized it slightly to handle multiple header lines, which my file happens to have. I changed the “treatFirstLineAsHeaders” boolean property to an integer “headerLineCount” (default = 1), then made this change in handleLine:

  • old: if treatFirstLineAsHeaders and lineNumber = 1 then
  • new: if lineNumber <= HeaderLineCount then[/quote]

thinking I should post this on github one day :stuck_out_tongue: