copy and past as csv string.

General
#1

I have some code that takes a listbox and when select all / copy is done a string is created that is supposed to look like:

[quote]data, data, data, data, carriage_return+line_feed
data, data, data, data, carriage_return+line_feed
data, data, data, data, carriage_return+line_feed
data, data, data, data, carriage_return+line_feed
data, data, data, data, carriage_return+line_feed[/quote]

When I look at the string in the debugger the cr/lf pair look correct in binary 0D 0A, they look like question marks in a diamond in the text display of the string…
But when I paste it from the clipboard the recipient says it can’t paste the data.

[code]Function EditCopy() As Boolean
dim r,c as integer
dim cb as new Clipboard
dim line as string=""
dim lines as string=""

// Heading row
for c=0 to me.ColumnCount-1
if lines <> “” then lines = lines + “,”
lines = lines + me.Heading©
next
lines = lines + chr(&h0d) + chr(&h0a)

// rows
for r=0 to me.lastindex
if me.Selected® = true then
line = “”
for c=0 to me.ColumnCount-1
if line <> “” then line = line + “,”
line = line + me.Cell(r, c)
next
lines = lines + line + chr(&h0d) + chr(&h0a)
end if
next

// Copy to clip board
cb.Text=lines
cb.Close
System.DebugLog(“data copied to Clipboard”)
Return True

End Function
[/code]

#2

You have an encoding problem. What is the encoding of the final string when you look at it in the debugger? I suspect it’s nil so you need to call lines = lines.DefineEncoding( Encodings.UTF8 ) before putting it in the clipboard. If you were to use line = line + EndOfLine.Windows instead, it would probably avoid the problem entirely, but you’d have to test.

Having said that, your code will be slow and you should look into using Join with an array instead. Let me know if you’d like my suggestion.

#3

This does the same thing. Nothing.

[code]Function EditCopy() As Boolean
dim r,c as integer
dim cb as new Clipboard
dim line(-1) as string
dim lines(-1) as string

for c=0 to me.ColumnCount-1
line.Append(me.Heading©)
next
lines.Append( Join(line, “,”) )

for r=0 to me.lastindex
if me.Selected® = true then
redim line(-1)
for c=0 to me.ColumnCount-1
line.Append(me.Cell(r, c))
next
lines.Append ( Join(line, “,”))
end if
next

cb.Text = join(lines, EndOfLine.Windows).DefineEncoding(Encodings.UTF8)
cb.Close
System.DebugLog(“data copied to Clipboard”)
Return True

End Function
[/code]

Is it possible that the number of lines of text is too many?

#4

I don’t think there is a limit, but are you trying it with only 1 or 2 selected rows to test? Also, does the string look right in the debugger before you try to put it into the clipboard?

Which platform is this? Are you examining the contents of the clipboard directly to see what’s in it? (On the Mac, you can do that in the Finder, but I have no idea how you’d do it in Windows/Linux.)

#5

everyone should probably be posting in one or the other
There seem to be more replies in the other one with the exact same title https://forum.xojo.com/7686-copy-and-past-as-csv-string

#6

Yeah, but my reply here came first. :slight_smile:

#7

The forum software frequently does double posts… and I don’t have a way of deleting them, nor to I have a way of editing my posts in case of spelling errors etc… So sorry, but this is the thread I’m following…

To answer your questions:
I am running on Mac OS X, and copying to the clipboard. That clip board should be pastable into a text editor like emacs, but nothing appears when paste. This works if there are four or five lines but doesn’t work when there are 99… I don’t know what the limit is or if it is a size limitation that I am unaware of.

I am pasting a few screen shots here so you can observer the debugger in action …

P.S. Thanks a million everyone.

#8

Show clipboard in finder works!!!
What the heck does that mean!!??
But in excel or word or emacs it doesn’t.
Word says “There is not enough memory or disk space to complete the operation.”
c’mon 99 lines of text?

#9

I just tried this code and was able to paste into Word without issue:

  dim oneLine as string = "first,second,third,fourth,fifth,sixth,seventh,eighth,ninth,tenth,eleventh,twelveth,thirteenth"
  dim lines() as string
  for i as integer = 1 to 1000
    lines.Append oneLine
  next i
  dim t as string = join( lines, EndOfLine.Windows )
  
  dim c as new Clipboard
  c.Text = t
  c.Close

Now I have to believe that there is some gremlin in your actual data. Try changing your code this way:

Function EditCopy() As Boolean
  dim r,c as integer
  dim cb as new Clipboard
  dim line(-1) as string
  dim lines(-1) as string
  
  for c=0 to me.ColumnCount-1
    line.Append(me.Heading(c).ConvertEncoding( Encodings.UTF8 ))
  next
  lines.Append( Join(line, ",") )
  
  for r=0 to me.lastindex
    if me.Selected(r) = true then
      redim line(-1)
      for c=0 to me.ColumnCount-1
        line.Append(me.Cell(r, c)).ConvertEncoding( Encodings.UTF8 )
      next
      lines.Append ( Join(line, ","))
    end if
  next
  
  // Remove gremlins
  dim t as string = join( lines, EndOfLine.Windows ).DefineEncoding( Encodings.UTF8 )
  dim rx as new RegEx
  rx.SearchPattern = "[\\x00-\\x09,\\x0B-\\x0C,\\x0E-1F]"
  rx.ReplacementPattern = "?"
  rx.Options.ReplaceAllMatches = true
  t = rx.Replace( t )

  cb.Text = t
  cb.Close
  System.DebugLog("data copied to Clipboard")
  Return True
  
End Function
#10

Well I did all the and still nothing… However the Finder Clipboard viewer seems happy as a pig in sheep.
I did find something odd that you might consider a ‘gremlin’.


Notice the EFBF…

#11

Good catch, and that’s probably the problem. &hEFBF is not a valid UTF8 sequence. In binary, that lays out as:

11101111 10111111

The first byte (the leading byte) indicates that there will be three trailing bytes, but there aren’t. Somehow your data got munged.

FYI, in UTF8, the starting bits of the first byte tell you how many bytes there will be in the sequence. In this case, it’s “1110…”, so three bytes in total are expected. Trailing bytes always start with “10…”. The only exception are the low-ascii characters (0 thru 127) which are all single-byte and whose first bits are all “0…”

#12

So pardon me Kem, but how do I filter that EFBF?

I am wondering if some where I missed a conversion from the source data.

Happy New Year thanks for all the help!
B.

#13

I posted code above. Did that not work?

#14

No. Not for this particular problem.

#15

Ah, sorry, the code I was referring to was posted in a different thread. This will turn any string into valid UTF-8:

(BTW, the “gremlins” pattern posted above should not have commas in it.)

Protected Function MakeValidUTF8(src As String, convertLongNULL As Boolean = False) As String
  // Turn the given string into valid UTF-8.
  // Filters out invalid characters, so it might return an empty string.
  // If convertLongNULL is true, then it will look for the "modified UTF-8" convention
  // of using &b11000000 10000000 to store a null and convert that to an ordinary
  // null (&h00).
  
  if src = "" then return src
  
  // If the string has an encoding, and is valid in its own encoding, then convert it to UTF-8.
  // My thinking here is that, if it's not valid in its defined encoding, that encoding is wrong and 
  // it should be treated the same as if the encoding were nil.
  if src.Encoding <> nil and src.Encoding <> Encodings.UTF8 and src.Encoding.IsValidData( src ) then
    src = src.ConvertEncoding( Encodings.UTF8 )
    return src // We assume Xojo did the conversion correctly 
  end if
  
  // If we get here, we have to start checking the bytes of the string
  if Encodings.UTF8.IsValidData( src ) then
    return src.DefineEncoding( Encodings.UTF8 )
  end if
  
  // If we get here, we have a non-empty string that is not valid UTF-8.
  // We have to remove the invalid bytes.
  
  dim mb as MemoryBlock = src
  dim p as Ptr = mb
  dim lastIndex as integer = mb.Size - 1
  dim writeIndex as integer
  dim readIndex as integer
  while readIndex <= lastIndex
    
    dim thisByte as integer = p.Byte( readIndex )
    if thisByte <= &b01111111 then
      p.Byte( writeIndex ) = thisByte
      readIndex = readIndex + 1
      writeIndex = writeIndex + 1
      
    else // It's a leading byte so figure out how many valid bytes should be in the group and check them
      dim byteCount as integer
      if thisByte >= &b11111110 then // Invalid byte
        // Do nothing
      elseif thisByte >= &b11111100 then
        byteCount = 6
      elseif thisByte >= &b11111000 then
        byteCount = 5
      elseif thisByte >= &b11110000 then
        byteCount = 4
      elseif thisByte >= &b11100000 then
        byteCount = 3
      elseif thisByte >= &b11000000 then
        byteCount = 2
      end if
      
      if byteCount = 0 then // Invalid byte so skip it 
        readIndex = readIndex + 1
        
        // Make sure we have enough bytes to make a complete character. If not, filter this out.
      elseif ( readIndex + byteCount - 1 ) > lastIndex then
        readIndex = readIndex + 1
        
      elseif convertLongNULL and byteCount = 2 and thisByte = &b11000000 and p.Byte( readIndex + 1 ) = &b10000000 then // It's a long null
        p.Byte( writeIndex ) = 0
        readIndex = readIndex + byteCount
        writeIndex = writeIndex + 1
        
      else
        
        // See if the sequence headed by this leading byte is valid.
        // If so, we will accept the entire sequence.
        dim chunk as string = mb.StringValue( readIndex, byteCount )
        if Encodings.UTF8.IsValidData( chunk ) then
          mb.StringValue( writeIndex, byteCount ) = chunk
          readIndex = readIndex + byteCount
          writeIndex = writeIndex + byteCount
        else // This can't be a leading byte so let's discard it
          readIndex = readIndex + 1
        end if
        
      end if // byteCount = 0
      
    end if // thisByte <= &b01111111
    
  wend // readIndex <= lastIndex
  
  dim r as string
  if writeIndex <> 0 then
    r = mb.StringValue( 0, writeIndex )
    r = r.DefineEncoding( Encodings.UTF8 )
  end if
  
  return r
  
End Function
#16