Memory Leak in CSV Parser

General
1

I’ve got a significant memory leak in this little bit of code that parses out a single line from a CSV file:

Public Function ParseCSVRow(row As String) As String()
  //  parses a single row of CSV data
  //  properly handles embedded commas, but nothing more complex than that
  
  Var flds() As String
  var fld As String
  Var i As Integer
  Var fieldStart As Integer = 0
  Var insideQuotes As Boolean = False
  Var currentChar As String
  
  
  For i = 0 To row.Length-1
    currentChar = row.Middle(i, 1)
    
    If currentChar = """" Then
      insideQuotes = Not insideQuotes
    ElseIf currentChar = "," And Not insideQuotes Then
      flds.Add(row.Middle(fieldStart, i - fieldStart).Trim)
      fieldStart = i + 1
    End
  Next
  flds.Add(row.Middle(fieldStart).Trim)  //  catch the last field
  
  // flds = row.ToArray(",")
  
  For i = 0 To flds.LastIndex
    fld = flds(i)
    If fld.Left(1) = """" And fld.Right(1) = """" Then
      flds(i) = fld.Middle(1, fld.Len - 2)
    End
  Next
  
  
  Return flds
End Function

The CSV file is about 5 MB, with about 32,000 lines of 8 fields. I don’t hold onto any of the data, it gets inserted directly into a SQLite db, but the app’s memory usage will go from 45MB to as high as 35GB.

i have replaced the above code with a simple row.ToArray(“,”) and there are no leaks, but the data has embedded commas so I can’t use ToArray().

I have also tried the Xcode Instruments app to narrow things down a little more, but haven’t been able to get it to record - it gives a “Required kernel recording resources are in use by another document.” error.

Does anyone have any ideas here? I’m stuck…

2

This code is incomplete in that we don’t see how and when you are initializing flds. Can you post the entire method?

3

I’ve updated the original post with the full method.

4

Norman has an open source CSV parser that may be of use:

In the repository is a 23mb test CSV, and I assume the code works well with that test. I can’t say I’ve ever had a project where I’ve used this, but it could be worth a shot.

1 Like
5

I don’t see anything here. What if you comment out the code where the data is actually stored to the DB, does that make a difference?

6

@Kem_Tekinay I have isolated it to just the code block that parses into fields. I have commented out the DB insert code, and the code that takes the fields and converts them to a class instance.

You can see in the code that I have a commented out row.ToArray() call. If I use this line and comment out the loop above it there is no memory leak.

I guess I can build a minimal test app to see if it really is just that code, but I can’t see why it would leak either.

7

I built a stripped-down test app and it shows very tiny increases (0.1 MB) in app memory usage when parsing that same 5 MB file.

I am going to start adding parts of the main app to the test app to see where that leak is really coming from.

1 Like
8

Alrighty… So it’s not the CSV parsing code. I don’t know how I came to that conclusion other than I must have been commenting out to many things at the same time when trying to isolate the problem.

The leak is in instantiating a class from the data. I’ll play some more with it, but I’ll start a new thread if needed.

Sorry for the false alarm…

2 Likes