Read your file in pieces. Here is a part of the code that I use to successfully read 10 GB files:
[quote]
do
'check how much data needs to be read
if Globals.StopArchiving then return 0 'parsing cancelled by user
dim FilePosition as Int64 = inputBinary.Position - LenB(lastLine)
LeftToRead = min(110241024, FileLength - FileAlreadyRead)
FileAlreadyRead = FileAlreadyRead + LeftToRead
if FileAlreadyRead = FileLength then FileIsRead = true
if FileIsRead then
mboxData = lastLine + inputBinary.Read(leftToRead)
else
mboxData = lastLine + inputBinary.Read(leftToRead - ReadLeftOver)
dim theRight as String = inputBinary.Read(ReadLeftOver)
dim theRightSplit as Pair = SplitLine(theRight)
lastLine = theRightSplit.Left
mboxData = mboxData + theRightSplit.Right
end if
'now do something with the result
loop until FileIsRead[/quote]
The principle is to read 1MB and then the rest of a line until I get one of the EndOfLine characters. This is for reading mbox files.