TextOutputStream Performance Question

General
1

Hi everyone,

i have to write about 1-5 Million lines to a Textfile by using TextOutputStream (ts). Whats more efficient? To append each line to a array of Text (ta) and then use ts.Write(ta) or write each line via ts.WriteLine(line)? Which one has the better RAM performance?

Best regards

2

Usually, writing in chunks is fastest. But try it out yourself.

1 Mio line = 10power6 = 1 MB. Let’s assume you have 100 characters per line = 1 MB * 100 = 100 MB.

That’s doable in one go. Might be better to do a couple of chunks.

3

Thanks Beatrix. What are chunks and how to use them?

4

You need to start somewhere with the chunk size. 1 MB? And then vary it.

Regarding the useage of the chunks it really depends on how you store and access the data. In pseudo-code

dim TextArray(-1) as string for currentThing = 0 to ubound(Things) TextArray.append(Things.getText) if currentThing = 1000 then theBinStream.write(Join(TextArray)) redim TextArray(-1) next

You could calculate the size of the text in the and do the check with the sum of the sizes.

5

Thanks Beatrix. The data is stored in various SQLite tables and temporarily I create objects, which organize all necessary Data from the database. I will have a look forward. Thank you.