Can this be optimized?

I am running tests looking at the speed of the compiled code and ran into a bottleneck dealing with file IO.
This routine takes 17.247 seconds the same code in .net takes .657 seconds. I must be doing something wrong writing to the file system for xojo to be that slow.

Dim f as FolderItem
f=new FolderItem(“ArrayText.txt”)
Dim Output as TextOutputStream = TextOutputStream.Create(f)
For x = 1 To 2000000

Output.WriteLine(cstr(x) + " : " + cstr(y(x)))
next
Output.Close

Possibly the .net version caches the write and writes as a chunk.
Try creating one large string and writing it in a single lump at the end of the loop.
You would do that by adding the rows to an array, then using JOIN on the array.

Don’t use writeline.
Put strings in array, use join and write it in one block.

4 Likes

On my machine using Str instead of Cstr is about 30% faster. (I don’t know if you need Cstr for this though)

2 Likes

This is probably the way to go.

As Christian indicated, writing or reading a single line is MUCH slower than reading or writing in one go - after all your hard disk or SSD is slower by a factor of 10,000 than your memory.

This was no joy as the larger the string became the slower the results became.

I like this it halved the time from 17 to 9 seconds.

Now try str() and use pragma to disable background tasks.

1 Like

I would not have believed it going from cstr to str dropped another 4 seconds. from 17 to 5 seconds overall. Still not close to the .net time but much better.

1 Like

You do not shared that code.

You can also add:
Output.Flush

Before Output.Close.
http://documentation.xojo.com/api/language/writeable.html#writeable-flush

The frustrating part is that the process to load and sort the array is MUCH faster in xojo, but when it comes to writing out to disk xojo is much slower. So far I have used an output array, Str() instead of cstr() and #pragma BackgroundTasks False and the time went from 17 seconds to 4 seconds.

This is part of a test to see if it I can save some time using xojo to create files to load into a data warehouse.

thank you for your help.

You can analyze it to check where time is spent.

And redim array before you fill it.

Unnecessary. Closing buffered channels flushes them before ending.

the .net code that writes out the file
Console.WriteLine("…Write to file")

    Using writer As StreamWriter = New StreamWriter("\array.txt")
        For x = 1 To 2000000
            writer.WriteLine(x.ToString & " : " & a(x).ToString)
        Next

    End Using

using output.flush did not make any difference still at 4 seconds does the close do a flush as well?

What does function a() ?

Sorry a is an array Dim a(2000000) as double
Just to make the test simple.

Just timing the write to file portion but here is the entire listing of the test

Print"The magic happens here."
Dim sOut as string

Dim x As Int32
Dim y(2000000) as Double
dim lineout(2000000) as string

#Pragma BackgroundTasks False

dim a as Double

'Dim StartTime As DateTime = datetime.Now
dim Interval as new DateInterval

For x = 1 To 2000000
y(x) = x/2000000
Next x
Print “Pass One”

For x = 1 To 2000000
a = y(x)
lineout(x) = str(x) + " : " + str(y(x))

Next x
Print “…”
Print “… Write to File”

'Start time here so we just time the writing to disk

Dim StartTime As DateTime = datetime.Now
Dim f as FolderItem
f=new FolderItem(“ArrayText.txt”)
Dim Output as TextOutputStream = TextOutputStream.Create(f)
For x = 1 To 2000000

Output.Write(lineout(x))
next
output.Flush
Output.Close

Dim EndTime as DateTime = datetime.now

Print("Start Time: " + StartTime.tostring)
Print("End Time: " + EndTime.tostring)

Dim ElapsedTime as DateTime

Interval = EndTime - StartTime

Print(“Elapsed Time: " + cstr(interval.nanoseconds ) + " nanoseconds”)
Print(“Elapsed Time: " + cstr(interval.nanoseconds/1000000 ) + " milliseconds”)
Print(“Elapsed Time: " + cstr(interval.seconds) + " Seconds”)
Print(“Elapsed Time: " + cstr(ElapsedTime) + " time”)
dim s as string

s = input

Please use join and a string array!

1 Like

So far we learnt that TextOutputStream in Xojo is not caching and flushes regularly, probably as it’s optimized for writing log files.

The C# class seems to cache items and write them later as block.

So does your C# code include closing the stream and do the write actually? Or just adding to the cache.

For Xojo, please use an array and fill it with strings, do a join() and write it in one block to the file.