My project is here:
5 Likes
I updated the code that creates the input.txt file. It originally took 2.5 hours to generate one billion rows of data. I got that down to around a minute.
I know that string processing can be costly, but always forget just how much. My new code uses a MemoryBlock to create an approximately 1 MB chunk that will be output to a BinaryStream, and it fills that MemoryBlock with StringValue for the city name and integers through a Ptr.
I don’t recommend this approach for anything other than situations where speed is the priority, but it’s nice that this kind of thing can be done.
5 Likes
And using preemptive threads, that’s now under 15s.
6 Likes