Bug with variant and some encodings?

General
21

I like your solution. But i just wanted to see how Claude.ai solves our issue. :wink:

BTW: If i + 1 > lines.LastIndex Then Exit will be triggered only 4 million times in our case. :smiley:

BTW2: Thereā€™s plenty of room for optimization. For example:

This can be replaced by putting lines.LastIndex into a variable before entering the loop and doing the lines.LastIndex over and over again.
And instead of doing the lineValue = lines(i + 1) for a 4 million times, we could use the lines(i + 1) direct within the Select Case ... End Select Section.

22

This should be checked. I remember being told by a Xojo engineer years and years ago that the end value in a Forā€¦Next loop was only accessed once, right when the loop begins. Perhaps things have changed, it really was a long time ago.

No, because LineValue is used in subsequent calculations, it is more efficient keep the code as-is.

23

OK, here is my optimized code, based on Sashaā€™s file. Iā€™ve noted the speed improvements at the bottom of the function as I made changes to it.

  • Improved odd line checking and Forā€¦Next bounds setting
  • Changed comparison from String to Integer
  • Changed Select Case to If..ElseIf. This provides a significant speed boost.

Let me know how it performs on your machine.

8mlines by Eric.xojo_binary_project.zip (5.5 KB)

1 Like
24

I just tested this on my M4 Pro MacBook, and got the following times:

Time (ticks) IDE Debug or Built App Optimization Level
75 IDE Default
38 App Default
37 App Aggressive

In your source code you are reporting times from 490 ticks down to 269. I wonder why yours is so much slower? What CPU are you running on?

25

Yeah, I noticed that too. I reported the numbers figuring that they would be useful relative benchmarks, even if only for my own testing. All values are from plain debug sessions.

Iā€™m on a 2019 MacBook Pro, 2.3Ghz i9 processor. So getting on the older sideā€¦ I keep it because I need to virtualize x86 Windows (I canā€™t use ARM Windows for software reasons). I have some hardware and performance suspicions about this laptop, I really should wipe it and rebuild.

26

Thank you again all for the responses.

The solutions canā€™t been all implemented in my code because the complete code is much more complicated than the little piece in the first post.

Reading is not done in a single loop. The main loop calls many others that continue reading the file.

Cordialement.

27

Itā€™s hard to say without seeing a lot more of your code, but if currently have many places that are reading the file as youā€™ve shown (read an indicator line, read a value line) thatā€™s going to be pretty inefficient code to maintain. I would suggest you look at reading all the data in at once, using the ToArray functionality, and then pass the array around to the various bits that do the processing. That way youā€™ll get the speed advantages of a single read and simplify your code overall.

If you like, upload your project and weā€™ll look at it. I have a hunch I know what it looks like. :wink:

28

Eric,

Only by using String and Val() instead of Variant, I read the 2 files that I gave, in 6 secondes on my MacBook Pro M3 Pro.

It is good enough, I think, to not made other big changes.

Thank you again.

29

OK! If itā€™s easy to do, you could also replace your Selectā€¦Case statement with If..ElseIf. This made a significant difference in the speed of my code and is easy to implement.

30

Eric,

I tried to replace Select Case with If elseIf not in all methods but only in which are called when reading the file with 8000000 lines because this file contains only DXF line objects.

That does not made a difference.

Thanks again.

31

I read the fil Select Case vs If..ElseIf..

and test it.

In compiled app, there is no difference.

32

Interesting. It made a big difference in debug mode, and it really should make a difference because it does a lot less work than Selectā€¦Case.