I use the ElfData plug-in for our app to speed up some large string manipulations that we need to do.
It appears that the author has no longer decided to continue this plug-in for Xojo. I’ve emailed him a couple times,
but haven’t been able to get a response.
Is there a replacement that would do this sort of functionality, or does anybody have insight as to this person
as to get them to simply add Xojo compatibility?
We do a large string replacement on XML documents, which can be huge (like 1M or more)… replacing tags, etc.
Also, lot of the things we do is “sanitize” the XML so that its not going to choke going to another Web service, removing
control characters, or replacing of bad characters that are done with the data which we have no control over.
Which raises an interesting question. One thing I do see missing from MBS is the ability to manipulate strings. Maybe looking at this ElfData plug-in might give you some ideas?
I do think there’s a lack of “good performance” string tools out there – just sayin’
Which raises an interesting question. One thing I do see missing from MBS is the ability to manipulate strings. Maybe looking at this ElfData plug-in might give you some ideas?
I do think there’s a lack of “good performance” string tools out there – just sayin’[/quote]
My understanding is that it was very quick because the author did things in a way that was not recommended.
Having said that, have you tried doing your cleanup with regular expressions? The RegExMBS plugin is very fast, especially if you turn off UTF8 checks.
For example, to clean control characters, except for linefeed, return, and tab, from a 10 MB string with 1 million control characters in it using RegExMBS takes about 160 ms here. This is the pattern:
"(?mi-Us)[\\x00-\\cH\\cK-\\cL\\cN-\\x1F]"
For a 1 MB string with 100,000 control characters, it takes about 15 ms, and for a 100 MB string with 10 M control characters, it takes about 1.5 s. I don’t know how that stacks up to that plugin, but I’d bet it’s favorable.
But I also do some massive replacements, where I’m replacing tags… so even though RegEx’s might work for something like that, its slower and far more complicated to read…
Do all those replacements you perform lead to smaller or equally long strings?
Are all replacements of the simple kind where you specifiy a search string and a replacement string, i.e. could you replace the Elfdata calls with RB’s own Replace functions?
The ElfData plugin - An advanced string processing library for REALbasic. Useful It’s beautifully clean design makes it easy to take on otherwise hard tasks. PC & Mac.
I might have 1000’s of rows coming back from a query that need a string replacement before it gets to any XML processing in Xojo.
The result set is ADO, the payload is ADO Record sets. I don’t have a lot of control over that as the source database could be a FoxPro database (yeah, I know).
So if a query reveals say, 1000 rows, I have to scrub the data before I do much else with it taking it from ADO to our XML format.
Xojo is far too slow for string processing to deal with a look up of 1000 rows in memory. When I had the normal RegEx / Xojo string manipulation, it was taking far too long. With Elf, its about its nominal.
Now this is where people would say, “limit your query”. Yeah, I’d love to that but if you know anything about ADO and VisualFox Pro, you don’t have a lot of options based on standard SQL, you can’t just do a “give me only 100 rows” and then keep tally. Its not that simple, so we’re stuck dealing with nonsense of the data provider and the best way to deal is to get the data, replace the strings and move on.
Its ugly. But its one of those situations where you’re not left with a lot of options.
By the way, RegExMBS can be much, much faster than the native RegEx class. I tested something that took 15 seconds by the native but milliseconds with the plugin.
[quote]I use the ElfData plug-in for our app to speed up some large string manipulations that we need to do.
It appears that the author has no longer decided to continue this plug-in for Xojo. I’ve emailed him a couple times,
but haven’t been able to get a response.[/quote]
Hello Gary.
I did not receive any email from you until yesterday. And I replied to it straight away. I’m still waiting for you to give me a proper response. I asked a question that was not answered in your one word reply email.
Well, I just tried it. The elfdata.rbx plugin works with Xojo 2013r3 just fine. No problem as far as I see.
So where do you have trouble?
The speed of the functions is truly amazing and there are a couple of example projects which show how to use it.
I looked into this again (last night), and it appears something is happening in my project causing a crash. Which is a whole other issue.
It was crashing doing a FastString (in Elf) and now I can’t seem to reproduce this. What I did was completely remove all the files
from my project and re-created it (which was a pain!)… anyway, I can’t seem to reproduce this, so I believe things are fine.
The odd thing, the same project loaded in 2012r2.1 was NOT crashing…