ElfData plugin Xojo replacement?

I use the ElfData plug-in for our app to speed up some large string manipulations that we need to do.

It appears that the author has no longer decided to continue this plug-in for Xojo. I’ve emailed him a couple times,
but haven’t been able to get a response.

Is there a replacement that would do this sort of functionality, or does anybody have insight as to this person
as to get them to simply add Xojo compatibility?

Thanks,
Gary

Well, we have a lot of memoryblock functions in the MBS Plugin, too.
What do you need?

We do a large string replacement on XML documents, which can be huge (like 1M or more)… replacing tags, etc.

Also, lot of the things we do is “sanitize” the XML so that its not going to choke going to another Web service, removing
control characters, or replacing of bad characters that are done with the data which we have no control over.

Gary

Maybe some combination of the MBS Tidy plugin and an XSLT transformation might do what you want, though I don’t know about the performance issues.

Are you able to create a Xojo XMLDocument object from the XML as you receive it, Gary, or is the string not valid XML at that point?

Often times XMLDocument will throw an exception based on the LoadXML so we “sanitize” the document before that.

ElfData is extremely fast. I mean, really fast.

I have Christian’s MBS plug-ins, so I might consider re-writing things using the Tidy or MemoryBlock functions.

ElfData is a bit “odd” in how it does some of the things it does, but its very good at doing it.

http://www.elfdata.com/plugin/technicalref/index.html

Hell, if the guys not going to maintain the code, I’d be willing to take over the project and port to Xojo for him.

Anybody know him?

Christian,

Which raises an interesting question. One thing I do see missing from MBS is the ability to manipulate strings. Maybe looking at this ElfData plug-in might give you some ideas? :wink:

I do think there’s a lack of “good performance” string tools out there – just sayin’

[quote=35521:@Gary MacDougall]Christian,

Which raises an interesting question. One thing I do see missing from MBS is the ability to manipulate strings. Maybe looking at this ElfData plug-in might give you some ideas? :wink:

I do think there’s a lack of “good performance” string tools out there – just sayin’[/quote]

+1

My understanding is that it was very quick because the author did things in a way that was not recommended.

Having said that, have you tried doing your cleanup with regular expressions? The RegExMBS plugin is very fast, especially if you turn off UTF8 checks.

For example, to clean control characters, except for linefeed, return, and tab, from a 10 MB string with 1 million control characters in it using RegExMBS takes about 160 ms here. This is the pattern:

"(?mi-Us)[\\x00-\\cH\\cK-\\cL\\cN-\\x1F]"

For a 1 MB string with 100,000 control characters, it takes about 15 ms, and for a 100 MB string with 10 M control characters, it takes about 1.5 s. I don’t know how that stacks up to that plugin, but I’d bet it’s favorable.

RegEx would do a good job of cleaning up, agreed.

But I also do some massive replacements, where I’m replacing tags… so even though RegEx’s might work for something like that, its slower and far more complicated to read…

Gary

Not recommended is an understatement.

I guess I’d have to know exactly what you’re replacing to comment.

Gary,

  1. Do all those replacements you perform lead to smaller or equally long strings?
  2. Are all replacements of the simple kind where you specifiy a search string and a replacement string, i.e. could you replace the Elfdata calls with RB’s own Replace functions?

from elfdata web site:

The ElfData plugin - An advanced string processing library for REALbasic. Useful It’s beautifully clean design makes it easy to take on otherwise hard tasks. PC & Mac.

REALbasic: it says all.

Xojo’s Replace is out of the question.

I might have 1000’s of rows coming back from a query that need a string replacement before it gets to any XML processing in Xojo.
The result set is ADO, the payload is ADO Record sets. I don’t have a lot of control over that as the source database could be a FoxPro database (yeah, I know).

So if a query reveals say, 1000 rows, I have to scrub the data before I do much else with it taking it from ADO to our XML format.

Xojo is far too slow for string processing to deal with a look up of 1000 rows in memory. When I had the normal RegEx / Xojo string manipulation, it was taking far too long. With Elf, its about its nominal.

Now this is where people would say, “limit your query”. Yeah, I’d love to that but if you know anything about ADO and VisualFox Pro, you don’t have a lot of options based on standard SQL, you can’t just do a “give me only 100 rows” and then keep tally. Its not that simple, so we’re stuck dealing with nonsense of the data provider and the best way to deal is to get the data, replace the strings and move on.

Its ugly. But its one of those situations where you’re not left with a lot of options.

Can you post the scrubbing code as it is now?

By the way, RegExMBS can be much, much faster than the native RegEx class. I tested something that took 15 seconds by the native but milliseconds with the plugin.

[quote]I use the ElfData plug-in for our app to speed up some large string manipulations that we need to do.

It appears that the author has no longer decided to continue this plug-in for Xojo. I’ve emailed him a couple times,
but haven’t been able to get a response.[/quote]

Hello Gary.

I did not receive any email from you until yesterday. And I replied to it straight away. I’m still waiting for you to give me a proper response. I asked a question that was not answered in your one word reply email.

Gary, if you just ramble on, I can’t help you. I was trying out understand what functionality you need, not how fast these solutions are.

Well, I just tried it. The elfdata.rbx plugin works with Xojo 2013r3 just fine. No problem as far as I see.
So where do you have trouble?
The speed of the functions is truly amazing and there are a couple of example projects which show how to use it.

I looked into this again (last night), and it appears something is happening in my project causing a crash. Which is a whole other issue.
It was crashing doing a FastString (in Elf) and now I can’t seem to reproduce this. What I did was completely remove all the files
from my project and re-created it (which was a pain!)… anyway, I can’t seem to reproduce this, so I believe things are fine.

The odd thing, the same project loaded in 2012r2.1 was NOT crashing…