RuntimeAddString aka REALAddStrings bugged

I think the RuntimeAddString aka REALAddStrings bugged has a horrible bug, (at least on windows but probably on all platforms?)

If calling it where Str1 has some value and Str2 is NULL then it returns Str1 without increasing the reference count.

Can anyone confirm what I see here ?

let me check

Works here just fine here

[quote]13.10.16 18:59:24,553 test.debug[28412]: MBS Debug: RealbasicVersion 2016.030000
13.10.16 18:59:24,554 test.debug[28412]: MBS Debug: 32-bit
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: testRuntimeAddString
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: a 0xd3b50
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: a# 3
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: b 0x0
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: s 0xd3b50
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: s# 4
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: testRuntimeAddString
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: a 0xd3b50
13.10.16 18:59:24,728 test.debug[28412]: MBS Debug: a# 4
13.10.16 18:59:24,729 test.debug[28412]: MBS Debug: b 0x0
13.10.16 18:59:24,729 test.debug[28412]: MBS Debug: s 0xd3b50
13.10.16 18:59:24,729 test.debug[28412]: MBS Debug: s# 5
13.10.16 19:00:01,894 test.debug[28417]: MBS Debug: RealbasicVersion 2012.020000
13.10.16 19:00:01,894 test.debug[28417]: MBS Debug: 32-bit
13.10.16 19:00:02,069 test.debug[28417]: MBS Debug: testRuntimeAddString
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: a 0x1858cc
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: a# 3
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: b 0x0
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: s 0x1858cc
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: s# 4
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: testRuntimeAddString
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: a 0x1858cc
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: a# 4
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: b 0x0
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: s 0x1858cc
13.10.16 19:00:02,070 test.debug[28417]: MBS Debug: s# 5
[/quote]
this is my log here.


REALstring testRuntimeAddString(REALstring a, REALstring b)
{
	DebugMessage(__FUNCTION__);
	REALstringDef* sa = (REALstringDef*) a;
	REALstringDef* sb = (REALstringDef*) b;

	DebugMessage("a", (void*) a);
	if (sa) { DebugMessage("a#", sa->mPrivateUsageCount); }
	DebugMessage("b", (void*) b);
	if (sb) { DebugMessage("b#", sb->mPrivateUsageCount); }
	REALstring s = REALAddStrings(a,b);
	REALstringDef* ss = (REALstringDef*) s;
	DebugMessage("s", (void*) s);
	if (ss) { DebugMessage("s#", ss->mPrivateUsageCount); }
	return s;
}

and test function.

Thanks for checking Christian.

And yes you are right, I found that memory clobbers before. (I don’t know why though but I know where)

formatMask = REALBuildStringWithEncoding("0",1, kREALTextEncodingASCII); // ASCII

formattedValue = EFormat(integerValue,formatMask);   // <---------- Need to unlock

REALUnlockString(formatMask); <-------- Unlocking here will clobber the memory causing all code that comes after it to become unreliable.

Where EFormat is defined as:
REALstring EFormat(double value, REALstring format)
{
static REALstring (_format)(double, REALstring) = nil;
if (!_format)
_format = (REALstring (
)(double, REALstring)) REALLoadFrameworkMethod(“Format(value as Double, format as String) as String”);

return _format(value, format);

}

If I skip unlocking the formatMask then everything will be good and all that comes after it including the AddStrings is find. It makes no sense.

I forgot to add that I only get this on Strangeness on Windows. But that does not mean it does not exist on Mac and Linux, Windows is usually the most sensitive one to memory clobbering.

yes, windows crashes quicker as it frees pages earlier, so you get an exception quicker.
Mac OS X seems to handle memory differently, so even being released the string is still useable for a while.

I cannot pinpoint it at all something has changed in Xojo its something around the string functions that clobbers memory.

And it started between 2015r4 and current Xojo. As in same code is fine on 2015r4.

Prime candidate is the Format function. Not that many plugins use that one. Calling the Format in most places affects variables in instructions just before it. Which of course is not right. And it does not do it in 2015r4.

At same time then trying to make similar strings in Xojo with same encodings and calling format there does not create any notable problems.

Can you reproduce this in a relatively simple plugin?

I will try I need a little time for it though. I will report back

Ok its reproduced, it was easier than I thought.

Here is link to the cpp file and a compiled plugin DLL:

https://sky.einhugur.com:4433/owncloud/index.php/s/oVliji1FuBRrcCi

The actual code:

#ifdef _MSC_VER
#include “winheader++.h”
#endif

#include “rb_plugin.h”

REALstring EFormat(double value, REALstring format)
{
static REALstring(_format)(double, REALstring) = nil;
if (!_format)
_format = (REALstring(
)(double, REALstring)) REALLoadFrameworkMethod(“Format(value as Double, format as String) as String”);

return _format(value, format);

}

static REALstring TestFunction(REALobject instance)
{
REALstring formatMask = REALBuildStringWithEncoding("#.##", 3, kREALTextEncodingASCII);
REALstring symbol = REALBuildStringWithEncoding(",", 1, kREALTextEncodingASCII);

REALstring someString = EFormat(179.20f, formatMask);

// For the purpose of this test we are not unlocking anything, that might crash anyhow since memory is clobbered at this point.

return symbol; // Expected here is "," but I actually get "179.2"

}

struct TestData
{
void* NoData;
};

REALmethodDefinition TestClassMethods[1] = {
{ (REALproc)TestFunction, REALnoImplementation, “MyTestFunction() as String”,REALconsoleSafe },
};

REALclassDefinition TestClass = {
kCurrentREALControlVersion, // Version
“TestClass”, // Name
nil, // Super Name
sizeof(TestData), // Data size
0, // Something for system use
nil, // Constructor index
nil, // Destructor Index
nil, // Pointer to property def
0, // Property count
TestClassMethods, // Pointer to method def
1, // Method count
nil, // Pointer to Event def
0, // event Count
nil, // *eventInstances
};

void PluginEntry(void)
{
SetClassConsoleSafe(&TestClass);
REALRegisterClass(&TestClass);
}

And code to test it in Xojo is:

Dim test as TestClass = new TestClass()

MsgBox test.MyTestFunction()

Expected result is “,” but result we get is “179.2”

I added also bug report now:
45476 - Format loaded from Plugin SDK clobbers memory on Windows systems

So is there any info on this ? How bad this issue is ? As in is it limited to Format ? or maybe all Dynamic loaded functions with double in it ? And when the issue surfaced ? Those are all important to know since it is source of unexplained support cases.

I believe it’s limited to Format and can be worked around by casting the function pointer to match:

REALstring StringFormat(double value, REALstring formatSpec, RBBoolean localeSavvy)

I’ll get this fixed for 2016r4, so make sure the workaround to 2011r1 through 2016r3.

I presume localeSavvy must be true?

Thanks Joe I will test tonight.

I am a little confused on your note at bottom though on the version range. 2015r4 does not have this issue as far as I know. So I think the workaround needs to be from some 2016 version to 2016r3 ?

Yes.

I suspect its reproducibility depends entirely on what’s already on the stack and I wouldn’t rely on it always working correctly in those versions.

Yes I can confirm it works

I have put it like this:

REALstring StringFormat(double value, REALstring format)
{
static REALstring(_format)(double, REALstring) = nil;
if (!_format)
_format = (REALstring(
)(double, REALstring)) REALLoadFrameworkMethod(“Format(value as Double, format as String) as String”);

if (gRBVersion >= 2011.1 && gRBVersion <= 2016.3)
{
	return ((REALstring(*)(double, REALstring, RBBoolean))_format)(value, format, true);
}

return _format(value, format);

}

Thanks