C/C++ External dynamic library linkage -- example project?

Getting Started
21

On the FunctionNotFoundException:

@Christian Schmitz – BINGO! (do they have “bingo” in Germany?)

Thanks, Christian – once I set both the Xojo project to 64-bit, and the .dylib to 64-bit, there was no issue. And, the Xcode build was making a64-bit only build as the “Debug” setting (“Active Architecture only”), instead of the combined 64-bit / 32-bit .dylib for “Release” build, which is why the 32-bit Xojo build failed.

This was a definite newbie mistake – in this case “newbie” being I’m not used to thinking about 64-bit after years of doing Carbon 32-bit development!

On Symbol Visibility:

[quote]Normally I would expect you need to mark functions to be exported and publicly available
[/quote]

Usually, for a dylib/dll I would create a #define like this:

      #define EXPORT __attribute__((visibility("default")))

And then use the -fvisibility=hidden command line / Xcode option to hide everything else. The way, the EXPORT can be redefined for Windows DLL symbol visibility:

      #define EXPORT __declspec(dllexport)

But I was so impatient to just see it work for a simple example that I didn’t flesh everything out.

I had best do a full-on test of the different intrinsic types, and test the C++ DLL with visible C symbols on the Visual Studio side. Maybe there’s a way to upload the resultant Xcode / Visual Studio projects (or makefiles), as well as Xojo project so someone else can just benefit – the kind of soup-to-nuts example I was asking about in my original posting.

22

@Eli Ott – I’ve come across that linking and install names article dozens of times over the past few years. It’s a great article!

23

@Garth Hjelle , @Christian Schmitz – this is a continuation of my trying to figure out the exact syntax for sending the C-API a pointer to a block of UTF-8 bytes, and getting them back. But my attempt to do so (below) just crashes inside the C++ library.

XOJO:

[code]soft declare function newMyCPPClass lib dylibLocation as Ptr
soft declare sub setACStringFromClass lib dylibLocation (clsPtr as Ptr, inUTF8Text as Ptr)

dim classPtr as Ptr = newMyCPPClass()

Dim t As Text = “Jåbberwøcky”
Dim utf8Data As Xojo.Core.MemoryBlock
utf8Data = Xojo.Core.TextEncoding.UTF8.ConvertTextToData(t)
dim utf8Ptr as Ptr = utf8Data.PtrValue(0)

setACStringFromClass(classPtr, utf8Ptr) // <-- crashes in dylib, when C++ std::string assign() called (below)
[/code]

C-API in dylib (which calls C++ in dylib):

void setACStringFromClass(HMyCPPClass theClassPtr, const char* inUTF8Text) { reinterpret_cast<MyCPPClass*>(theClassPtr)->SetTestString(inUTF8Text); }

C++ in dylib:

void MyCPPClass::SetTestString(const char * inUTF8String)
{
fPrivateCopyOfString.assign(inUTF8String); // <— this crashes in std::string – probably inUTFString doesn’t point to valid data?
}

I’ve scoured the web for this exact case (Text to UTF-8 buffer, passed to a C-API expecting a const char *, used to initialize a std::string) – but just can’t find anything like it.

24

This takes the first 4(or 8) bytes of the MemoryBlock and interprets it as a Ptr, but you want a Ptr to the MemoryBlock itself. Try using the Data property of the Memoryblock instead.

dim utf8Ptr as Ptr = utf8Data.Data
25

@Andrew Lambert – YES!

One more question – going the OTHER way, if I have a Ptr returned that points to a block of UTF-8 bytes, how do I get those into a MemoryBlock and convert back to Text? This seems like it should work, but doesn’t:

dim returnedUTF8Ptr as Ptr = getACStringFromClass(classPtr) dim data As New Xojo.Core.MemoryBlock(returnedUTF8Ptr) dim theText as Text = Xojo.Core.TextEncoding.UTF8.ConvertDataToText(data) // <-- results in "unsupportedOperationException"

In the above example, I am literally returning a pointer to the same UTF-8 I had passed in. The returned data LOOKS correct, but the ConvertDataToText raises an exception: unsupportedOperationException – even if I pass true as the second parameter.

26

My guess is that ConvertDataToText expects a MemoryBlock with a known/valid Size property. Try:

dim data As New Xojo.Core.MemoryBlock(returnedUTF8Ptr, SizeInBytes)
27

Hmmmm. I don’t return the SizeInBytes, but if the buffer is valid UTF-8, then I guess there shouldn’t be any embedded null bytes, and I could count them…

28

@Andrew Lambert – Again, right you were. Thanks!

Seems like a long way to go just to construct a Text object with some UTF-8 bytes, but maybe there’s a more compact / efficient method.

[code]dim returnedUTF8Ptr as Ptr = getACStringFromClass(classPtr)

// Get the size of the UTF-8 string by counting bytes until first null byte
// Should make this a function (in fact, all this UTF-8 to Text should be a function!)

Dim count As Integer
dim nthByte as UInt8
For count = 0 to 50000
nthByte = returnedUTF8Ptr.Byte(count)
If nthByte = 0 then
Exit
End If
Next

dim data As New Xojo.Core.MemoryBlock(returnedUTF8Ptr, count)
dim theText as Text = Xojo.Core.TextEncoding.UTF8.ConvertDataToText(data, true)

TextArea1.SetString(theText)
[/code]

29

Most C style API’s I’ve used use a CString (null-terminated). Xojo automatically converts your String variable to a CString in the declare function call - to pass a string in. You use MemoryBlock.CStringValue() to get a Xojo string back out. It would be a lot simpler than what you’re doing.

30

@Tim Hare – One thing that’s confusing is that the Xojo documentation says to prefer “Text” over “CString”, and since our app is heavily unicode, that seems to make sense.

All strings going in and out of our C++ dylib / DLL via the C-API are expected to be 100% UTF-8, so all those bytes are going to be null-terminated “c-style strings”.

So I’m a little perplex, and I didn’t see a lot of examples of dynamic library calls showing the difference between CString and Text. Can you demonstrate the syntax of what you’re suggesting for going in and out of the C-API using Cstring and MemoryBLock.CStringValue().

31

Conversion of an UTF8 string in C to Text via String:

[code]// const char * object_getClassName(id obj);
Declare Function object_getClassName Lib “Cocoa” (obj As Ptr) As CString

// Explicit version
Dim c1 As CString = object_getClassName(Ptr(Window1.Handle))
Dim s1 As String = c1 // automatic conversion from CString to String
Dim t1 As Text = s1.ToText()

// Shortened version: CString is directly auto-converted to String
Dim s As String = object_getClassName(Ptr(Window1.Handle))
Dim t As Text = s.ToText()

// One-line alternative with CType conversion
Dim t2 As Text = CType(object_getClassName(Ptr(Window1.Handle)), String).ToText()[/code]
Note that this only works without using DefineEncoding because object_getClassName is returning an UTF8 string.

32

Conversion of an UTF8 string in C to Text via Xojo.Core.MemoryBlock:

[code]// const char * object_getClassName(id obj);
Declare Function object_getClassName Lib “Cocoa” (obj As Ptr) As CString

Dim mb As New Xojo.Core.MutableMemoryBlock(255)
mb.CStringValue(0) = object_getClassName(Ptr(Window1.Handle))
Dim t As Text = Xojo.Core.TextEncoding.UTF8.ConvertDataToText(mb)[/code]
Note that this only works without using DefineEncoding because object_getClassName is returning an UTF8 string.

33

Hi Stephen,
all variable of type String in Xojo are by default UTF8 encoded.
A String has an encoding method to support a different type of encoding from the default one.
This encoding is used when a String i transferred to/from other entities that are not Xojo code like, for example, files or sockets or external dll.

When you write:

dim mystring As String = "The String Text"

the variable mystring contains the above text encoded in UTF8.

A String is an intrinsic Xojo data type that does not have an equal type in C.
To transfer Xojo String to/from C code there is a CString type that it is exactly what is used in C i.e. an array of bytes where the last byte is 0 to mark the end of the string.
When the documentation says to prefer “Text” or “String” over “CString” is because Xojo use the above referred encoding info to better handle the string content.
In C what a char* is referring to is left to the programmer.

When you use an external C function, like in your case, you can simply declare the external function as having CString as parameters and/or return values.
When you call the external C function Xojo automatically create a CString from the String content to pass to the called function.
If the external function returns a CString you can simply assign the returned value to a String because Xojo does the conversion from CString to String automatically.

What the string really contains is left to you i.e. if the Xojo String has an UTF8 encoded value the external C function gets a char* to an UTF8 value.
When you assign to a Xojo String variable a CString containing an UTF8 value returned from an external C function is better to explicitly assign the UTF8 encoding to the Xojo String variable with the DefineEncoding method.

The only thing that you must remember is when the CString value must be retained in a variable contained in the external dll.
In this case don’t store the char* you gets from Xojo but make a copy of the value pointed by the char*.
This must be done because the CString created when you call the external C function is automatically destroyed by Xojo when the external function returns.

Regards.

34

@Eli Ott, @Maurizio Rossi –

Thank you for your detailed explanations – and code examples.

I have integrated your helpful info into (what I hope) would be a very clear, accurate and efficient example for taking the unicode Text from one TextArea, sending it to my dylib as a UTF-8 CString, and then retrieving from the dylib that same text as a UTF-8 CString, converting to Text, and setting TextArea2 to that Text.

[code]CONST dylibLocation = “/Users/stephen/dev/XOJODevelopment/Tests for dynamic library communication/TestLibForXojo/DerivedData/Build/Products/Debug/libTestLibForXojo.dylib”

soft declare function newMyCPPClass lib dylibLocation as Ptr
soft declare sub deleteMyClass lib dylibLocation (clsPtr as Ptr)

dim classPtr as Ptr = newMyCPPClass()

// Get contents of TextArea1 and send it to my dylib

soft declare sub setACStringFromClass lib dylibLocation (clsPtr as Ptr, inUTF8Text as CString)

Dim userEnteredText as Text = TextArea1.Text.ToText
setACStringFromClass(classPtr, userEnteredText.toCString(Xojo.Core.TextEncoding.UTF8))

// Now retrieve that same data from the dylib, and set it in TextArea2

soft declare function getACStringFromClass lib dylibLocation (clsPtr as Ptr) as CString

dim returnedUTF8Ptr as CString = getACStringFromClass(classPtr)
dim theText as Text = Text.FromCString( returnedUTF8Ptr, Xojo.Core.TextEncoding.UTF8 )

TextArea2.SetString(theText)

deleteMyClass(classPtr)[/code]

So is the above efficient, and Xojo-canonical?

For safety’s sake, I would always assume that whatever CStrings come out of the dylib are in fact copies – but if I needed a true const char *, then I could always get a Ptr and keep that – risky and usually not worth the danger.

35

@Eli Ott – Eli, in your second example:

[code]Declare Function object_getClassName Lib “Cocoa” (obj As Ptr) As CString

Dim mb As New Xojo.Core.MutableMemoryBlock(255)
mb.CStringValue(0) = object_getClassName(Ptr(Window1.Handle))
Dim t As Text = Xojo.Core.TextEncoding.UTF8.ConvertDataToText(mb)
[/code]

A problem I ran into (in my previous use of MemoryBlock) was that if the MemoryBlock had a defined length that was LARGER than the actual number of bytes in the returned UTF-8, then the conversion would include the spurious contents of memory after the UTF-8’s null byte. Or does the assignment of mb.CStringValue(0) effectively set the size of the MutableMemoryBlock to the number of bytes (including the null byte)? Or because it’s “mutable”?

36

Things were so much simpler in the old framework. It seems like the new framework has favored safety/explicitness over ease of use. Just for comparison sake

CONST dylibLocation = "/Users/stephen/dev/XOJODevelopment/Tests for dynamic library communication/TestLibForXojo/DerivedData/Build/Products/Debug/libTestLibForXojo.dylib"

soft declare function newMyCPPClass lib dylibLocation as Ptr
soft declare sub deleteMyClass lib dylibLocation (clsPtr as Ptr)

dim classPtr as Ptr = newMyCPPClass()

// Get contents of TextArea1 and send it to my dylib 

soft declare sub setACStringFromClass lib dylibLocation (clsPtr as Ptr, inUTF8Text as CString)

setACStringFromClass(classPtr, TextArea1.Text)

// Now retrieve that same data from the dylib, and set it in TextArea2

soft declare function getACStringFromClass lib dylibLocation (clsPtr as Ptr) as CString

TextArea2.Text=  getACStringFromClass(classPtr)

deleteMyClass(classPtr)
37

@Tim Hare – That IS a lot simpler!

I guess I should watch out OLDER examples the were posted before the “new” framework. When did the new framework become part of Xojo?

38

[code]Declare Function object_getClassName Lib “Cocoa” (obj As Ptr) As CString
Declare Function strlen Lib “usr/lib/libSystem.dylib” (str As CString) As UInt16

Dim cs As CString = object_getClassName(Ptr(Window1.Handle))
Dim len As Integer = strlen(cs)
Dim mb As New Xojo.Core.MutableMemoryBlock(len + 1)
mb.CStringValue(0) = cs
Dim t As Text = Xojo.Core.TextEncoding.UTF8.ConvertDataToText(mb)[/code]

39

The new framework was introduced to support iOS but, up to now, can’t be used as a replacement of the Classic framework in other targets other than iOS.
The Text type was introduced to make a clear distinction between a String containing printable characters from a String containing other values.

My suggestion, like Tim, is to use String instead of Text: as you can see the Text property of a TextArea is of type String.

40

@Maurizio Rossi –

When you say, “a String containing printable characters” – do you mean “ASCII” characters?

All my text MIGHT contain UTF-8 text – is that what you mean by “a String containing other values”?

All our underlying connections to our dylib MUST be able to function across macOS, Windows, IOS, and web. Isn’t that the rational for using the Text type?