Big Sur and dylib not working

Targets macOS
#1

Hi. My previous method for connecting with Hunspell dylib is no longer working on macOS 11:

In the code below “dylibLocation” is a constant that points to “/usr/lib/libhunspell-1.2.dylib”

Soft Declare Function Hunspell_create lib dylibLocation (aff_path As CString, dic_path As CString) As Ptr
Soft Declare Function Hunspell_spell lib dylibLocation (obj As Ptr, word As CString) As Integer
Soft Declare Function Hunspell_suggest lib dylibLocation (obj As Ptr, ByRef sugs As Ptr, word As CString) As Integer
Soft Declare Sub Hunspell_destroy lib dylibLocation (obj As Ptr)
Soft Declare Sub Hunspell_free_list lib dylibLocation (obj As Ptr, ByRef sugs As Ptr, count As Integer)

I think it has something to do with the library path no longer existing but a cache of the library instead… does anyone know how to access the cache using declares?

Thanks!

#2

Spell checking is built in to macOS and has been for some time. I don’t have a suggestion to fix this issue, but rather that it might not be an issue needing a fix?

#3

The path is right?

#4

Is this related?

New in macOS Big Sur, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache.


#5

It’s important that I access it directly in this manner.

#6

It’s in Finder as an alias but the original location to the alias does not exist

#7

Thanks. Yes this is related and it suggests using a dlopen() call but I’m not sure how to do this and how to use that in tandem with my current declares or as a replacement.

#8

I’ve tried the code below but no luck.

Const dylibLocation = “/usr/lib/libhunspell-1.2.dylib”

Declare Function dlopen Lib dylibLocation (name As CString, flags As Int32) As Ptr
Declare Function dlerror Lib dylibLocation () As CString
Declare Function dlclose Lib dylibLocation (handle As Ptr) As Integer

Dim handle As Ptr = dlopen(dylibLocation, 1 Or 8)

If handle = Nil Then
Dim exc As New RuntimeException()
exc.Message = dlerror()
Raise exc
End If

Soft Declare Function Hunspell_create lib dylibLocation (aff_path As CString, dic_path As CString) As Ptr
Soft Declare Function Hunspell_spell lib dylibLocation (obj As Ptr, word As CString) As Integer
Soft Declare Function Hunspell_suggest lib dylibLocation (obj As Ptr, ByRef sugs As Ptr, word As CString) As Integer
Soft Declare Sub Hunspell_destroy lib dylibLocation (obj As Ptr)
Soft Declare Sub Hunspell_free_list lib dylibLocation (obj As Ptr, ByRef sugs As Ptr, count As Integer)

Call dlclose(handle)

This raises an exception of function not found on dlopen call.

Any ideas what I’m doing wrong?

#9

dlopen is in /usr/lib/libSystem.dylib

#10

I see. I tried that and it now compiles but handle = Nil with error “dlopen(/usr/lib/libhunspell-1.2.dylib, 9): image not found”

#11

Does the library still exist at that path in Big Sur?
They moved Applications around so I wouldn’t be surprised.

#12

If I use “Go” in Finder it points to an alias of it but if I click the alias to show the original location it says the original location does not exist.

#13

When I look on Mojave in /usr/lib it has libhunspell-1.2.0.0.0.dylib and two aliases libhunspell-1.2.0.dylib and libhunspell-1.2.dylib pointing to it. This makes sense because it allows developers to reference the library using a shortened version of the library name in their declares. But in Big Sur it only has the aliases.

#14

I tried searching online and from what I can see, what you are stating appears to be true for others also. In that the executable is no more, only the symbolic links to a non-existent executable.

Suggests to me that hunspell is a python thing… Bad news…

Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)

#15

Ah. Thanks Sam. If you’re right that means Hunspell (which I think macOS has used up until now for its spellchecking) must have been replaced with something else. Not sure what. Do you know?

Although this suggests they are there but hidden in the cache and that you have to use dlopen() to access them somehow:

New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

But does that mean I could theoretically just include the dylib with my app, use the original soft declare code and set the path of it to @executablepath within the bundle resources etc?

#16

You may have missed my blog post:

#17

Thanks Christian but this only confuses me more. It seems that this is a way to extract libraries from the shared cache via command line but it doesn’t seem to solve my issue with trying to access the library through declares in my Xojo app. Am I missing something?

#18

I’ll be honest and say that I don’t know. I’ve only interacted with the spell check system through a TextArea.

Removing all the stuff that used to come with the OS has two advantages, less maintenance for the macOS engineers and it brings it more inline with iOS, so that once the migration to iOS is complete, there doesn’t need to be a dedicated macOS team. This will help Apple to maintain its high stock price as it will reduce costs.

If this were the case, then surely your declares would work and not report function not found?

Unless… Lemme look at your code again… Ah… I see the problem now, why didn’t I see it earlier, I don’t know and I must apologize for that. Couldn’t see the wood for the tress! The below code will not generate a FNF error, but the dlopen fails and dlerror reports the library is not found.

// --- These declares are part of the System, not hunspell.
//     The fact that this worked at all, is the bug!
const systemLibrary = "system"
Declare Function dlopen  Lib systemLibrary (name As CString, flags As Int32) As Ptr
Declare Function dlerror Lib systemLibrary () As CString
Declare Function dlclose Lib systemLibrary (handle As Ptr) As Integer

// --- These declares are for the hunspell library
// const hunspellLibrary = "/usr/lib/libhunspell-1.2.dylib"
const hunspellLibrary = "/usr/lib/libhunspell-1.2.0.0.0.dylib" // --- Trying the full name, JIC
Soft Declare Function Hunspell_create    lib hunspellLibrary (aff_path As CString, dic_path As CString) As Ptr
Soft Declare Function Hunspell_spell     lib hunspellLibrary (obj As Ptr, word As CString) As Integer
Soft Declare Function Hunspell_suggest   lib hunspellLibrary (obj As Ptr, ByRef sugs As Ptr, word As CString) As Integer
Soft Declare Sub      Hunspell_destroy   lib hunspellLibrary (obj As Ptr)
Soft Declare Sub      Hunspell_free_list lib hunspellLibrary (obj As Ptr, ByRef sugs As Ptr, count As Integer)


Dim handle As Ptr = dlopen( hunspellLibrary, 1 Or 8)

If handle = Nil Then
  Dim exc As New RuntimeException()
  exc.Message = dlerror()
  Raise exc
End If

if handle <> nil then
  // --- In Objective-C closing/releasing an object that doesn't exist can cause a crash.
  Call dlclose(handle)
end if
#19

My guess at this point, is that hunspell being related to python means it’s been removed from the system, otherwise I would have to assume that if it’s hidden away like Apple say and you can load it with a dlopen call, it should open.

But I am unable to get it to work on BS. The code I posted above does return a valid ptr on 10.14.

So…

  • My first recommendation is to copy the library from a previous version of the macOS into your application and update your declares to use the library within your application.
  • If that fails because it might be a python thing. You can try talking to http://pankdm.github.io/hunspell.html to see if they have a version that is independent of python.
  • If it requires the installation of python on your customers Mac, I would then suggest looking for an alternative, unless the customer is in an environment that this can be easily done.
  • My concerns about including this library within your application is it might run into two issues. 1. It will probably fail if the user moves the application when it is running. 2. Apple may not approve of the inclusion of a ex-system library on the  App Store.

Sorry I wasn’t able to get it working for you.

#20

Finally…

A Pro-Tip for the future, when dealing with declares like this, try to recall to include the flags as constants. This will make it easier for you to see what flags you’re using and also anyone else reading the code.

const RTLD_LAZY     = &h01
const RTLD_NOW      = &h02
const RTLD_LOCAL    = &h04
const RTLD_GLOBAL   = &h08
const RTLD_NOLOAD   = &h010
const RTLD_NODELETE = &h080
const RTLD_FIRST    = &h0100    // * Mac OS X 10.5 and later */

Dim handle As Ptr   = dlopen( hunspellLibrary, RTLD_NOW Or RTLD_GLOBAL )