Use Foundation Models in Xojo

Extras
1

Did you know that macOS 26 (and iOS 26) come with a small local LLM from Apple?

Since it runs local there is no cost involved and no network access. So why not tap into this resource when available to enhance your Xojo application?

You could use the LLM for:

  • Generate text based on key words
  • Summarize texts
  • Tone editing to make some text sound more professional.
  • Translate text

With the FoundationModels classes in MBS Xojo Plugins, you can load a model and send in requests. Let us show you a sample code, which uses our FoundationModelsMBS module to check if this is available and then start a session with the system model:

	if not FoundationModelsMBS.Available then
		MessageBox "Please run on macOS 26."
		quit
	end if
	
	SystemLanguageModels = new SystemLanguageModelsMBS
	
	if SystemLanguageModels.Available then
		'MessageBox "Available"
	else
		var u as integer = SystemLanguageModels.UnavailableReason
		Select case u
		case SystemLanguageModels.UnavailableReasonAppleIntelligenceNotEnabled
			MessageBox "Apple Intelligence not enabled."
		case SystemLanguageModels.UnavailableReasonDeviceNotEligible
			MessageBox "Device not eligible."
		case SystemLanguageModels.UnavailableReasonModelNotReady
			MessageBox "Model not ready."
		else
			MessageBox "UnavailableReason: "+u.ToString
		end Select
		
		quit
	end if
	
	// and let's start a session
	var instructions as string = "Be polite. You run within a Xojo application. "
	
	session = new LanguageModelSessionsMBS(SystemLanguageModels, instructions)

This loads a model. When picking a model Apple let’s you provide a use case. Either you pick the one for content tagging or the general model. Then there is the choice on whether to use guardrails for the model or the one for permissive content transformations.

Then you can start a session and provide instructions. The idea is that instructions tell the LLM what to do and then provide the text to work on as prompt. You may run FoundationModels.Respond multiple times to ask questions within the session and refer to previous prompts and responses.

We pass our input text to the model to get an asynchronous response:

	Sub SendRequest()
		AskButton.Enabled = false
			
		var p as PromptMBS = PromptMBS.promptString(InputField.Text)
			
		AddText InputField.Text
			
		session.respond( p, AddressOf Responded)
			
		InputField.Text = ""
	End Sub

Later you get the callback to the Responded method:

	Sub Responded(Response as ResponseMBS, error as string, tag as Variant)
		if Response <> nil then
			var content as string = Response.Content
			
			AddText "> " + content
		end if
			
		if error <> "" then
			AddText "> " + error
		end if
			
		AskButton.Enabled = true
	End Sub

This allows you to run a chat bot. Or pass in some instructions to summarize, translate or complete and then the text to work on.

Please try the new functions in 15.5 version and let us know. And always remember that Apple will probably improve the models over time, so this is the worst one shipping. We look forward to enhancements in upcoming macOS versions.

10 Likes
2

OMG

Christian strikes again !! after the ollama plugin in 2 days, now this ! :scream: :scream: thanks a lot !!

we should rename you Christian Blitz !!

how do you get the content generated by the screen shot ? you fed it web page ?

pretty ccol ! :slight_smile: !!

1 Like
3

The foundation models carry some knowledge. so I just asked the question you see there.

4

Wonderful, thank you!

1 Like
5

yes but at some point it needs internet access to feed the local LLM? the stock lllm doesn’t knwo what xojo is ?they made presentation somewhere will search ? the model is embeded in the OS ?

6

last century was RTFM, now it’s VTFY :slight_smile:

7

It tooks me some seconds to understand (the meaning of VTFY). Thank you for the laugh.

8

How are multiple languages handled in Foundation? The question “what is xojo” got an answer. But “was ist xojo” just had “Ich kann dir bei der Suche nach Informationen zu Xojo nicht helfen.” as result.

9

You can ask Apple :slight_smile:

And you can query it:
class SystemLanguageModelsMBS

So it knows Xojo if asked in English. But the training for German may not be as well.

2 Likes
10

Please remember though, that this approach is limited to the few languages and regions enabled for Apple Intelligence. Much better and universal way is to use the llama.cpp wrapper, where you don’t have any imposed artificial limitations.

1 Like
11

What can I expect from the Foundation Models? I grabbed some text from a page of my manual and fed that into the Foundation Models. I asked a really basic question about a key concept in my app “what is a plan”. The reply was: “A plan is the core of your email archiving. It allows you to conveniently save frequent email archives.” This is grammatically correct but “save frequent email archives” is wrong. I know I can’t expect ChatGPT quality from a local model. But this result is not encouraging at all.

12

I’ve been exploring the Foundation Models in my app and, so far, am pleased with the results. I’m using the local LLM to extract metadata (authors, title, etc) from web pages and PDFs. The results are almost always incomplete, but what is found is usually correctly identified and, for this purpose, very useful. I’ve found that the quality of the instructions and prompts you supply is key to getting good results. I’ve also set the temperature value to 0, minimizing the “creativity” of the model and favoring reproducibility. I initiate a new session for every query, and limit the text to be analyzed to ≤ 5000 characters. That limit empirically works well for me, but will vary depending on the complexity of the instructions/prompts and questions being asked. Ironically, I asked Google AI and ChatGPT how to create good instructions and prompts for the Foundation Models, and their answers were very instructive.

13

Perhaps I’m doing something wrong? I’ve extracted data from webpages. The context in the Foundation Models is too small for me. But then I want some explanations out of a manual.

My second step would have been to reduce “what is a plan” to “plan” (with AI of course) and then build the context from there. But the result of such a simple query with “save frequent email archives” was just so discouraging in contrast to ChatGPT.

14

What are your instructions? They carry more weight than prompts.

15

I am sorry, but I don’t know much details.
You may check Apple’s documentation on details about the models.