ChatGPT with Stream Chat

General
1

Included with XOJO is the example “ChatGPTConnection Class” which shows how super easy it is to incorporate ChatGPT into XOJO projects.
I’ve modified the example to utilize the ability to stream ChatGPT’s responses to provide a better user experience.
I have also added “Assistants” which is a way to personalize ChatGPT’s responses.

I would greatly appreciate any advice on how to handle server-sent events. I’m far from sure if my solution is robust enough.
Grumpy

6 Likes
2

Great job.

Heads up, the TrimContext() method does absolutely nothing (this is present in the original code also, it’s not the OP’s fault). I mention this because some guys will start playing around and notice how their openai credits are going downhill and also reach the point where the AI will tell them the context limit was reached.

1 Like
3

Thanks Gabriel
Made a temp fix for “TotalContextTokens”

''Count how many tokens have been used in all the prompts and answers in Messages
'Var totalTokens As Integer
'For i As Integer = 0 To ContextHistory.LastRowIndex
'Var message As JSONItem = ContextHistory.ValueAt(i)
'If message.HasKey("usage") Then
'Var usage As JSONItem = message.Value("usage")
'totalTokens = TotalTokens + usage.Value("total_tokens")
'End If
'Next
'
'Return totalTokens

//We don't get "usage" when streaming – this is just a temp fix
Var MessageLength As  Integer
MessageLength = ContextHistory.ToString.Length
Return MessageLength

Need to find a way calculate tokens the proper way

4

I for one, don’t bother with token calculation… Instead, every few messages, I use the tools function and ask the ai to summarize previous context, remove the messages from the storage, and replace them with the summarization.

1 Like