Hi, I am trying to process an image in a low priority thread, so the processing can happen in the background and that the user interface performance is not affected. To be precise, I am generating a picture of audio waveforms using MBS AVFoundation and MBS Picture Plugin. But the image rendering raises a ThreadAccessingUIException. Even though the picture is not affecting the UI yet, but is simply a variable inside the thread.run. This is the code in the Thread.run:
dim error as NSErrorMBS
dim a as new AVAudioFileMBS(pFolderitem, error)
if error <> nil then
MsgBox error.LocalizedDescription
end if
dim ff as AVAudioFormatMBS = a.fileFormat
dim duration as Double = a.Length/a.fileFormat.sampleRate
dim pf as AVAudioFormatMBS = a.processingFormat
Var Channel As Integer = 0
// new buffer to receive data
dim buf as new AVAudioPCMBufferMBS(a.processingFormat, a.Length)
// request data
if a.readIntoBuffer(buf, a.Length, error) then
// only one array will have data
dim Waveform as picture
dim memFloat as MemoryBlock = buf.floatChannelDataCopy(Channel)
if memFloat <> nil then
// RenderSamplesMBS in Picture plugin
Waveform = RenderSamplesMBS(memFloat, memFloat.Size / 4, 2, pMW.TimeLine.Width*20, pMW.TimeLine.Height*2, 0, waveformDarkBlue, waveformLightBlue, waveformDarkBlue, -32, true)
end if
dim MemInt16 as MemoryBlock = buf.int16ChannelDataCopy(Channel)
if memInt16 <> nil then
Waveform = RenderSamplesMBS(memInt16, memInt16.Size / 2, 1, pMW.TimeLine.Width*20,pMW. TimeLine.Height*2, 1, waveformDarkBlue, waveformLightBlue, waveformDarkBlue, 15, true)
end if
end if
Is there a way to use a Thread to process the image? If so, how do you do it? Or are Xojo Threads for text processing only?
I just found out, that the Error was not generated by the picture rendering, but because I tried to access Timline.Width and .Height of the mainWindow… Delivering those values as variables to the thread solved the problem.
But the image processing is still blocking the UI interface. This is how I call the thread:
var m as new FolderItem(MoviePath, FolderItem.PathModes.URL)
WaveformThread.Priority = Thread.LowestPriority
WaveformThread.run(Self, m, Timeline.Width, Timeline.Height)
WaveformThread.pause
// do other stuff
WaveformThread.resume
All the “do other stuff” is blocked, until the image is processed. Any ideas, why this is happening?
Threads in Xojo are cooperative.
This only works well, if your thread can yield often.
So calculating big pictures with RenderSamplesMBS will not yield.
Thanks Beatrix, I understand. Might the solution be to divide the AVAudioPCMBufferMBS into chunks and process it piece by piece? Would that yield? If so, I would be glad to get some help on that…
I’d like to try that. But I don’t know, how I can split AVAudioPCMBufferMBS into chunks. I am guessing something lik this, but obviously that does not work as mid() is only for strings:
var bufArray() as AVAudioPCMBufferMBS(a.processingFormat, 2000000)
var chunkStart as Integer = 0
var BufChunk as integer = 0
for chunkStart to a.Length
bufArray.ResizeTo(BufChunk)
bufArray.Add(BufChunk, mid(buf, chunkStart, 2000000)
chunkStart = chunkStart+2000000
BufChunk = BufChunk+1
next
After digging into MemoryBlock, I came up with this solution:
var chunkSize as Integer = 20000000
var chunkIndex as Integer
if memFloat <> nil then
for chunkIndex = 0 to memFloat.Size
if chunkSize >= memFloat.Size - chunkIndex then // for the last chunk, adjust the chunksize
chunkSize = memFloat.Size - chunkIndex
end
// RenderSamplesMBS in Picture plugin
pWaveformArray.Add( RenderSamplesMBS(memFloat.StringValue(chunkIndex, chunkSize), chunkSize / 4, 2, TimeLine.Width*20, TimeLine.Height*2, 0, waveformDarkBlue, waveformLightBlue, waveformDarkBlue, -32, true) )
chunkIndex = chunkIndex + chunkSize
next
end if
now I just have to find out, how to stitch together the Pictures in the pWaveformArray to the Timeline Canvas. Any suggestions are warmly welcomed.
Well, I claimed victory too early. The above code splitting the memoryBlock does only produce some correct images. pWaveformArray(0) is ok, but the next ones are painted full from the middle line up. No waveforms, just a block from beginning to end…
To draw a picture smaller, you need to use DrawPicture in Graphics class with all parameters to define the input and output rectangle. See this scale function:
Function ProportionalScaled(extends pic as Picture, Width as Integer, Height as Integer) As Picture
// Calculate scale factor
dim faktor as Double = min( Height / Pic.Height, Width / Pic.Width)
// Calculate new size
dim w as Integer = Pic.Width * faktor
dim h as Integer = Pic.Height * faktor
// create new picture
dim NewPic as new Picture(w,h,32)
// draw picture in the new size
NewPic.Graphics.DrawPicture Pic, 0, 0, w, h, 0, 0, Pic.Width, Pic.Height
// return result
Return NewPic
End Function
Well, I work with the same memFloat Variable from the sample code of the MBS Show Samples.xojo_binary_project. But instead of generating the image with the whole memFloat using RenderSamplesMBS(), I work through parts of it using MidB.
But only every other image is rendered properly.
