Get Audio from Soundcard Input-Device and do fft to get Levels of Frequency (realtime)

General
1

Hi,

I want to get the audio spectrum of the set audio device (Mic, LineIn, Stereomix… Whatever is set) in realtime.

The goal is it, to get a realtime beat detection of the current playing music.
I have to get the peaks of the deep frequencies about 50-100 Hz, from the playing music.

Are there any examples available which do something like this?

I think its not easy to program something like that on my own.

Thank you.

2

I’d be interested in this as well. I was just thinking of an application for something that could do this…

3

These may not work anymore (since Quicktime is no longer supported), but these guys used to work this sort of stuff: http://www.sonicamigos.com/etejkowski/code.html

4

What OS? What robustness?

I’m doing this on mac using apples Audio Toolbox framework. I have the realtime part mostly finished (need to polish off error handling) and working on file read/writing now. For FFT I use vecLib for making spectrograms which is uber fast but there might be something better for beat detection.

The realtime part though requires a callback from the OS that I’ve been told can’t be made safe in Xojo. I’ve been stress testing it and can’t make it crash and haven’t got a straight answer why it shouldn’t work. The limitation is it has to run on the main thread. Anyways, I believe MBS has this feature for both mac and windows.

5

Mainly it should run on Windows, but if it works on Mac and Linux too, it would be a nice addition.

With the Windows API and the winmm.dll it could be possible to get the Input-Device Data with “waveInOpen”.
Has someone tried that already?

6

Looks like PortAudioStreamMBS in the Audio plugin does this for mac/win/linux. $60

Or are you looking for something open source? With declares to winmm there might be the same callback problem as with audio toolbox.

7

I have downloaded the MBS Plugins and tried the “Portaudio Example”

If its possible to add some filters to the recorded Audio, it would be possible to get the Beats. I had to add a lowpassfilter and a threshold value.

Also some intelligent dynamic features will need to integrate. I need a suitable input signal, because of the varying input volume of different Music parts.

8

Its me again. The MBS Sound Input is working, and i think if i would add some mathematic functions i am able to calculate the FFT: But the problem is, that im not allowed to compile this plugin without paying the license.

Maybe there are better alternatives available, which i can use instead. I searched a bit and found “Bass.dll”, its free for non commercial use. It also works on Windows, Mac and Linux.

http://www.un4seen.com/doc/#bass/BASS_ChannelGetData.html

With this i can do FFT directly, without programming functions (FFT) myself.
Does someone has experiences with this?

Thank you.

10

If the source is encrypted let me know and I’ll decrypt it. It’s been up for a while and cannot remember. For Windows only.

[quote=182652:@peter fricke]Mainly it should run on Windows, but if it works on Mac and Linux too, it would be a nice addition.

With the Windows API and the winmm.dll it could be possible to get the Input-Device Data with “waveInOpen”.
Has someone tried that already?[/quote]

11

Yes, BASS can work. I found some discussions using BASS with Realbasic but no projects so far. I’ve never used a dylib file before but a search here led directly to what I needed to know and managed to make it move :slight_smile:

This project only works on mac, where the library is a dylib file. I added a Copy Files Build Step to put that dylib into the built apps Frameworks folder and then the library constant for the declares points to that file. I believe if you just do the same with a dll and add the Windows Platform to the library constant it should work on Windows, but I’m not sure if the path will be the same.

Once that got worked out the functions are pretty easy to declare, at least so far. Really only 3 are needed to begin pulling in samples. The thing is there’s 2 ways to get this data: install a callback (which I don’t know if it’ll have the same callback problem as Audio Toolbox (but this discussion with Aaron Ballman makes me think not!?)) or you can poll it using that function you linked above. I guess maybe you already know all this; anyways I chose to poll.

In this project Module BASS wraps all the library stuff. Constants were implemented as methods returning UInt32 (maybe Xojo constants will work instead). WindowSignal tests the Module by drawing part of the incoming waveform. First, in Open, space is made for the sample data retrieved in each polling, which is going to be 44.1khz mono float.

samples = new MemoryBlock(44100 * 4)

Then the start button initializes the device, starts the recording and starts the polling timer.

dim e As boolean = BASS.RecordInit(-1)

hrec = BASS.RecordStart(44100, 1, BASS.kSample_Float, nil, nil)

theTimer.Period = 1000 / 30  //30fps
theTimer.Mode = Timer.ModeMultiple

hrec is some sorta ‘channel’ identifier. In the API for BASS_RecordStart the return type is HRECORD, which I guessed is a Ptr, and named it before knowing what it is. I think it’s really more a channel object, but everywhere else it’s used it’s typed as DWORD.

Then in polling get the bytes available, copy those bytes to the samples Memoryblock and finally draw

dim bytesAvail As UInt32 = BASS.ChannelGetData(hrec, nil, BASS.kData_Available)

dim e As UInt32 = BASS.ChannelGetData(hrec, samples, bytesAvail)

display.Invalidate

WindowFFT is a duplicate of WindowSignal with just theTimer.Action and display.Paint tweaked to draw frequency.

project http://home.comcast.net/~trochoid/bassRec.zip

12

Nice, it’s working on Windows:

I changed the lib with “bass.dll” and put the file in Windows System32 / SysWOW64 Folder.

The refresh is very slow and the GUI Response also.

The first point could be because of the buffer-length setting in “Bass”. Default setting is 500ms:
Bass Buffer

I think i have to change this value to a lower one, to get more realtime response.

13

Thanks Will, works fine in OSX

14

Glad it’s working.

[quote=183642:@peter fricke]The refresh is very slow and the GUI Response also.
The first point could be because of the buffer-length setting in “Bass”. Default setting is 500ms:[/quote]

For recording, the buffer is 1000-5000ms, default 2000
http://www.un4seen.com/doc/#bass/BASS_CONFIG_REC_BUFFER.html

The Timer should be polling about 30 times a second, every 33.3ms, and getting about 1470 samples each time. On my mac it’s running at that 30fps but will slow down if I resize the window really big; Canvas can’t handle it.

My guess for the slowdown is drawing is just slower or some Windows specific graphics switch needs to be flipped. You can check if it’s the drawing time by commenting out this line in theTimer.Action

display.Invalidate

and replace it with this to see the fps the timer is actually firing at.

//+ add Label1 and property oldTime As double

static pass As integer
pass = pass + 1
if pass mod 30 = 0 then
  dim t As double = Microseconds
  Label1.Text = Format(30/((t-oldTime)/1000000), "-0.0000")
  oldTime = t
end

If it is just the drawing time, well, what do you need to draw when beat detecting? If you do and Canvas isn’t cutting it OpenGL will. Here I duplicated WindowSignal to WindowSignalGL and replaced the Canvas with an OpenGLSurface and it now draws fullscreen at 60fps. (Added bass.dll to kLibBass so it should work out of the box)
http://home.comcast.net/~trochoid/bassRecGL.zip

15

If i try to run your bassRecGL on Windows i got these Errors:

Errors

16

Hmmm, in the 1st and 3rd error somehow BASS. got doubled up.
Change BASS.BASS.ErrorGetCode to just BASS.ErrorGetCode

In the other it seems “aCString.Split” no longer works but I can’t tell why. Cstring implicitly casts to String which has a Split, and Text has a Split, but neither String nor CString implicitly cast to Text so I don’t see where it’s getting confused.

Anyways, comment out all the code in that method, it isn’t needed.

17

Thank you, works good now with OpenGL.

I got about 30-32 FPS now, no matter how the window size is scaled.

(I added a Scale-Factor, because my input-volume wasn’t that high)

18

I just want show you the current state.

I scaled the frequency, to display only low frequencies and added some adjustable threshold values, to get a working beat detection.
Works good at this time.

I captured tje Video with a video-camera, because my Computer slows down, if i use some Screen capture Software.

http://klomeister.eimer-24.de/Daten/Beat_Detect_Test-Video.mp4

Now i got everything what i needed, to realize my detection
Thanks for your Support.