This is the goal that I am trying to achieve:
I would like to be able to take the audio recorded from the phones microphone and process the levels at given frequencies into an array that I could use to create a sort of "Bar graph" visualizer. I also need to calculate the BPM of the song playing to know the cadence at which to update the visualizer at.
What I am really looking for is just getting the frequency array and BPM calculations. I can deal with the actual visualization part if I can just figure out how to process the audio.
This is pretty much what I'm looking to make:
From the research I have done, it looks like a possible solution is using FFT (Fast Fourier Transform). For that I found this: stackoverflow. But that stackoverflow thread is for Java, and my app is currently written in Kotlin. I have read that it is possible to import Java into Kotlin, but I haven't been able to make it work in the couple of attempts I made. So maybe some advice on that would help.
Anyways though, I did end up finding a library that was written for Kotlin Nier Vizualizer. It does have a vizualization similar to one I'd like to reproduce, but for the life of me I cannot figure out where I would pull the frequency array from. I tried reading the buffers before they were passed into the visualizer, but it's just an insanely long string that I'm recieving. I mean, I'm not sure what I was expecting, but I could definitely use some help understanding how I would translate that into a data format that I can actually use.
Here's an example of where I'm at: KeyFrameMaker.kt
fun makeKeyFrame() {
val waveFrac = mWaveAnimator.computeCurrentValue()
val fftFrac = mFftAnimator.computeCurrentValue()
if (mWaveAnimator.hasValueUpdated) {
computedWaveData.originMap { idx, _ ->
(((mDestWaveData[idx].toInt() and 0xFF) - (mPrevWaveData[idx].toInt() and 0xFF)) * waveFrac + (mPrevWaveData[idx].toInt() and 0xFF)).toByte()
}
// Create String for computedWaveData ByteArray
var hexBuffer = ""
for (b in computedWaveData) {
hexBuffer += String.format("%02X", b)
}
println("WAVE COMPUTED: $hexBuffer")
}
if (mFftAnimator.hasValueUpdated) {
computedFftData.originMap { idx, _ ->
((mDestFftData[idx] - mPrevFftData[idx]) * fftFrac + mPrevFftData[idx]).toByte()
}
// Create String for computedFftData ByteArray
var hexBuffer = ""
for (b in computedFftData) {
hexBuffer += String.format("%02X", b)
}
println("FFT COMPUTED: $hexBuffer")
}
}
Here I am printing the ByteArray that is the supposed computedFftData, and the values it prints are just insanely long strings for example: "ED000F01020A0D01FAFAFF0003F700FE00FFFFFFFFFE0000000000FF0000000000000000FF000000010000000001000000000000000000FF...". To say the least, I'm not really sure how to use this to achieve my goal of an array of frequency levels.
At this point I think I will just leave it at that and see if I can get any advice. I've done a bunch of other research as well, but I'm not sure that adding anything more will actually help. Definitely looking to just talk to someone more knowledgeable about this to point me in the right direction.
For reference I will be using this Android Visualizer Class and the RECORD_AUDIO permission found here: Android Visualizer reference
Edit 3: Finally got the visualizer initialized!
So it turns out that from @Tenfour04's digging, he was correct and some android devices have an issue with visualizer initialization. More info on that here: Github Thread.
At this point I am getting data from the FftDataCapture listener from the Visualizer and the audio record session. I can likely figure things out at this point, but if anyone wants to offer advice on best implementations for me and anyone else that ends up reading this, it wouldn't hurt! I'm definitely going to keep updating this post with what I find so that people in the future have an example to work from at least.
The next step is taking the magnitudes and phases array that are being generated from the FFT function and figuring out the best way to take the massive array of values down to about 12 or so... @Tenfour04 mentioned just taking set values at given positions, and I think that would be a fine solution. From what little I know about frequencies though, there isn't an equal distribution of across the spectrum - so if I'm not wrong we need to take more values from the begging of the spectrum than the end... Could be wrong about that though and will have to look into it more.
The other issue I see is that the arrays arent always of an equal length. I believe this means that I need to wait for the next datacapture and recapture the buffer? But not sure if that is accurate either, maybe I can just pull values from certain positions of whatever size array is given? It does look like maybe the default declaration of onWavFormDataCapture has a way of waiting for the size of the ByteArray to equal the size of the mWaveBuffer byteArray, but that ByteArray seems to be of variable size - so this doesn't seem to be a valid solution from my understanding.
Anyways, I'm just spitballing and will update with better info as I get it.
Here's a snip of the audio data with FFT applied:
Magnitudes: 57.0, 32.649654, 123.967735, 31.622776, 121.49486, 48.104053, 47.0, 14.142136, 5.0, 7.2111025, 6.708204, 5.0, 5.8309517, 2.236068, 6.708204, 9.899495, 11.401754, 9.219544, 4.472136, 3.6055512, 4.472136, 1.4142135, 4.1231055, 8.944272, 8.5440035, 18.973665, 17.888544, 20.09975, 15.264338, 9.055386, 6.708204, 4.472136, 3.1622777, 3.1622777, 2.236068, 2.236068, 2.0, 2.236068, 2.0, 0.0, 2.828427, 4.1231055, 3.6055512, 3.6055512, 2.828427, 4.472136, 2.828427, 2.236068, 2.0, 2.0, 2.0, 1.4142135, 2.0, 2.236068, 2.0, 1.4142135, 1.0, 1.0, 2.0, 2.0, 1.0, 1.0, 1.0, 1.0, 2.0, 2.0, 0.0, 1.0, 1.4142135, 1.0, 1.0, 1.0, 1.4142135, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.4142135, 1.4142135, 1.0, 1.0, 1.4142135, 1.0, 1.4142135, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.4142135, 1.0, 1.0, 1.0, 0.0, 1.0, 1.4142135, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0
Magnitudes: 25.0, 69.289246, 132.63861, 129.71121, 21.095022, 23.537205, 23.537205, 9.219544, 22.847319, 14.422205, 10.816654, 8.944272, 2.236068, 0.0, 3.1622777, 8.062258, 20.615528, 2.828427, 4.1231055, 7.81025, 4.0, 2.236068, 3.1622777, 5.656854, 13.038404, 34.713108, 53.712196, 9.848858, 21.023796, 8.062258, 5.0990195, 5.3851647, 3.6055512, 3.1622777, 3.1622777, 3.1622777, 3.1622777, 3.1622777, 2.236068, 2.236068, 2.236068, 2.828427, 2.236068, 1.0, 4.0, 4.0, 3.1622777, 2.0, 2.0, 1.0, 2.0, 1.4142135, 1.4142135, 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 1.0, 2.236068, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.4142135, 1.0, 1.0, 1.0, 1.0, 1.0, 1.4142135, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0
Magnitudes: 2.0, 19.104973, 30.232433, 73.3553, 6.4031243, 27.513634, 76.537575, 43.462627, 17.262676, 12.369317, 13.601471, 13.341664, 9.486833, 11.18034, 15.811388, 13.601471, 45.01111, 37.01351, 44.64303, 21.023796, 1.4142135, 7.28011, 8.5440035, 4.2426405, 13.416408, 21.023796, 16.492422, 11.401754, 3.0, 5.0, 5.0990195, 2.0, 3.6055512, 3.0, 2.0, 2.828427, 4.472136, 2.0, 3.1622777, 3.0, 2.0, 2.828427, 3.6055512, 0.0, 3.1622777, 2.236068, 2.0, 3.0, 2.236068, 3.1622777, 2.236068, 2.0, 2.236068, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 1.0, 2.0, 2.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0, 2.0, 2.0, 1.0, 1.0, 2.0, 1.4142135, 1.4142135, 2.0, 1.0, 1.0, 2.0, 1.4142135, 1.4142135, 1.0, 2.236068, 1.0, 1.4142135, 3.0, 2.828427, 1.4142135, 3.0, 1.4142135, 2.236068, 4.0, 1.0, 1.4142135, 2.0, 1.0, 1.4142135, 1.0, 0.0, 1.4142135, 1.0, 1.4142135, 2.0, 1.4142135, 1.0, 1.4142135, 1.4142135, 0.0, 1.4142135, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.4142135, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0
Phases: 0.0, 2.0638473, 0.52183425, 0.15991312, 1.94013, -1.1383885, -1.2847449, 0.8519663, 2.4980915, 2.3561945, 2.7367008, 2.9441972, 2.7610862, 3.1415927, 2.0899425, 2.0344439, -0.32175055, -2.2794225, 2.3561945, 1.2490457, -1.5707964, -0.98279375, 2.4668517, 1.5707964, 2.3561945, 1.5707964, 3.1415927, 3.1415927, 1.5707964, 2.3561945, 3.1415927, 1.5707964, 1.5707964, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 2.3561945, 1.5707964, -1.5707964, 3.1415927, 0.0, 3.1415927, -2.6779451, 0.0, 3.1415927, -2.3561945, 2.3561945, 0.0, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 0.0, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 0.0, 3.1415927, 3.1415927, 3.1415927, 0.0, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 0.0, 3.1415927, 1.5707964, 0.0, 0.0, 0.0, 3.1415927, 2.6779451, 3.1415927, 2.3561945, -0.7853982, 0.0, -2.819842, 0.0, -1.5707964, -2.0344439, 2.6779451, 3.1415927, 1.5707964, 2.6779451, 1.5707964, -2.3561945, 0.0, 3.1415927, 2.3561945, 2.3561945, 0.0, -2.3561945, 2.3561945, -1.5707964, 3.1415927, 0.0, -2.6779451, 1.5707964, 0.0, 0.0, 3.1415927, 0.0, 0.0, 3.1415927, 3.1415927, 3.1415927, 0.0, -1.5707964, 3.1415927, -2.3561945, -2.3561945, 3.1415927, 0.0, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 2.3561945, 0.0, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 0.0, 3.1415927, 0.0, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 0.0, -2.3561945, 3.1415927, -2.3561945, -2.3561945, 3.1415927, 3.1415927, 3.1415927, -2.3561945, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, -2.3561945, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, 3.1415927, -2.3561945, 3.1415927, 3.1415927, -2.3561945, 3.1415927, -2.3561945, -2.3561945, 3.1415927, 3.1415927, -2.3561945, -2.3561945, -2.3561945, -2.3561945, -2.3561945, 0.0
Phases: 0.0, 2.2694561, -1.7492068, -1.8925469, -0.15702978, 1.2095926, 0.0, 1.4288993, -0.9272952, -0.5880026, -1.1071488, -0.6435011, -0.5404195, 0.4636476, -2.6779451, -2.3561945, -2.2318394, -0.86217004, 1.1071488, 2.158799, -2.6779451, -0.7853982, 1.815775, 2.6779451, -1.929567, 0.32175055, 1.1071488, 1.4711276, 2.1224513, -1.6814536, -1.1071488, -1.1071488, -1.2490457, -1.2490457, -1.1071488, -1.1071488, -1.5707964, -1.1071488, 0.0, 0.0, -2.3561945, -1.3258177, 0.5880026, 2.55359, -2.3561945, -1.1071488, -0.7853982, -1.1071488, -1.5707964, -1.5707964, -1.5707964, -0.7853982, 3.1415927, -2.6779451, -1.5707964, -0.7853982, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, 3.1415927, -2.3561945, -1.5707964, -1.5707964, -1.5707964, -2.3561945, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -2.3561945, -2.3561945, -1.5707964, -1.5707964, -2.3561945, -1.5707964, -2.3561945, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -2.3561945, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -2.3561945, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, 0.0, 0.0, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, 0.0, 0.0, 0.0, 0.0, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, 0.0, 0.0, -1.5707964, 0.0
Phases: 0.0, 1.2170932, -2.5222197, 1.1243883, 0.094951704, 0.21406068, 1.784857, -2.4329665, -0.4048918, -0.5880026, -0.98279375, -1.1071488, 1.1071488, 0.0, 1.2490457, 2.0899425, 2.7430701, 0.7853982, -1.815775, -0.87605804, 1.5707964, -1.1071488, -2.819842, -0.7853982, 1.0040671, 2.2950463, 2.6363025, 2.7233684, -1.1284221, -1.0516502, -1.3734008, -1.19029, -0.98279375, -1.2490457, -1.2490457, -1.2490457, -1.2490457, -1.2490457, -1.1071488, -1.1071488, -1.1071488, -0.7853982, 0.4636476, -1.5707964, -1.5707964, -1.5707964, -1.2490457, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -0.7853982, -0.7853982, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -2.0344439, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -2.3561945, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -2.3561945, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, -1.5707964, -1.5707964, 0.0, 0.0, -1.5707964, 0.0, -1.5707964, -1.5707964, -1.5707964, 0.0, 0.0, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, -1.5707964, 0.0, 0.0, -1.5707964, -1.5707964, 0.0, 0.0, 0.0
Here's the code that I'm working with now:
class MainActivity : AppCompatActivity() {
val SAMPLING_RATE = 44100
val REQUEST_CODE_AUDIO_PERMISSION = 1
private var visualizer: Visualizer? = null
lateinit var magnitudesArray: FloatArray
private var mWaveBuffer: ByteArray? = null
private var mFftBuffer: ByteArray? = null
private var mDataCaptureSize: Int = 0
private var mAudioBufferSize: Int = 0
private var mAudioRecord: AudioRecord? = null
private var mAudioRecordState: Boolean = false
@TargetApi(Build.VERSION_CODES.KITKAT)
@RequiresApi(Build.VERSION_CODES.JELLY_BEAN)
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
ensurePermissionAllowed()
println("OUTSIDE PERMISSIONS CHECK")
mAudioBufferSize = AudioRecord.getMinBufferSize(SAMPLING_RATE, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_8BIT)
mAudioRecord = AudioRecord(MediaRecorder.AudioSource.MIC, SAMPLING_RATE, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_8BIT, mAudioBufferSize)
println("mAudioBufferSize: $mAudioBufferSize")
if (mAudioRecord!!.state != AudioRecord.STATE_INITIALIZED) println("AudioRecord init failed")
else println("AudioRecord init success")
try {
println("BEGIN INITIALIZING VIZUALIZER.")
println("Audio Session ID: ${mAudioRecord!!.audioSessionId}")
//visualizer = Visualizer(mAudioRecord.audioSessionId).apply {
visualizer = Visualizer(0).apply {
enabled = false
captureSize = 512
//captureSize = captureSizeRange[1]
try {
scalingMode = Visualizer.SCALING_MODE_NORMALIZED
} catch (e: NoSuchMethodError) {
println("CANT SET SCALING MODE.")
}
measurementMode = Visualizer.MEASUREMENT_MODE_NONE
setDataCaptureListener(object : Visualizer.OnDataCaptureListener {
override fun onFftDataCapture(visualizer: Visualizer?, fft: ByteArray?, samplingRate: Int) {
val n = fft?.size
val magnitudes = FloatArray(n!! / 2 + 1)
val phases = FloatArray(n / 2 + 1)
magnitudes[0] = Math.abs(fft[0].toFloat())
magnitudes[n / 2] = Math.abs(fft[1].toFloat())
phases[n / 2] = 0f
phases[0] = phases[n / 2]
for (k in 1 until n / 2) {
val i = k * 2
magnitudes[k] = Math.hypot(fft[i].toDouble(), fft[i + 1].toDouble()).toFloat()
phases[k] = Math.atan2(fft[i + 1].toDouble(), fft[i].toDouble()).toFloat()
}
println(magnitudes.joinToString(", "))
println(phases.joinToString(", "))
}
override fun onWaveFormDataCapture(visualizer: Visualizer?, waveform: ByteArray?, samplingRate: Int) {
val waveBuffer = mWaveBuffer ?: return
if (waveform == null || waveform.size != waveBuffer.size) {
return
}
System.arraycopy(waveform, 0, waveBuffer, 0, waveform.size)
}
}, Visualizer.getMaxCaptureRate(), true, true)
}.apply {
mDataCaptureSize = captureSize.apply {
mWaveBuffer = ByteArray(this)
mFftBuffer = ByteArray(this)
}
}
} catch (e: RuntimeException) {
println("ERROR DURING VISUALIZER INITIALIZATION: $e")
}
record_button.setOnClickListener {
if (!mAudioRecordState) {
mAudioRecord!!.startRecording()
visualizer?.enabled = true
mAudioRecordState = true
}
else {
mAudioRecord!!.stop()
visualizer?.enabled = false
mAudioRecordState = false
}
}
}
private fun ensurePermissionAllowed() {
if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
println("PERMISSION TO RECORD AUDIO DENIED. REQUESTING.")
ActivityCompat.requestPermissions(this, arrayOf(android.Manifest.permission.RECORD_AUDIO), REQUEST_CODE_AUDIO_PERMISSION)
}
else {
println("PERMISSION TO RECORD AUDIO GRANTED.")
}
}
override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<out String>, grantResults: IntArray) {
when (requestCode) {
REQUEST_CODE_AUDIO_PERMISSION -> {
if (grantResults[0] != PackageManager.PERMISSION_GRANTED) {
Toast.makeText(this, "Demo need record permission, please allow it to show this visualize effect!", Toast.LENGTH_LONG).show()
finish()
}
}
}
}
If you're using the Android Visualizer class, you don't need some other FFT library, because the Visualizer class provides a getFFT() method that returns to you audio that has already had an FFT applied to it. You just have to convert it to magnitudes on a dB scale to get it to look nice in your graphics.
You need to request microphone permission before you try to instantiate Visualizer or it will fail to initialize. Visualizer is largely implemented in C and is mostly a JNI wrapper, and as such throws lots of RuntimeExceptions if things go wrong, so you need to wrap your initialization and setup calls in try/catch blocks.
lateinit var magnitudesArray: FloatArray
//...
val captureSizeRange = Visualizer.getCaptureSizeRange()
try {
visualizer = Visualizer(0)
visualizer?.let {
scalingMode = Visualizer.SCALING_MODE_NORMALIZED
captureSize = captureSizeRange[1]
setDataCaptureListener(this, Visualizer.getMaxCaptureRate(), false, true)
enabled = true
}
} catch (e: RuntimeException) {
//..log it
}
magnitudesArray = FloatArray(captureSizeRange[1] / 2 + 1)
Then you can implement the data capture listener's onFftDataCapture function to transfer the incoming FFT to the magnitudesArray using the formulas given in the documentation. Apply a log10 function to these values to get magnitudes that are appropriate for visualizing (human hearing interprets relative sound energy on approximately log10 scale, which is why sound is usually described in dB units).
They typical capture size is 1024 or 2048, so you will have like 1024 magnitudes in your array. To get a nice spectrum bars visualization like in your picture, I find that it seems to look best if you just pick evenly spaced values out of your magnitudes array rather than trying to average ranges of them.
In regards to your comment about Java code...it simply works to add a Java library to your project and call it from Kotlin. You don't really "import" it. The Kotlin documentation has all the details.