Pitch Extraction

[trogluddite]

the sensor would recognize what note present in the audio instantly

From what I've gathered so far,(mostly just looking at the results from what I've so far been able to build), that is not really so easy.
The fastest method I've found to date is counting the zero crossings - certainly no use for anything polyphonic, but detection reliably takes one cycle of the input waveform.
FFT, Goertzel, and the examples I posted before, all essentially do the same thing - correlation with sine/cosine functions, and their precision is always a function of the size of the window. For sure, the peak of the response will be at the desired frequency, but another note close by and lower in volume could easily be masked if the window is too short.

The way I think of it is like detuning oscillators until you can hear that lovely slow beating as they drift in and out of phase - now imagine listening to a very short burst of that sound, just a few cycles.
How sure could you be that there are two frequencies and not just one? You might even catch the point of cancellation and hear almost nothing!
The beating between the notes is what gives you the extra information telling you that there are two tones very close together - and for that, you need to hear a longer section of the waveform.

If you have a look at the 'waterfall plot' schematic that I noted earlier, it has an input for changing the averaging time, and it makes a huge difference to the shape of the plot - a setting of one approximates to just a couple of cycles (if I got my maths right) and all you see then are big round fuzzy lumps; multiply the time by five, and you can start to pick out individual notes and their harmonics. At twenty you can discern notes within chords, but the display is looking decidedly sluggish by this point!
The correlations I've used and the Goertzel algorithm are pretty much just different forms of the same thing in principle, and the response I got with a Goertzel were very much the same. (I ditched it in the end, the need to 'bin' the results every N samples oddly made it less CPU efficient - SSE assembly makes arbitrary length loops a little tricky)

From what I've read about quality 'Gutar to MIDI' convertors, they attain a fast response by analysis of the picking transient of the string (a variation of physical modelling I assume) - but I never saw any detail in the public domain about the algorithms, and of course, it would be of no use for anything other than plucked strings.

[datsound]

You could do some very good polyphonic pitch extraction in real time IMHO.

I dont know anything about the FFT and the maths functions for this, so i did my silly pitch extraction program once based on simple modules, but it worked pretty well!!! (except for pitches under 200hz that are slower to detect of course)

Ok here is the magic idea: (drumroll)

Take a bandpass filter, arrange it with a very narrow range, sweep it in pitch range from C1 to C10 in a continuous sweep, so that every second it scans through the entire range and sends big peak information on the audio frequences that are present.... from my experience you can detect a regular instrument, such as guitar, piano, or whistle about 4 times a second from one bandpass filter, and you can draw a pretty accurate graph of the pitches being played.

so you can use 10 filters, and it is pretty computationally efficent.

Here is a photo of frequency detection using a sweeping bandpass filter: the sound is a high pitched sine wave. also cant you just use FFT?

i did a frequency detector for whistling anyway and it was vaguely accurate, enough to write some midi, and that was only with 2 BP filters to collect the data...

[floranicolas]

"Average Magnitude Difference Function" (AMDF) or the similar average squared difference function (sometimes called "ASDF") or autocorrelation. these methods make no assumptions regarding the signal other than that there is some degree of periodicity. they are mostly equivalent to each other in theory (the ASDF hits a minimum exactly where the autocorrelation hits a max).

[nix]

0 crossing doesn't go any faster than 2000 samples on MIDI note 28
@44,100 hz samplerate
I was wondering, hmm,
so could we take the most common value before 2000 samples?
Say if we take the first MIDI note that occurs 8 times-
reading the values at samplerate-
can we correlate our data like that to increase speed?
Cheers

[Chaoslala]

Don't know how much latency it could take but i experiment with the FFT display module.

Three note played and visible in the graph.
Try to extract the max value and his index of the array. It works for the first pitch.
I could extract the segond one but with some bug. trying to erase the first note from
the array.. Must continue later..

[trogluddite]

I was wondering, hmm,
so could we take the most common value before 2000 samples?

This depends a lot on the input waveform - it can be prone to octave errors.
For example, I have experimented a lot with tracking of guitar signals, where very often the first harmonic (2 * pitch) is dominant.
To some extent this can be made better by careful filtering of the input signal, or by multiplying the FFT values by weighting constants to shift the "focus" of the algorithm to lower frequencies.
Another method uses the fact that harmonic peaks must be separated by multiples of the root frequency. So you can look at several peaks in the FFT, and determine the common factor that separates them - though even this may be fooled by waves that contain e.g. only odd harmonics.

[nix]

Thanks Trog,
I loaded up a display and looked at the first values of the 0 crossing pitch detection method.
The same numeber recurs at a high multiple of times, as I was watching at samplerate-
so u r soon dealing with quite a lot of values.

I hope yous find this interesting, there is a sample of some playing at the end of the thread.
This is my home-made polyphonic synth guitar.

The hex pickup was made by ubertar, and Oddson wrote the counter->
http://www.kvraudio.com/forum/viewtopic.php?t=323713

[oddson]

Oddson wrote the counter->

I did? What counter?

[nix]

Yep,
It was my first SM project, apart from 3 osc VA.
Here's the thread:
viewtopic.php?f=12&t=5435&hilit=pitch+to+midi
You built the 0 crossing counter,
which I modded a little(the .osm was called 'zx').
I took out a stabilising variable.
I'm loving this iteration,
which doesn't trigger MIDI, just drives the osc freq input.
There is no noticeable latency.

[RunBeerRun2]

Wow the feel of this when used to control an internal synth (not by MIDI output) is very good! I've started an audio triggered synth for my bass guitar, not only does it track fast but very accurately.

[mccy]

Yes, the same here. Today I realized a thing that was in my head for a longer time now and ended up... yes, in a bass/guitar synth. These are the coolest bass sounds I've ever played.
There is this new (was it new, or did I just found it?) Surfer EQ, which does exactly what is possible with Martins pitc hrecognition and which was the second thing which came into my mind when I found it here.
It's just great to get that combined with synth sounds & original bass sound... WOW!!!! For now it takes way too much DSP, but is in no way optimized.
10000000000THANKS for this wonderfull modul!!!!!!!

[trogluddite]

For now it takes way too much DSP, but is in no way optimized

Strange that this should come up right now.
Philter prompted me to have another look at my old 'GRrr' pitch-follow synth. I've been trying to improve the tracking for the last couple of weeks (it was only a basic "zero crosser"); but all my attempts at other methods were no better. So I thought I would give up trying to stretch my head around the maths, and have a go at optimising this one - as you both say, nothing else can seem to manage to track a bass guitar as well as this does.
I have about half the code converted to assembly already - no optimisation yet, but I can see plenty of places that some CPU cycles can be saved. I'll post up anything that looks like progress.

Wonder what it is that attract so many bass players to this? Guess we must be more adventurous than our stick-in-the-mud short-necked brethren!

[oddson]

I've been trying to improve the tracking for the last couple of weeks (it was only a basic "zero crosser"); but all my attempts at other methods were no better.

...
There is no noticeable latency.

Yeah... it tracks OK (although prone to octave error) but it doesn't matter if it's off base (bass?) in the first milliseconds as your brain takes longer to track the pitch than it does. Low frequency fundamentals have a longer theoretical limit for pitch detection but I think the brain v. machine match-up is more favourable at low frequency.

[mccy]

I finally decided to post 'my' basssynth (too simple to call it a synth somehow, but I love it) in the registrated users forum. It's not really my work, I just took, what I found in this place and combined it for a good idea. All the real coding others did. Anyway, I don't want to see thisone get a free vst for now, but I wanted to thank this GREAT community and not allways hide away with my output.

I would love to see a cooperation to make this thing (maybe nobody will need it anyway, when trog will release his new Grrrr) THE basssynth. I love the attempt of the hamonics dialing EQs & next simple step will be to make all EQs working with Attack & release for cool Wah sounds!

1000 thanks to dozius, exo & martinvicanek - hope I did not forget anyone!!!!

[nix]

I've been trying to improve the tracking for the last couple of weeks (it was only a basic "zero crosser"); but all my attempts at other methods were no better.

...
There is no noticeable latency.

Yeah... it tracks OK (although prone to octave error) but it doesn't matter if it's off base (bass?) in the first milliseconds as your brain takes longer to track the pitch than it does. Low frequency fundamentals have a longer theoretical limit for pitch detection but I think the brain v. machine match-up is more favourable at low frequency.

The bass pitches are noticeably laggy for MIDI.
It is 1600 samples for high e,
and 2600 for low e(@44100).
The mind registers this.
With the direct freq conversion it's sweet,
just like playing guitar.
I'm getting it better all the time.
Will post up some demos soon.
BTW, I am doing it polyphonically,
as I have 2 guitars with hex pickups.
I have heaps of smoothing tricks in there,
can share the .osm with you oddson, if u would like it.