"Deformable Spectrograms"
Manuel Reyes-Gomez           Nebojsa Jojic DanielP.W.Ellis
ColumbiaUniversity        Microsoft Research ColumbiaUniversity

VI.- APPLICATIONS.
     VIDEO 8, VIDEO 9VIDEO 2, VIDEO 3VIDEO 5VIDEO 6.

Formants and Harmonics Tracking.
Analyzing a signal with the two-layer model permits separate tracking of the harmonic and formant of
any given point in the spectrogram.  The user clicks on the spectrogram to select a bin and the system
reveals the harmonics and formant "history" for that bin.  Figure 12 b) shows an example of harmonics
tracking for the bin chosen in part a); c) shows an example of formant tracking for the bin chaosen in
part a).


                           a) Chosen Bin                                                                           b) Harmonics tracking                                                             c) Formants tracking.
                                                                                                                                      Figure 12

Watch another example on the following video.

Description of Video 8.
The video, displays information in five panels. Panel 1 displays the spectrogram of the signal. Panels
2 and 3 show the harmonics and formants layers.Panels 4 and 5 display the harmonics and formants
transformation maps. The user can "click" anywhere on the spectrogram and then track the
harmonics or formants "history" by selecting the harmonics tracking button or the formants tracking
button. The tracking results are displayed on panel 1 and in the correspondent panel for the component
being tracked.

VIDEO 8.- Formants and Harmonics Tracking.

CLICK ON THE SCREEN TO ACTIVE THE VIDEO !

Two Speakers Semi-Supervised Source Separation.
After modeling the input signal, the user "clicks" on those regions of the spectrogram that he believes
belong to the speaker that is intended to be eliminated. The system then masks all neighboring
time-frequency bins with the same prominent transformation as the one of the selected bin, the bins
remaining in the spectrogram belong to the other speaker. Figure 13, depicts an instance of this
application. Part a) shows the user "clicks" on the spectrogram, b) shows where the "clicks" map
into the harmonics transformation map, and part c) shows the resultant mask.

CLICK ON FIRST AND THIRD PANES TO HEAR THE SIGNALS !

                                       a)                                                                                                      b)                                                                                             c)
                                                                                                                          Figure 13

Description of Video 9.
This video shows the how the demo generates the example in figure 13.

VIDEO 9.- Two Speakers Semi-Supervised Source Separation.

CLICK ON THE SCREEN TO ACTIVE THE VIDEO !

 

Missing data interpolation and Harmonics/Formants Separation.

Check VIDEO 2, VIDEO 3, VIDEO 5, and VIDEO 6  for examples of these applications.

BACK TO INDEX
PREVIOUS
NEXT