Dan Ellis : Resources: Matlab:

## Beat Tracking by Dynamic Programming

This page illustrates the use of the beat_* functions to implement a simple music audio beat tracker based on dynamic programming, as described in: D. Ellis "Beat Tracking by Dynamic Programming" J. New Music Research 36(1): 51-60, March 2007, DOI: 10.1080/09298210701653344.

```% First, load an example sound file
wavfilename = 'train01.wav';

% Now, calculate the "onset strength" function as the sum of the
% differentiated and half-rectified energy in a Mel-scale
% time-frequency transform:
[onset_fn, osr, sgram, tt, ff] = beat_onset(d,sr);
% osr returns the sampling rate of the frames of onset_fn
% sgram returns the mel spectrogram as an array, with tt and ff
% being the time and frequency labels.

% The dynamic programming approach needs an estimate of global
% tempo, so we calculate one by autocorrelating the onset function,
% applying a bias window, and choosing the biggest peak.
% Optionally, it plots the windowed autocorrelation for us
subplot(211)
display = 1;
tempo = beat_tempo(onset_fn, osr, display);

% Now we can run the dynamic programming beat tracker
beats = beat_simple(onset_fn, osr, tempo);

% We have some helper functions: one to plot the beat times on top
% of the Mel spectrogram (which we saved from beat_onset)
subplot(212)
beat_plot(beats, '-r', tt, ff, sgram);

% And we can listen to the result; the system-found beats are
% marked by little bursts of white noise superimposed on the
% original:
beat_play(beats, d, sr);

% We can go straight from soundfile to beats with beat_track, which
% just provides a wrapper around the steps above:
beats = beat_track(wavfilename);
```
```Warning: The playback thread did not start within one second.
```

## Ground truth

We can also read in ground truth for the mirex06 McKinney/Moelants tapping data:

```tapfilename = 'train01.txt';
truth = beat_ground_truth(tapfilename);
% By default this only returns the subset of tap records that are
% consistentn with the most popular tempo.  Multiple sequences are
% returned in separate cells of a cell array; we can plot and
% listen to them too:
beat_plot(truth{1}, 'xb');
beat_play(truth{1}, d(1:10*sr), sr);

% We can also plot all the individual ground-truth taps in a
% "scatter" format:
subplot(211)
beat_gt_plot(truth)
```

## Scoring results

We can score a beat track against a ground truth by counting how many true beats are missed (deletions), and how many system-generated beats don't correspond to true beats (insertions):

```collar = 0.2; % accept a beat within +/-20% of the tempo period
verbose = 1;
[err,ins,del,tru,hh,dd] = beat_score(beats,truth{1},collar,verbose);
% In this case, only the first beat is 'wrong', because the human
% was late to pick up the beat.
% We can score against all the ..
length(truth)
% .. 35 different tap records for this tempo by passing them all to
% the scoring function:
[err,ins,del,tru,hh,dd] = beat_score(beats,truth,collar,verbose);
% We don't line up with all the human taps, but then no single beat
% track ever could because the humans have too much spread.
```
```Overall error=   3.2% (   1 ins,    1 del,   63 true)

ans =

40

Overall error=  25.1% ( 330 ins,   89 del, 2279 true)
```

## Testing against a set of examples

We can evaluate the beat tracker against a whole set of examples:

```dirname = 'mirex06examples';
files = dir(fullfile(dirname,'*.wav'));
for i = 1:length(files); fnames{i} = fullfile(dirname,files(i).name); end
beat_test(fnames);
% Overall average error rate is high, but it's highly variable
% across examples.
```
```Error for train01 (tempo est=129.2, users=129.3 BPM, 35 true tracks): Overall error=  11.6% ( 159 ins,   84 del, 2130 true)
Error for train02 (tempo est=168.1, users= 83.9 BPM, 26 true tracks): Overall error= 113.3% (1129 ins,   13 del, 1016 true)
Error for train03 (tempo est=101.8, users=153.7 BPM, 26 true tracks): Overall error= 110.1% ( 740 ins, 1291 del, 1851 true)
Error for train04 (tempo est=127.6, users= 42.0 BPM, 28 true tracks): Overall error= 223.6% (1179 ins,    4 del,  538 true)
Error for train05 (tempo est= 68.4, users= 68.5 BPM, 34 true tracks): Overall error=  11.0% (  71 ins,   36 del, 1087 true)
Error for train06 (tempo est=164.1, users= 82.0 BPM, 25 true tracks): Overall error= 164.6% (1128 ins,   22 del,  920 true)
Error for train07 (tempo est=103.4, users= 56.5 BPM, 34 true tracks): Overall error= 143.4% (1035 ins,  214 del,  879 true)
Error for train08 (tempo est=147.7, users=147.8 BPM, 26 true tracks): Overall error=  21.5% ( 238 ins,  155 del, 1815 true)
Error for train09 (tempo est=128.4, users=127.7 BPM, 25 true tracks): Overall error=  17.0% ( 165 ins,   88 del, 1498 true)
Error for train10 (tempo est= 61.2, users= 61.2 BPM, 20 true tracks): Overall error=  10.3% (  41 ins,   15 del,  574 true)
Error for train11 (tempo est=139.2, users=139.9 BPM, 26 true tracks): Overall error=  20.0% ( 202 ins,  133 del, 1725 true)
Error for train12 (tempo est=122.0, users= 54.1 BPM, 27 true tracks): Overall error= 161.4% ( 999 ins,   81 del,  675 true)
Error for train13 (tempo est=119.5, users=181.3 BPM, 30 true tracks): Overall error= 112.3% (1039 ins, 1810 del, 2541 true)
Error for train14 (tempo est=130.0, users=130.7 BPM, 29 true tracks): Overall error=  12.2% ( 126 ins,   94 del, 1824 true)
Error for train15 (tempo est=185.4, users= 62.0 BPM, 28 true tracks): Overall error= 207.2% (1709 ins,    4 del,  831 true)
Error for train16 (tempo est=182.9, users= 90.7 BPM, 20 true tracks): Overall error= 131.3% (1022 ins,   28 del,  806 true)
Error for train17 (tempo est= 93.1, users= 46.1 BPM, 24 true tracks): Overall error= 127.0% ( 611 ins,   66 del,  535 true)
Error for train18 (tempo est=129.6, users= 61.2 BPM, 34 true tracks): Overall error= 151.1% (1320 ins,  177 del,  996 true)
Error for train19 (tempo est= 93.5, users=188.1 BPM, 34 true tracks): Overall error=  63.5% ( 221 ins, 1730 del, 3073 true)
Error for train20 (tempo est=110.0, users=219.8 BPM, 38 true tracks): Overall error=  76.9% ( 560 ins, 2551 del, 4043 true)
Overall average error: 94.5% (13694 ins,  8596 del, 29357 true)
```

## Acknowledgements

This material was developed for my course ELEN E4896 Music Signal Processing, under partial support from the NSF under project IIS-1117015.

2012-03-28 Dan Ellis dpwe@ee.columbia.edu