beatles_fprint - fingerprint-based alignment of labels to Beatles audio
This package provides a set of routines and a precomputed database that will automatically modify the Beatles annotations available from http://isophonics.net/ to local versions of the relevant audio.
Because of variability in the mastering process, classic albums that have multiple digital masterings will end up with multiple, slightly different, digital versions. Not only may there be different silent gaps at the beginning of each track, but even the timing within the track will differ (thanks, e.g., to imperfections in the tape running speed, or stretching of the master tapes). In our experience, differences of 0.1% or larger are quite common; this is entirely imperceptible, but labels made on the basis of one version of the track will be off by entire beats by the end of a typical track of several hundred seconds. Hence the need for a tool that estimates these deviations, and modifies the original label files to match the locally-available audio.
The routines here run an audio fingerprint on any input track, then identify it within the 180 track Beatles cannon (in the provided database). The fingerprint is robust to small timing changes, however, by looking at the trend of timing matches throughout the track, we can estimate a timing offset and scaling that maximizes agreement between the input audio and the reference landmarks.
These values are reported. They can also be applied to label files that were based on the reference audio to construct a modified label file whose labels are the same, but whose timings are matched to the local audio.
Contents
Example usage
Passing a waveform as a vector plus a sampling rate will perform the fingerprint matching and timing estimation. A third argument set to one causes it to plot the scatter of matching landmarks, highlighting the linear fit used to estimate the slope.
[d,sr] = wavread('come_together.wav'); doplot = 1; [T,O,S] = find_beatles_timeskew(d,sr,doplot); % T returns the track name; O returns the offset (in sec), and S % returns the timing slope (ratio) % Zoom in on the matching part of the matching fingerprint scatter % plot to show the slight drift in time difference axis([0 260 0.5 1.5])
Track <input data> matched as The Beatles/11_-_Abbey_Road/01_-_Come_Together Best match for time T sec in reference track is -0.92948+0.99891*T in input track
Modifying Label Files
You can rewrite label files to be modified using the parameters returned by find_beatles_timeskew. Here, I have separately installed the isophonics beatles data in the directory beatles/ .
ref_labeldir = 'annotations/chordlab'; dst_labeldir = 'scratch'; skew_labelfile(T, ref_labeldir, dst_labeldir, O, S); % This reads a label file with the stem T from the ref_labeldir, % and writes a modified version with the same name to dst_labeldir.
wrote scratch/The Beatles/11_-_Abbey_Road/01_-_Come_Together.lab
Bulk usage
Passing a filename will read an audio file (using my audioread package, available at http://www.ee.columbia.edu/~dpwe/resources/matlab/audioread/ ):
[T,O,S] = find_beatles_timeskew('come_together.wav'); % Passing in a cell array of filenames will return a cell array of % matched track names, and vectors of offsets and slopes. TT = myls('beatles/mp3s-32k/Abbey_Road/*.mp3'); [T,O,S] = find_beatles_timeskew(TT); % Then you can rewrite the label files in a single stroke % by passing a cell array of names, and vectors of offsets and slopes skew_labelfile(T, ref_labeldir, dst_labeldir, O, S);
Track come_together.wav matched as The Beatles/11_-_Abbey_Road/01_-_Come_Together Best match for time T sec in reference track is -0.92950+0.99891*T in input track Track beatles/mp3s-32k/Abbey_Road/01-Come_Together.mp3 matched as The Beatles/11_-_Abbey_Road/01_-_Come_Together Best match for time T sec in reference track is 1.54713+1.00031*T in input track Track beatles/mp3s-32k/Abbey_Road/02-Something.mp3 matched as The Beatles/11_-_Abbey_Road/02_-_Something Best match for time T sec in reference track is 0.89583+0.99973*T in input track Track beatles/mp3s-32k/Abbey_Road/03-Maxwell_s_Silver_Hammer.mp3 matched as The Beatles/11_-_Abbey_Road/03_-_Maxwell's_Silver_Hammer Best match for time T sec in reference track is 0.87203+1.00020*T in input track Track beatles/mp3s-32k/Abbey_Road/04-Oh_Darling.mp3 matched as The Beatles/11_-_Abbey_Road/04_-_Oh!_Darling Best match for time T sec in reference track is 0.69478+1.00019*T in input track Track beatles/mp3s-32k/Abbey_Road/05-Octopus_s_Garden.mp3 matched as The Beatles/11_-_Abbey_Road/05_-_Octopus's_Garden Best match for time T sec in reference track is 0.38705+0.99981*T in input track Track beatles/mp3s-32k/Abbey_Road/06-I_Want_You_She_s_So_Heavy_.mp3 matched as The Beatles/11_-_Abbey_Road/06_-_I_Want_You Best match for time T sec in reference track is 0.95725+0.99991*T in input track Track beatles/mp3s-32k/Abbey_Road/07-Here_Comes_The_Sun.mp3 matched as The Beatles/11_-_Abbey_Road/07_-_Here_Comes_The_Sun Best match for time T sec in reference track is 1.14190+0.99974*T in input track Track beatles/mp3s-32k/Abbey_Road/08-Because.mp3 matched as The Beatles/11_-_Abbey_Road/08_-_Because Best match for time T sec in reference track is 0.51249+1.00011*T in input track Track beatles/mp3s-32k/Abbey_Road/09-You_Never_Give_Me_Your_Money.mp3 matched as The Beatles/11_-_Abbey_Road/09_-_You_Never_Give_Me_Your_Money Best match for time T sec in reference track is 1.14550+1.00031*T in input track Track beatles/mp3s-32k/Abbey_Road/10-Sun_King.mp3 matched as The Beatles/11_-_Abbey_Road/10_-_Sun_King Best match for time T sec in reference track is 5.63256+1.00015*T in input track Track beatles/mp3s-32k/Abbey_Road/11-Mean_Mr_Mustard.mp3 matched as The Beatles/11_-_Abbey_Road/11_-_Mean_Mr_Mustard Best match for time T sec in reference track is 0.53458+1.00024*T in input track Track beatles/mp3s-32k/Abbey_Road/12-Polythene_Pam.mp3 matched as The Beatles/11_-_Abbey_Road/12_-_Polythene_Pam Best match for time T sec in reference track is 0.29665+1.00028*T in input track Track beatles/mp3s-32k/Abbey_Road/13-She_Came_In_Through_The_Bathroom_Window.mp3 matched as The Beatles/11_-_Abbey_Road/13_-_She_Came_In_Through_The_Bathroom_Window Best match for time T sec in reference track is 24.76955+1.00030*T in input track Track beatles/mp3s-32k/Abbey_Road/14-Golden_Slumbers.mp3 matched as The Beatles/11_-_Abbey_Road/14_-_Golden_Slumbers Best match for time T sec in reference track is 0.55578+0.99980*T in input track Track beatles/mp3s-32k/Abbey_Road/15-Carry_That_Weight.mp3 matched as The Beatles/11_-_Abbey_Road/15_-_Carry_That_Weight Best match for time T sec in reference track is 0.56222+0.99995*T in input track Track beatles/mp3s-32k/Abbey_Road/16-The_End.mp3 matched as The Beatles/11_-_Abbey_Road/16_-_The_End Best match for time T sec in reference track is 1.01142+1.00035*T in input track Track beatles/mp3s-32k/Abbey_Road/17-Her_Majesty.mp3 matched as The Beatles/11_-_Abbey_Road/17_-_Her_Majesty Best match for time T sec in reference track is 0.98169+1.00033*T in input track wrote scratch/The Beatles/11_-_Abbey_Road/01_-_Come_Together.lab wrote scratch/The Beatles/11_-_Abbey_Road/02_-_Something.lab wrote scratch/The Beatles/11_-_Abbey_Road/03_-_Maxwell's_Silver_Hammer.lab wrote scratch/The Beatles/11_-_Abbey_Road/04_-_Oh!_Darling.lab wrote scratch/The Beatles/11_-_Abbey_Road/05_-_Octopus's_Garden.lab wrote scratch/The Beatles/11_-_Abbey_Road/06_-_I_Want_You.lab wrote scratch/The Beatles/11_-_Abbey_Road/07_-_Here_Comes_The_Sun.lab wrote scratch/The Beatles/11_-_Abbey_Road/08_-_Because.lab wrote scratch/The Beatles/11_-_Abbey_Road/09_-_You_Never_Give_Me_Your_Money.lab wrote scratch/The Beatles/11_-_Abbey_Road/10_-_Sun_King.lab wrote scratch/The Beatles/11_-_Abbey_Road/11_-_Mean_Mr_Mustard.lab wrote scratch/The Beatles/11_-_Abbey_Road/12_-_Polythene_Pam.lab wrote scratch/The Beatles/11_-_Abbey_Road/13_-_She_Came_In_Through_The_Bathroom_Window.lab wrote scratch/The Beatles/11_-_Abbey_Road/14_-_Golden_Slumbers.lab wrote scratch/The Beatles/11_-_Abbey_Road/15_-_Carry_That_Weight.lab wrote scratch/The Beatles/11_-_Abbey_Road/16_-_The_End.lab wrote scratch/The Beatles/11_-_Abbey_Road/17_-_Her_Majesty.lab
Working with beat files
Colin Raffel modified this code to work with the beat annotation files instead of the chord annotations. The changes were all in skew_labelfile, and I include his modified version as skew_beatfile. You should be able to run this directly on directories of beat annotations from isophonics.net to adjust their timings to match your audio.
Thanks to Colin for sharing these changes.
Installation
You can download a zip file containing the Matlab scripts and the hash table database from beatles_fprint.zip.
This includes the script rebuild_ref which can be modified and used to rebuild the reference hash table for new audio (or for any other purpose).
% Last updated: $Date: 2011/09/20 20:53:35 $ % Dan Ellis <dpwe@ee.columbia.edu>