MEAPsoft-2.0 This is a basic implementation of the MEAPsoft framework that we described on 2006-02-27. It contains a Java implementation of an onset detector, a feature extractor, a synthesizer based on Mike's mashup generator, as well as a number of different composers. In addition, a sample perl implementation of a composer that just sorts a column of feature file is given in the examples directory. All of these tools operate on audio files and two simple text file formats. The basic idea is to use the Segmenter to segment some audio files, and use FeatExtractor to extract some feature description of these chunks. Then use one of the composer tools to operate on these chunks (reorder them somehow) and generate an edit decision list which Synthesizer then uses to synthesize some audio. There is more information on the MEAP wiki: http://works.music.columbia.edu/MEAP/MeapSoft and at the MEAPsoft web page: http://labrosa.ee.columbia.edu/meapsoft Documentation for the MEAPsoft GUI can be found in doc/manual.html. A quickstart guide can be found in doc/quickstart.html. * Building: MEAPsoft comes with a pre-built MEAPsoft.jar, so building from source is not necessary to run the application. However, if you want to, you can build MEAPsoft from scratch. Make sure you have java and make in your path. Open a terminal, change into the src directory and type "make". You'll see some diagnostic text output, and if the build is successful it will create a MEAPsoft.jar file in ./bin * Running: ** Graphical interface: Run "java -jar /path/to/MEAPsoft.jar" to bring up a GUI interface. Note that these programs like eating up memory so you might want to pass an option like -mx300m to java. MEAPsoft.jar is in the /bin directory. Alternatively you can use the appropriate runme script in the /bin directory. Open a terminal, change into the /bin directory, and type "runme.bat" (Windows) or "runme.sh" (OSX/Linux) and the GUI application should launch. ** Command line apps: You can also run parts of MEAPsoft from the command line. Just run $ java -cp /path/to/MEAPsoft.jar com.meapsoft.(Segmenter|FeatExtractor|Synthesizer|composers.ComposerName) options Note that the jar files in the bin directory also need to be in your classpath in order for the above command line to work. Again, these programs like eating up memory (FeatExtractor especially), so you might want to pass an option like -mx300m to java. 1. com.meapsoft.Segmenter - Segments an audio file and creates a list of chunks in the form of a MEAP feature file. Usage: Segmenter [-options] source1.wav source2.mp3 ... where options include: where options include: -o output_file the file to write the output to (defaults to ./out.segments) -m N [0] use mixer N for input -t D.D [1.0] onset detector threshold (event detector), tempo multiplier (beat detector) -s D.D [0.1] length of smoothing window in seconds for event detector -d use the old style onset detector (defaults to BeatOnsetDetector) -0 add an onset at the very beginning of the file 2. com.meapsoft.FeatExtractor - extracts features from the segments listed in some feature files. Usage: FeatExtractor [-options] file1.feat file2.feat ... where options include: -o output_file append features into output file (defaults to input file) -f feat_name use feat_name features (defaults to AvgMelSpec, can have many -f arguments) Supported feat_names are: AvgChroma AvgChromaScalar AvgChunkPower AvgFreqSimple AvgMelSpec AvgMFCC AvgPitchSimple AvgSpec AvgSpecCentroid AvgSpecFlatness ChunkLength ChunkStartTime Likelihood SpectralStability 3. com.meapsoft.Synthesizer - synthesizes a wav from from an EDL file. Usage: Synthesizer [-options] file.edl where options include: -o output_file output sound file (defaults to line out) 4. com.meapsoft.composers.MashupComposer - try to recreate a song using the closest chunks from chunkdb.feat Usage: MashupComposer [-options] dest.feat chunkdb.feat where options include: -o output_file the file to write the output to (defaults to mashup.edl) -i feat_dim what feature dimentions to use (defaults to all) where feat_dim is a comma separated list (no spaces) of integer indices and ranges (e.g. 1-5,7:9,11) -d dist_metric distance metric to use (defaults to EuclideanDist, can string them together with many -d arguments) Supported distance metrics are: EuclideanDist CosineDist -c command apply command to all chunks in the EDL output file Supported commands include: reverse, crossfade(time), overlap(time) (time in seconds) 5. com.meapsoft.composers.SortComposer - java implementation of sort_composer.pl, can sort across multiple feature dimensions Usage: SortComposer [-options] features.feat where options include: -o output_file the file to write the output to (defaults to sorted.edl) -g debug mode (prints out chunk features on each line of output file) -r sort in reverse order -i feat_dim what feature dimentions to use (defaults to all) where feat_dim is a comma separated list (no spaces) of integer indices and ranges (e.g. 1-5,7:9,11) -d dist_metric distance metric to use (defaults to EuclideanDist, can string them together with many -d arguments) -c command apply command to all chunks in the EDL output file Supported commands include: reverse, crossfade(time), overlap(time) (time in seconds) Supported distance metrics are: EuclideanDist CosineDist -c command apply command to all chunks in the EDL output file Supported commands include: reverse, crossfade(time), overlap(time) (time in seconds), gain(x). 6. com.meapsoft.composers.NNComposer - reorder a song's chunks by hopping from each chunk to its nearest neighbor. Usage: NNComposer [-options] features.feat where options include: -o output_file the file to write the output to (defaults to sorted.edl) -g debug mode (prints out chunk features on each line of output file) -i feat_dim what feature dimentions to use (defaults to all) where feat_dim is a comma separated list (no spaces) of integer indices and ranges (e.g. 1-5,7:9,11) -d dist_metric distance metric to use (defaults to EuclideanDist, can string them together with many -d arguments) Supported distance metrics are: EuclideanDist CosineDist -c command apply command to all chunks in the EDL output file Supported commands include: reverse, crossfade(time), overlap(time) (time in seconds) 6. com.meapsoft.composers.BlipComposer - adds a blip after each chunk Usage: BlipComposer [-options] features.feat where options include: -o output_file the file to write the output to (defaults to blipped.edl) -f blip_file the audio file to insert at the beginning of each chunk (defaults to data/blip.wav) -c command apply command to all chunks in the EDL output file Supported commands include: reverse, crossfade(time), overlap(time) (time in seconds) 7. There are loads more composers in the com.meapsoft.composers package... 8. Another composer can be found in ./examples/sort_composer.pl - sort over a single feature dimension Usage: perl sort_composer.pl options feat_file edl_file command where options include: -d sort_dim column of the feature file to sort over (defaults to 3 - first feature dimension) -r 1 to sort in reverse order 9. DataDisplayPanel can be used to visualize the features contained in a feature file Usage: DataDisplayPanel [-options] features.feat where options include: -i feat_dim what feature dimentions to use (defaults to all) where feat_dim is a comma separated list (no spaces) of integer indices and ranges (e.g. 1-5,7:9,11) * File Formats: There are two basic file formats. Comments are just like your favorite scripting language : # everything to the right of the hash is a comment 1. Feature file format (output by Segmenter and FeatExtracter, input to composer) Every line is formatted as follows (features are optional, but the parser expects there to be the same number of features on every line in a given file): /path/to/file start_time(seconds) chunk_length(seconds) feat1 feat2 ... e.g. # filename onset_time chunk_length [features] # Features: AvgChunkPower(1) stairway.wav 1.486077070236206 0.12190476059913635 3.1384155179764343E-6 stairway.wav 1.60798180103302 0.19736962020397186 2.71758525671205E-6 stairway.wav 1.8053514957427979 0.1915646195411682 2.916578754218279E-5 stairway.wav 1.9969160556793213 0.49342402815818787 7.191902857682654E-6 ... 2. Edit decision list format (output by composer, input to Synthesizer) Basic line format (commands are optional): dest_time(seconds) /path/to/file start_time(sec) chunk_length(sec) cmd1 ... e.g.: 0.121904760599136 stairway.wav 1.60798180103302 0.19736962020397186 reverse 0.319274380803108 stairway.wav 1.8053514957427979 0.1915646195411682 reverse 0.510839000344276 stairway.wav 1.9969160556793213 0.49342402815818787 reverse ... The Synthesizer currently supports the following commands: - crossfade(time) - adds a fade to both ends of the chunk and overlaps it with the previous one. - fade(fadein_time, fadeout_time) - fades the chunk in and out with the given fade times. - gain(A) - scales the chunk waveform by A. - reverse - reverses the chunk in time. * Example: See /examples/reorder_stairway.sh and examples/make_psycho_stairway.sh for some code to make some simple compositions. * More Information: Code documentation is available here: Doxygen: http://www.ee.columbia.edu/~ronw/code/MEAPsoft/doc/doxygen/ Javadoc: http://www.ee.columbia.edu/~ronw/code/MEAPsoft/doc/javadoc/ I'm going to maintain this in a Darcs (http://www.darcs.net) repository for now. Once you have Darcs installed (oh yes, it is even available in fink), just run: $ darcs get http://www.ee.columbia.edu/~ronw/code/MEAPsoft to get a local copy of the repository in the current directory. Edit the files as you please. Run "darcs record" to save any changes to your local repository. If you have something working that you want to commit to the main repository, just run "darcs send" to email me the patch. * Licensing: This software is distributed under the terms of the GNU General Public License version 2. See the file "COPYING" for details. MEAPsoft depends on the Tritonus (http://www.tritonus.org) implementation of the JavaSound API for audio support and on GNU GetOpt for command line argument parsing. These libraries can be found in the bin directory and are both distributed under the GNU LGPL license. Some portions of MEAPsoft (specifically SpectrogramPanel.java and the contents of the util package) have their own licenses. These licenses are included below. ** License for SpectrogramPanel.java: Copyright 1999-2004 Carnegie Mellon University. Portions Copyright 2002-2004 Sun Microsystems, Inc. Portions Copyright 2002-2004 Mitsubishi Electric Research Laboratories. All Rights Reserved. Use is subject to license terms. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Original authors' names are not deleted. 4. The authors' names are not used to endorse or promote products derived from this software without specific prior written permission. This work was supported in part by funding from the Defense Advanced Research Projects Agency and the National Science Foundation of the United States of America, the CMU Sphinx Speech Consortium, and Sun Microsystems, Inc. CARNEGIE MELLON UNIVERSITY, SUN MICROSYSTEMS, INC., MITSUBISHI ELECTRONIC RESEARCH LABORATORIES AND THE CONTRIBUTORS TO THIS WORK DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY, SUN MICROSYSTEMS, INC., MITSUBISHI ELECTRONIC RESEARCH LABORATORIES NOR THE CONTRIBUTORS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.