Sparse Interpretible Audio Model

Table of Contents

• Architecture
• Sound Samples

Model Architecture

This small model attempts to decompose audio featuring acoustic instruments into the following components:

• Some maximum number of small (16-dimensional) event vectors, representing individual audio events
• Times at which each event occurs

Each sound sample below includes the following elements:

1. The original recording
2. The model's reconstruction
3. New audio using the original timings, but random event vectors
4. New audio using the original event vectors, but with random timings

Future Directions

There are several areas that could provide further gains in compression and interpretibility:

• Imposing more severe sparsity constraints on the number of events produced. You may notice that there are often many redundant events that could be merged into one.
• Performing vector quantization on the event vectors such that there is a discrete set of possible events

Cite this Work

                    
@misc{vinyard2023audio,
    author = {Vinyard, John},
    title = {Sparse Interpetable Audio},
    url = {https://JohnVinyard.github.io/machine-learning/2023/11/15/sparse-physical-model.html},
    year = 2024
}