Recurrent Neural Network Instrument

About

First, we overfit a single-layer RNN, factoring a short segment of classical music into two components:

• a sparse "control-signal"
• a recurrent network that models the resonances of the instrument(s) being played.

Then we map MediaPipe hand-tracking landmarks into the RNN's input space so that the learned network can be "played" via hand-movement.

Training details can be explored further here