HTTPS SSH
RBM Drum Sequencer
Author: Jennifer Hsu (jsh008@ucsd.edu)

The RBM Drum Sequencer is a MIDI-based Drum Sequencer built as a Pd patch with the ability to generate rhythmic pattern variations of a 3-instrument, input pattern.  The rhythmic variations are generated using a restricted Boltzmann machine (RBM) trained over a set of rhythmic patterns.  The rhythmic patterns used for the training are from here: https://github.com/ebattenberg/crbm-drum-patterns.  The accompanying paper to that dataset by Eric Battenberg and David Wessel, "Analyzing Drum Patterns using a Conditional Deep Belief Networks" (ISMIR 2012) provided inspiration for this project.

-- Editing the absolute paths in rbmMIDI.c --
For some reason, my Pd extern code needs absolute paths for the trained RBM weights.  You will need to change the code in rbmMIDI.c for lines 176, 201, and 484 to reflect the absolute path for weights.txt and weights2.txt.  These files are included in the rbmDrumSequencer directory, so point the path to this directory in lines 176, 201, and 484.


-- Compiling the Pd extern rbmMIDI --
1. Take the pd_extern/rbmMIDI.c file and move it into the Applications/Pd-XXX/Contents/Resources/doc/6.externs/ directory.  The XXX should be replaced with whatever Pd distribution you are working with. In my case, it would be Pd-extended.

2. Modify the makefile to include rbmMIDI.c in the compiling process.  For my Mac, I scroll down to the line that says "pd_darwin: obj1.pd_darwin obj2.pd_darwin 
     obj3.pd_darwin obj4.pd_darwin obj5.pd_darwin dspobj~.pd_darwin" and I add rbmMIDI.pd_darwin to the list.

3. Navigate to the Applications/Pd-XXX/Contents/Resources/doc/6.externs/ directory in your terminal and compile.  I type in "make pd_darwin" in my terminal for this step and push return. 

4. Copy the compiled object (rbmMIDI.pd_darwin) into Applications/Pd-XXX/Contents/Resources/extra.

5. Test that the extern works by opening up your copy of Pd and opening the pd_extern/rbmMIDI_example.pd patch.  If the rbmMIDI extern was properly compiled and loaded, you should be able to see the rbmMIDI object in a solid object box. You should be able to send messages to the object and click the bang button with no errors.  If not, please do a search on how to compile Pd externs before moving on.


-- Running the Pd patch --
If your rbmMIDI Pd extern is working properly, you can open rbmDrumSequencer.pd patch in your Pd distribution. Here is a description of how to get it all running:

1. You need to provide your own samples.  Create a samples/ directory within the rbmDrumSequencer directory.  Please choose 3 .wav files and move them into the samples/ directory.

2. Open Pd and open the rbmDrumSequencer.pd patch.

3. In the upper left hand part of the patch labeled with POLYPHONIC SAMPLERS, click the "pd players" subpatch. This will open up a new window with the subpatch.

4. There are 3 object boxes that look like [samplebank samples/xxx.wav]. Please replace each of the xxx with the names of the samples that you placed in the samples/ directory.

5. After you change the sample file in each object box, click on that [samplebank samples/xxx.wav] object.  This should open up a new patch called samplebank.pd.  Click the bang in the upper left hand corner to load the sample. Repeat this for all 3 samplebank objects.  If you save the players subpatch in the current state, you will not need to repeat this step each time you start up the drum machine patch. Feel free to close all subpatches when you complete this step.

6. Turn on the DSP processing in Pd. Make sure audio is working in Pd.

7. Click the DEFAULT SEQUENCE message box on the left side of the patch.  You should see the three arrays in the left panel above that message box (light gray) change to reflect the number sequence in the message box. This is the set of arrays that you will hear playing and are referred to as "user arrays."

8. Turn up the output level slider at the top left part of the patch in the POLYPHONIC SAMPLERS section.

9. Click the black toggle button that is labeled with CLICK HERE TO BEGIN PLAYING.

10. Generate new rhythmic patterns for each sample by clicking on the bang button on the upper, right panel labeled GENERATE A NEW PATTERN. The three arrays in the right array panel (dark gray) should change.  This is the set of arrays that holds the generated patterns, referred to as "generated arrays." This section takes the information from the user arrays and inputs it into the RBM pd extern rbmMIDI.  rbmMIDI then generates a new pattern and outputs it into the generated arrays. By defuault, the trained RBM used for generating variations is trained on Drum and Bass rhythms.  I have provided another set of weights that the user may switch to in real-time.  In the GENERATE A NEW PATTERN panel, you can switch between an RBM trained on Drum n' Bass beats or an RBM trained on both Drum n' Bass and Brazillian style beats by clicking either of the genreMode message boxes.

11. By default, the drum sequencer plays from the user arrays (the left panel of arrays in the light gray box). You can switch the playing to reflect the generated arrays (the right panel of arrays in the dark gray box), by clicking [1] message box in the CLICK HERE TO START PLAYING blue panel. Switch to play from the user arrays by clicking the [0] message box.

12. You can also copy over the generated pattern variations to the user arrays by clicking on the bang buttons in the dark gray box labeled with COPY GENERATED ARRAYS TO USER ARRAYS.  This is useful if you want to generate variations of a generated variation. 

13. Control other parts of the sound with:
   a) LEVELS: control the level of each sample separately
   b) REPEAT: add extra ghost notes for each sample and specify the probability for those extra notes to appear
   c) CIRCULAR SHIFT: shift the pattern around in a circle to the right for each specific sample


-- Training an RBM on different types of rhythmic patterns --
The RBM training is accomplished using Python.  If you would like to train your own RBMs on different datasets, continue on with this section. The code described in this section can be found in the rbmDrumSequencer/python_notebook/ directory.

I based my code off of the code found here: https://github.com/echen/restricted-boltzmann-machines.  I modified the RBM implementation found in that repo so that it could handle the type of input and output that I need for my drum sequencer.  This code is found in JH_RBM.py.  The file midi_tools.py is from Eric Battenberg's code at: https://github.com/ebattenberg/crbm-drum-patterns and provides an easy way to work with MIDI and drum instruments in Python.

The code for training the RBMs is saved in Python notebook files.  Sorry if you dislike Python notebook, but I love it because I can easily jot down notes and test things out in them. The Drum and Bass dataset is trained in RBM and MIDI tests.ipynb, and the weights are saved to weights.txt.  The Drum and Bass combined with Brazillian beats RBM is trained in RBM and MIDI with 2 genres.ipynb, and the weights are saved to weights2.txt.  These are the weights files that the Pd extern needs to be pointed to in lines 176, 201, and 484 in rbmMIDI.c.

If you would like to change the dataset that the RBM is trained on, open up  RBM and MIDI tests.ipynb.  Say you want to change the dataset to data/techno.mid.  Scroll to the code box in this notebook that is labeled "MIDI file to drum matrix." Change

filename = '../data/Drum_n_bass.mid'

to

filename = '../data/techno.mid'.

You may want to change the file in which we save our trained RBM weights. To do this, scroll down to the code box that has the line

np.savetxt('weights.txt', r.weights, delimiter='\n')

and change 'weights.txt' to something else like 'weights_techno.txt.'

Now run every single code box from the beginning to this code box that saves our weights as weights_techno.txt.  By the end of this process, a new weights file called weights_techno.txt should have been generated in the rbmDrumSequencer/python_notebook/ directory. Now, in the Pd extern code, pd_extern/rbmMIDI.c, you can swap out either weights.txt or weights2.txt with the newly generated weights_techno.txt.  Recompile the Pd extern, move it to the extras/ directory and restart the rbmDrumSequencer.pd patch.  By toggling on the genreMode message boxes in the top right corner panel, you can switch with the trained RBMs and observe how the generated patterns differ based on different training sets.


Hope you have fun! Send me an email at jsh008@ucsd.edu if you have any questions.