1. Mytech Avatar Mytech
  2. ROBOBO
  3. robobo-programming

Wiki

Clone wiki

robobo-programming / native / sound-library

View History

Using sound library

The sound library allows the developer to use a series of sound processign algorithms to detect and produce sounds. The library is composed by six ROBOBO modules:

Sound dispatcher module:

This is an auxiliar module that captures the audio and feeds it into the sound processors. It must be declared before the other modules in the module.properties file. The interface of this class is as follows: 

    /**
     * Adds a sound processor to the list of active processors
     * @param processor The processor
     */
    public void addProcessor(AudioProcessor processor);

    /**
     * Removes a sound processor from the list of active processors
     * @param processor The processor
     */
    public void removeProcessor(AudioProcessor processor);

    /**
     * Starts the sound dispatcher
     */
    public void runDispatcher();

    /**
     * Stops the sound dispatcher
     */
    public void stopDispatcher();

    public AudioDispatcher getDispatcher();

The only methods relevant to the robobo apps developer are runDispatcher() and stopDispatcher() that are used to start and stop the sound processing. The other methods are used internally by the other sound processing modules.

Obtaining the instance of the module

ISpoundDispatcherModule dispatcherModule = roboboManager.getModuleInstance(ISoundDispatcherModule.class);

Current implementation

TarsosDSPSoundDispatcherModule: 

robobo.module.12=com.mytechia.robobo.framework.hri.sound.soundDispatcherModule.TarsosDSP.TarsosDSPSoundDispatcherModule

Clap detection module:

This module can be used to detect percussive sounds such as claps. It depends on the Sound Dispatcher Module.

The interface of this module allow to suscribe to the feed of clap events.

    /**
     * Suscribes a listemer to the clap notifications
     * @param listener The listener to be added
     */
    public void suscribe(IClapListener listener);

    /**
     * Unsuscribes a listener form the clap notifications
     * @param listener The listener to be removed
     */
    public void unsuscribe(IClapListener listener);

To detect claps a class must implement the interface IClapListener.class

    public interface IClapListener {
        /**
        * Notifies when a clap is detected
        * @param time The moment of the detection
        */
        void onClap(double time);
    }

The method onClap() is called on every percussive sound detected.

This module can be configured via the sound.properties file by changing the value of the following keys

clap_threshold = 8
clap_sensitivity = 50

Obtaining the instance of the module

IClapModule clapModule = roboboManager.getModuleInstance(IClapModule.class);

Current implementation

TarsosDSPClapDetectionModule:

robobo.module.20=com.mytechia.robobo.framework.hri.sound.clapDetection.tarsosDSP.TarsosDSPClapDetectionModule

Pitch detection module:

This module allows to detect sound frequencies. Depends on the Sound Dispatcher Module. The interface is the same of the Clap detection module. To detect pitch, a class must implement the IPitchListener.class listener.

public interface IPitchListener {

    /**
     * Called when a pitch is detected
     * @param freq The frequence of the sound
     */
    void onPitchdetected(double freq);
}

onPitchDetected() is called when a sound frequency is detected and returns the value of the frequency in hertz.

Obtaining the instance of the module

IPitchDetectionModule pitchModule = roboboManager.getModuleInstance(IPithcDetectionModule.class);

Current implementation

TarsosDSPPichDetectionModule:

robobo.module.20=com.mytechia.robobo.framework.hri.sound.pitchDetection.tarsosDSP.TarsosDSPPichDetectionModule

Note detection module:

This module allows the user to detect musical notes and measure their duration. Depends on Pitch Detection Module to work, that must be declared before in the modules.properties file. The interface is the same as the Pitch Detection module. To detect notes, a class must implement the INoteListener.class interface.

    /**
     * Called when a note is detected
     * @param note The detected note
     */
    void onNoteDetected(Note note);

    /**
     * Called when a note ends
     * @param note The note
     * @param time The time elapsed
     */
    void onNoteEnd(Note note, long time);

    /**
     * Called when a new note is detected
     * @param note The detected note
     */
    void onNewNote(Note note);
onNoteDetected() is called in continuous while a note is being detected while onNewNote() and onNoteEnd() are called at the beggining and end of a detection.

The enumerated class Note.class is used to represent the musical notes and its atributes. The interface is the following:

    Note(int octave,int index,String note){
        this.octave = octave;
        this.index = index;
        this.note = note;
    }
Each note has 3 atributes, the octave, the index, and a string representation using the english notation.

This module can be configured in the sound.properties file by changing the value of this keys:

minThreshold = 0.1
maxThreshold = 0.9

Obtaining the instance of the module

INoteDetectionModule noteDetectionModule = roboboManager.getModuleInstance(INoteDetectionModule.class);

Current implementation

TarsosDSPNoteDetectionModule:

robobo.module.20=com.mytechia.robobo.framework.hri.sound.noteDetection.tarsosDSP.TarsosDSPNoteDetectionModule

Note generation module:

This module can be used to synthesize musical notes. The interface of the module is:

public interface INoteGeneratorModule extends IModule {
    void suscribe(INotePlayListener listener);
    void unsuscribe(INotePlayListener listener);

    /**
     * Plays a note
     * @param note the note to be played
     * @param timems the duration of the note
     */
    void playNote(Note note,int timems);

    /**
     * Add a note to the internal sequence
     * @param note the note to be added to the sequence
     * @param timems the duration of the note
     */
    void addNoteToSequence(Note note,int timems);

    /**
     * Plays the internal sequence
     */
    void playSequence();

}
suscribe() and unsuscribe() can be used to to the end of reproduction notifications, playNote() allows to play notes (represented by the Note.class class, with the same interface as the Note in the NoteDetection module, but not compatible) addNoteToSequence() allows to add a series of notes to be played sequentially, playSequence() plays the stores sequence of notes.

To be notified of the events on note playing a class must implement the INotePlayListener interface.

public interface INotePlayListener {

    /**
     * Called when a note ends
     */
    public void onNotePlayEnd();

    /**
     * Called when a sequence of notes end
     */
    public void onSequencePlayEnd();
}

Obtaining the instance of the module

INoteGeneratorModule noteGenModule = roboboManager.getModuleInstance(INoteGeneratorModule.class);

Current implementation

AndroidNoteGenerationModule:

robobo.module.18=com.mytechia.robobo.framework.hri.sound.noteGeneration.android.AndroidNoteGenerationModule

Emotion sound module:

This module allows the reproduction of different emotion sounds.

public interface IEmotionSoundModule extends IModule {


    int ANGRY_SOUND = 0;
    int APPROVE_SOUND = 1;
    int DISAPPROVE_SOUND = 2;
    int DISCOMFORT_SOUND = 3;
    int DOUBTFUL_SOUND = 4;
    int LAUGH_SOUND = 5;
    int MUMBLE_SOUND = 6;
    int LIKES_SOUND = 7;
    int MOAN_SOUND = 8;
    int OUCH_SOUND = 9;
    int PURR_SOUND = 10;
    int THINKING_SOUND = 11;
    int VARIOUS_SOUND = 12;


    /**
     * Plays a prefixed emotion sound
     * @param Sound  The sound to be played
     */
    void playSound(int Sound);
}

Obtaining the instance of the module

IEmotionSoundModule emotionSoundModule = roboboManager.getModuleInstance(IEmotionSoundModule.class);

Current implementation

AndroidEmotionSoundModule: 

robobo.module.7=com.mytechia.robobo.framework.hri.sound.emotionSound.android.AndroidEmotionSoundModule
The javadoc documentation of this module can be found here

Updated