Using the speech library

The speech library provides an easy interface for accessing the speech capabilities of the smartphone. The library is composed by two ROBOBO modules:

Speech production module:

This module gives to the robot the capacity to talk.

The interface of the module is the following:

public interface ISpeechProductionModule extends IModule {
    Integer PRIORITY_HIGH = 1;
    Integer PRIORITY_LOW = 0;


    void sayText(String text, Integer priority);

    void setLocale(Locale newloc);

    void selectVoice(String name) throws VoiceNotFoundException;

    void selectTtsVoice(ITtsVoice voice) throws VoiceNotFoundException;

    Collection<ITtsVoice> getVoices();

    Collection<String> getStringVoices();

    void suscribe(ISpeechProductionListener listener);

    void unsuscribe(ISpeechProductionListener listener);
  }

sayText() is used to send a phrase to the TTS engine to be pronounced, setLocale() is used to pass Locale information to the TTS, changing the language, selectVoice() and selectTtsVoice() can be used to select the different voices available. getVoices() and getStringVoices()' return the list of available voices. suscribe() and unsuscribe() are used to register listeners to the speech notifications feed.

There is an interface to represent the voices on a high level, the interface is the following:

public interface ITtsVoice {

    String getVoiceName();

    String getVoiceLanguage();
}

The following interface must be implemented to receive speech notifications:

public interface ISpeechProductionListener {
    public void onEndOfSpeech();
}

Obtaining the instance of the module

ISpeechProductionModule speechProdModule = roboboManager.getModuleInstance(ISpeechProductionModule.class);

Current implementation

AndroidSpeechProductionModule:

This implementation uses the text to speech engine provided by the android API.

Note: The first use of each voice requires an internet connection

To use this module on your ROBOBO project you must declare it on the modules.properties file of the project.

  robobo.module.4=com.mytechia.robobo.framework.hri.speech.production.android.AndroidSpeechProductionModule

Speech recognition module:

This module enables the ROBOBO to hear and understand speech. the module provides two approaches to the speech recognition module, a keyword spotting mode and a grammar guided search.

The interface of this module is as follows:

public interface ISpeechRecognitionModule extends IModule {

    void addPhrase(String phrase);

    void removePhrase(String phrase);

    void updatePhrases();

    void cleanPhrases();

    void pauseRecognition();

    void resumeRecognition();

    Boolean hasStarted();

    void setKeywordSearch();

    void setGrammarSearch(String searchName, String grammarFileName);

    void suscribe(ISpeechRecognitionListener listener);

    void unsuscribe(ISpeechRecognitionListener listener);
}

addPhrase() and removePhrase() are used to manage the internal listing of recognizable phrases, to make this changes effective the ´updatePhrases()´ method must be called.

pauseRecognition() and resumeRecognition() can be used to start and stop the recognizer.

hasStarted() returns a boolean indicating whether the module has started or not. The initialization of some speech recognition engines can take several seconds.

setKeywordSearch() and setGrammarSearch() are used to switch between searches.

At last the suscribe() and unsuscribe() are used to suscribe listeners to the keyword spotted notifications. The interface of such listeners is the following:

public interface ISpeechRecognitionListener {

    void phraseRecognized(String phrase, Long timestamp);

    void onModuleStart();
}

Two callbacks are implemented, one that notifies the user when the module is loaded and ready to be used onModuleStart() and another that is called when a phrase is detected phraseRecognized().

If the grammar method is chosen for the detection, the grammar must be defined in the assets\sync folder of the application in the JSFG, a example is shown below.

#JSGF V1.0;

grammar movements;
<starter> = rob;
<direction> =front|back|up|down|right|left;
<ending> = now;

public <movements> = <starter> <direction> <ending>;

This module requires the permission to record audio:

<uses-permission android:name="android.permission.RECORD_AUDIO" />

This module is not compatible with the Sound Modules that use the microphone.

Obtaining the instance of the module

ISpeechRecognitionModule speechRecogModule = roboboManager.getModuleInstance(ISpeechRecognitionModule.class);

Current implementation

PocketSphinxSpeechRecognitionModule

To use this module on your ROBOBO project you must declare it on the modules.properties file of the project.

  robobo.module.4=com.mytechia.robobo.framework.hri.speech.production.android.PocketSphinxSpeechRecognitionModule