Using the vision library

The vision library provides an easy interface for using the smartphone camera and a series of different computer vision algorithms. The library is composed by three ROBOBO modules:

Basic camera module:

This module is the base of all the vision modules, it manages the capture of camare frames, and and provides a method to access those camera frames from applications and other modules.

The interface of this module is shown below:

public interface ICameraModule extends IModule {

    public void suscribe(ICameraListener listener);

    public void unsuscribe(ICameraListener listener);

    public void signalInit();

    public void passOCVthings(CameraBridgeViewBase bridgebase);

    public void changeCamera();
}

signalInit() is used to signal the camera nodule to start capturing frames.

passOCVthings() is required only for the OpenCv implementation of the module, and is used to pass the module the preview surface where the images are captured, this will be further explained at the OpenCv basic camera module section.

The last one is changeCamera(), and is used to switch between the cameras of the smartphone.

The classes which want to have access to the image feed need to implement the following interface:

public interface ICameraListener {

    void onNewFrame(Frame frame);

    void onNewMat(Mat mat);
}

onNewFrame() notifies the listener when a frame is available and returns a Frame object.

onNewMat() notifies the listener when a frame is available and returns an OpenCv Mat object.

Some parameters of the module can be configured via a vision.properties file located at the assets folder of the Android project.

resolution_height = 240
resolution_width  = 160

Obtaining the instance of the module

ICameraModule cameraModule = roboboManager.getModuleInstance(ICameraModule.class);

Current implementation

OpenCv basic camera module

This module is implemented using the OpenCV library. To use this module on your Robobo project you must declare it on the modules.properties file of the project as follows: java robobo.module.XX = com.mytechia.robobo.framework.hri.vision.basicCamera.opencv.OpenCVCameraModule

Face detection module:

This module provides the functionality of detecting faces on captured images.

The interface of the module only contains methods to manage the subscription of the listeners:

public interface IFaceDetectionModule extends IModule{

  public void suscribe(IFaceListener listener);

  public void unsuscribe(IFaceListener listener);
}

The listener interface is the following:

public interface IFaceListener {

  void onFaceDetected(PointF faceCoords, float eyesDistance);

  void onFaceAppear(PointF faceCoords, float eyesDistance);

  void onFaceDissapear();
}

onFaceDetected() returns the coordinates of the detected face and the distance between eyes, measured in pixels, onFaceAppear() is called when a face is detected for the first time and onFaceDissapear() when a face is lost for five frames.

Note: Only one face detection per image is supported at this moment

This module depends on the basicCameraModule

Obtaining the instance of the module

IFaceDetectionModule faceModule = roboboManager.getModuleInstance(IFaceDetectionModule.class);

Current implementation

Android face detection module:

This implementation was developed using the FaceDetector class provided by the Android API.

To use this module on your ROBOBO project you must declare it on the modules.properties file of the project.

robobo.module.0 = com.mytechia.robobo.framework.hri.vision.faceDetection.android.AndroidFaceDetectionModule

Color measurement module:

This module allows to detect the main color of the scene, spliting it in the three RGB channels.

The interface of the module only allows the developer to suscribe and unsuscribe to the feed of color notifications. If we want to receive those notifications, the interface 'IColorMesauredListener' must be implemented.

public interface IColorMesauredListener {

    void onColorMesaured(int r, int g, int b);
}

onColorMesaured() returns the main color of the scene splitted in the three RGB channels.

Obtaining the instance of the module

IColorMeasurementModule colorMeasurementModule = roboboManager.getModuleInstance(IColorMeasurementModule.class);

Current implementation

OpenCvColorMeasurementModule:

To use this module on your ROBOBO project you must declare it on the modules.properties file of the project.

robobo.module.0 = com.mytechia.robobo.framework.hri.vision.colorMesaurement.opencv.OpenCVColorMesaurementModule

Blob tracking module:

This module allows to track color blobs the scene, spliting it in the three RGB channels.

public interface IBlobTrackingModule extends IModule{
    void configureDetection(boolean detectRed, boolean detectGreen, boolean detectBlue, boolean detectCustom);
    void suscribe(IBlobListener listener);
    void unsuscribe(IBlobListener listener);
    void setThreshold(int th);


}

The interface of the module only allows the developer to suscribe and unsuscribe to the feed of color blob notifications ad configurate the behavior of the module. If we want to receive those notifications, the interface 'IBlobListener' must be implemented.

public interface IBlobListener {
    void onTrackingBlob(Blob blob);
    void onBlobDisappear(Blobcolor c);

}

onTrackingBlob() returns a Blob object, with the information of the currently tracked blob. onBlobDisappear() is used to notify the user when a tracked blob is lost.

Obtaining the instance of the module

IBlobTrackinModule blobTrackingModule = roboboManager.getModuleInstance(IBlobTrackingModule.class);

Current implementation

OpenCvBlobTrackingModule:

To use this module on your ROBOBO project you must declare it on the modules.properties file of the project.

robobo.module.0 = com.mytechia.robobo.framework.hri.vision.blobTracking.opencv.OpenCVBlobTrackingModule

The javadoc documentation of this module can be found here