1. Tommy Anderberg Avatar Tommy Anderberg
  2. Why not
  3. RealSense for Singularity

Wiki

Clone wiki

RealSense for Singularity / About the code

View History


TARealSense

The squashed commit adding RealSense support involves no less than 41 files, but the number looks worse than it is. The bulk of the new code is in just two files, tarealsense.h and tarealsense.cpp, which introduce the new TARealSense class. All interfacing with the RealSense libraries is done there.

There is only one instance of TARealSense: a private member of LLAppViewer called mRealSense (see file llappviewer.h). It is initialized in LLAppViewer::init().

When the user logs in, LLAppViewer::handleLoginComplete() calls TARealSense::loadPersonalRealSenseSettings(). With one exception, all RealSense-related settings are account-specific. The exception is the main RealSense enable/disable switch, which applies to all accounts.

All account-specific RealSense settings are stored in the file indra/newview/app_settings/settings_realsense.xml, which is included in indra/newview/app_settings/settings_per_account.xml. Note that the names of all its entries are prefixed with "RealSense".

RealSense input is handled when the main viewer loop, LLAppViewer::mainLoop(), calls TARealSense::processFrame().

When the viewer shuts down, LLAppViewer::cleanup() calls mRealSense.enableRealSense(false) to stop all further processing of RealSense input.

You can obtain a pointer to LLApViewer::mRealSense by calling the public method LLAppViewer::getRealSensePointer():

TARealSense* pRS = LLAppViewer::instance()->getRealSensePointer();


Currently 64 bit Windows only!

Note the macro

#define TA_INCLUDE_REALSENSE (LL_WINDOWS && _WIN64)


near the top of tarealsense.h. It is used throughout the code to determine whether to include RealSense-related functionality. As currently defined, it is only true when building for 64 bit Windows. The code also compiles for 32 bit Windows, but the resulting executable promptly crashes as soon as RealSense is enabled. When (if?) this gets fixed, the "_WIN64" part will go. Further ahead, if RealSense shows up on Linux and Mac too, this macro will have to be modified accordingly.


UI additions

There are two new floaters:

The camera view floater is truly minimal; all it does is display the visible light image seen by the camera (if any). When drawn, it calls TARealSense::setOnImageData(), which tells TARealSense to call the static method TAFloaterRealSenseCameraView::onImageData() when there is a new image to display. The only (slight) complication is TAFloaterRealSenseCameraView::flipXandColors(), which turns the image data returned by RealSense into the format expected by the viewer and flips it horizontally (because watching your avatar turn left when your head turns right is just too confusing). The closed/open status and position of this floater is saved in gSavedPerAccountSettings, and llstartup.cpp restores it accordingly after login.

The RealSense Stup floater is a typical configuration dialog, with a little twist. The ordinary part reads/writes the account-specific settings in gSavedPerAccountSettings and calls the appropriate public setter methods on TARealSense to apply changes. The twist is a call to TARealSense::setRSFloaterCallbacks(), which tells TARealSense to call the static methods TAFloaterRealSense::onVoiceStatus() and TAFloaterRealSense::onVoiceRecognition() when there is new speech recognition data. This is used to display alerts and parsed voice commands on the Voice tab.

The new floaters can be opened using the new main menu entries Edit | RealSense and View | RealSense Camera View (registered in the file llmenucommands.cpp) or the corresponding keyboard shortcuts and voice commands. There is also a new "RealSense Setup" button on the "Input & Camera" tab of the Preferences dialog (with coorresponding additions to llpanelinput.h and llpanelinput.cpp).


Other changes

There are several minor changes in other files, done to expose functionality needed by TARealSense:

  1. llmenugl.h and llmenugl.cpp now expose a global S32 gMenusVisible, which gets incremented when menus are opened and decremented when they are closed. It is used to temporarily suspend head tracking when navigating menus, so moving your head doesn't cause menus to disappear. It is a lot less than 100% reliable, but mostly good enough.
  2. llfloatercolorpicker.cpp adds/removes itself to/from gFloaterView in order to be acessible by XUI clicks.
  3. llfloatertools.cpp calls TARealSense::setBuildMode() when the tools floater opens and closes.
  4. llpanelobject.h now exposes a getObject() method returning a pointer to the object being edited, and llpanelobject.cpp makes sure to set mObject = NULL when nothing is being edited.
  5. The three manipulator tools (llmaniprotate.h/cpp, llmanipscale.h/cpp and llmaniptranslate.h/cpp) now expose handleStartDrag() methods, and in the latter two, monolithic internal methods have been broken up to allow only the parts needed by TARealSense to be called.
  6. lltoolcomp.h/cpp exposes the corresponding methods (startRotating() and rotate; startScaling(), scaleAll() and scaleOne(); startTranslating() and translate()) which are actually called by TARealSense.

Also, llmanifest.py includes a fix (by SianaGearz) to a bug in the run_command() function.


Tommy Anderberg
February 25, 2015
Updated April 5, 2015

Updated