1. Samir Menon Avatar Samir Menon
  2. Untitled project
  3. scl-manips-v2

Wiki

Clone wiki

scl-manips-v2 / docu / scl-distrib

View History

Distributed Interfaces for SCL

Requirements for the Multi-Robot Interface

add stuff here...

Requirements for the Web Interface

Motivation

SCL's focus on providing a strong back-end framework to tie together different simulation and control modules has limited its utility for many common use cases. In particular, there is a strong need for:

  1. Data introspection
    • Present options : C++ logging and offline rendering || tedious & complex OpenGL/Qt programming
    • Desired : Real-time introspection through a command interface && high level interaction using some scripting language
  2. Systematic logging
    • Present options : Hard coded C++ logging; lots of crufty code and very tedious to look at everything
    • Desired : Simple logging with dynamic log introspection, rendering and adjustable sampling

Given the requirements, it was almost imperative to interface some sort of scripting language with SCL. The obvious choices were either python or javascript. In the end, many developments in pseudo real-time rendering and webGL made javascript the winner. The biggest advantage is being able to push complex simulations to one's website rapidly, and be completely platform independent.

SCL as a Distributed System

The program to enable web interfaces introduces a strong and systematic push for SCL to become a distributed system. This requires many systemic changes in how things are structured and promises to have strong repercussions for SCL's future.

Summarizing our design choices:

  1. Use JSON as a serialization and data sharing format
    • Readable, easily parsed, and is also YAML.
    • Can be converted
  2. Store SCL data structures as bulk JSON in Redis, 2 keys per robot (static, dynamic)
  3. Node server on the backend
    • Javascript (or coffeescript) on both frontend and backend keeps things simple
    • Great Redis bindings with node_redis or even redis-atomic-json.
  4. Socket.IO for streaming data from server to client
    • Fast, uses websockets
    • Super simple with Node

Some numbers

Fast Redis IO with Raw Data

  • Stream a Puma's joint angles (a string of doubles) and energy using hiredis (sync) while running the scl integrator.. 
Iterations : 50,000
Time : 5.65 sec
  • Raw SCL integration time: 
Time : 4.39 sec

Real-time Redis IO with 4ms Integ and 25Hz JSON SRobotIO Flush

  • Stream a Puma's IO state and energy using hiredis (sync) while running the scl integrator (@250Hz).. 
Total iterations      : 7,409
Total real-world time : 29.6502
Total simulated time  : 29.636

Fast Redis IO with 4ms Integ and 1:1 JSON SRobotIO Flush

  • Looks like we can dump a Puma's state into redis at about 5KHz 
Total iterations      : 50,000
Total real-world time : 10.0302
Total simulated time  : 200
  • Removing the redis calls from the sim, we get (raw integration time) 
Total iterations      : 50,000
Total real-world time : 4.39417
Total simulated time  : 200
  • Keeping all the serialization and message formatting etc. Just avoid the actual redis call 
Total iterations      : 50,000
Total real-world time : 7.19048
Total simulated time  : 200

127.0.0.1:6379> keys SCL:PumaBot*
1) "SCL:PumaBot:Graphics"
2) "SCL:PumaBot:Parsed"
3) "SCL:PumaBot:Energy"
4) "SCL:PumaBot:IO"

127.0.0.1:6379> get SCL:PumaBot:Energy
"395.235732"

127.0.0.1:6379> get SCL:PumaBot:IO
"{\"actuators_\":{\"force_gc_commanded_\":[0,0,0,0,0,0]},\"dof_\":6,\"sensors_\":{\"ddq_\":[-1.833860,2.812860,6.156520,-0.1120020,0.09710940,0.002315630],\"dq_\":[-0.3392980,3.249640,1.913830,0.2955420,4.324490,-1.623790000000000e-05],\"force_gc_measured_\":[0,0,0,0,0,0],\"forces_external_\":null,\"q_\":[-0.1211070,1.054540,99.76580,-5.329780,114.6510,0.003457050]}}\n"

Hayk's speed notes

I'm testing scl_redis_serv_puma with no sleeping. Running 50k iterations, counting the loop time by dividing the real-world time by 50000. Compiled in release mode but steps taken to make sure steps don't get optimized out.

Starting out with no integration even, with a completely blank simulation loop except for energy printout every 1000 steps.

Do basically nothing: 0.58 us/step

This will form the baseline to test each part by itself. Below is each component by itself in the loop, to let us know what kind of bottleneck it causes. Keep in mind that for a 10 kHz loop, we need 100 us/step (including everything).

Send short dummy command w/ Redox sync: 37 us/step
Send SCL:PumaBot:IO string (no serialization) w/ Redox sync: 42 us/step

Send short asynchronous command w/ Redox async: 4.2 us/step
Send SCL:PumaBot:IO string (no serialization) w/ Redox async: 9.2 us/step

Call serializeToJson, don't write: 39-68 us/step (quite variable)
Call serializeToJson, and writer.write: 52-86 us/step (quite variable)

serializeToJson, writer.write, and Redox async: 61-96 us/step

Just integration, nothing else: 126-186 us/step

Integration, serialization, Redox async: 234-360 us/step

Conclusions: Sending an asynchronous command is very fast, and even with no compression (full string output) takes under ten microseconds. The JSON serialization takes 6-9x longer than sending commands, with about 70/30 split in time between serializeToJson and writer.write. For some reason, my integration is taking quite a long time (cannot reach 10kHz even by itself, which is odd because in my iron dome project I could do 10kHz).

I tried just using a stringstream and manually creating the JSON string. Performance was very similar to just using the JSON serialization, which probably means that json-cpp uses stringstreams. Also, a note is that the sensor and actuator data should be split up in practice - a controller should only be outputting actuator data, and a driver should only be outputting sensor data.

All said, we aren't hitting 10kHz right now on my laptop, but rather 3-4 kHz in real time. However, that's with strangely large integration times. We have room to try to optimize, but for now I'll move to the web side of things.

Update: I realized this example was using a different integrator than I had been before. I tried out Tao instead of Spatial (not sure what the tradeoffs are), and it is much faster. I can get something like 8kHz. However, energy seems to be all over the place.

Technical Issues and Challenges

Issue 1 : Real-time introspection

  1. SCL should be completely reentrant ASAP!
  2. All SCL data structures should support serialization

Updated