Sky consumes significant portion of CPU, even when clouds disable.

Create issue
Issue #1579 closed
Nate Koenig created an issue

Running callgrind, I see SkyX::VClouds::DataManager::_performCalculations use 63% of the relative CPU cost.

Comments (15)

  1. Jose Luis Rivero

    The main callers of SkyX::VClouds::DataManager::_performCalculations are:

    • SkyX::VClouds::DataManager::_getDensityAt
    • SkyX::VClouds::DataManager::_getLightAbsorcionAt

    The sky creation stack is:

    Scene::setSky() ->
      VcloudsManager::create() ->
         Vclouds::create()
    

    I was approaching to the problem by not creating the sky until the first Image Sensor was adding to the world, but that fail since the gzclient GUI also needs to represents the clouds.

    @nkoenig would make sense to create the sky only while creating the world, only if the proper sdf sky tag is defined? Any other recommended approach?

  2. Jose Luis Rivero

    I'm having some fun days trying to discover how to connect the SDF data, read by the gzserver, with gzclient which the one that starts the sky generation unconditionally. Here is the situation:

    Approach 1: pass sdf data to gzclient and create the sky only if sdf enable it

    Both, gzserver and gzclient call RenderEngine, which seems to be a singleton so my approach would try to get the enable/disable sky flag into the RenderEngine.

    gzserver (read bottom to up, gdb bt)

    gazebo::rendering::RenderEngine::Init 
    gazebo::rendering::init 
    gazebo::sensors::init 
    gazebo::setupServer
    gazebo::Server::PreLoad
    gazebo::Server::ParseArgs (sdf could be read here)
    

    gzclient (read bottom to up, gdb bt)

    gazebo::rendering::Scene::Init
    gazebo::rendering::RenderEngine::CreateScene 
    gazebo::rendering::create_scene
    gazebo::gui::GLWidget::GLWidget
    

    Going with this approach would imply to modify API calls between gazebo::Server::ParseArgs -> gazebo::Server::PreLoad -> gazebo::setupServer, which seems not very elegant to me.

    Approach 2: use scene message to disable sky if not present

    When gzclient is starting there is a scene msg that contains data about the whole scene, including the sky. The message is processed by Scene::ProcessSceneMsg(ConstScenePtr &_msg).

    My approach here was to leave the sky to be created but if the message lacks of the sky information, then disable and remove the sky on the Scene::ProcessSceneMsg method. Not very nice but seems to work.

    Callgrind shows a decrease of 80% the time spend on skyx (and probably it could decrease even more if I leave gzclient open more time).

    Any other idea? Should I test more the second approach and go with it? @nkoenig @iche033 @chapulina

  3. Ian Chen

    @_jrivero_. Thanks for looking into this. We probably don't want to create the sky any more until we get a scene msg that has the sky field (the msg is sent from World after it parses the sdf). So the approach I was thinking of is to remove SetSky in Scene::Init and create the sky in Scene::OnSkyMsg if it's not already created. This is just a slight variation of your approach 2. How does that sound? I am happy to spend some time helping with this.

  4. Jose Luis Rivero

    That was my first try, I liked that approach: do not generate the sky until the first message arrives. However, if apply the following patch and run the empty_sky.world, I see no sky in gzclient.

    diff -r 7c6a01286318 gazebo/rendering/Scene.cc
    --- a/gazebo/rendering/Scene.cc Fri May 29 13:02:53 2015 -0700
    +++ b/gazebo/rendering/Scene.cc Tue Jun 02 20:14:48 2015 +0200
    @@ -325,17 +325,6 @@
       for (uint32_t i = 0; i < this->dataPtr->grids.size(); ++i)
         this->dataPtr->grids[i]->Init();
    
    -  // Create Sky. This initializes SkyX, and makes it invisible. A Sky
    -  // message must be received (via a scene message or on the ~/sky topic).
    -  try
    -  {
    -    this->SetSky();
    -  }
    -  catch(...)
    -  {
    -    gzerr << "Failed to create the sky\n";
    -  }
    -
       // Create Fog
       if (this->dataPtr->sdf->HasElement("fog"))
       {
    @@ -1475,9 +1467,26 @@
       // Process the sky message.
       if (_msg->has_sky())
       {
    +    // Create Sky. This initializes SkyX, and makes it invisible. A Sky
    +    // message must be received (via a scene message or on the ~/sky topic).
    +    try
    +    {
    +      std::cout << "Scene::Init()::SetSky()" << std::endl;
    +      this->SetSky();
    +    }
    +    catch(...)
    +    {
    +      gzerr << "Failed to create the sky\n";
    +    }
    +
         boost::shared_ptr<msgs::Sky> sm(new msgs::Sky(_msg->sky()));
         this->OnSkyMsg(sm);
       }
    +  else
    +  {
    +    this->dataPtr->skyx->remove();
    +    this->dataPtr->skyx = NULL;
    +  }
    
       if (_msg->has_fog())
       {
    

    Giving a look into what's is different when moving the setSky call out of Scene::Init, the only relevant command executed after the setSky is the message publication of "scene_info". I tried to add the same into the previous patch, but it also did not show the sky on gzclient.

    If we go up in the call stack, the call to Scene::Init comes from RenderEngine. After that call there are two relevant comands: this->scenes.push_back(scene); and rendering::Events::createScene(_name); which probably are the ones that do the trick.

  5. Steve Peters

    I'm guessing that the else block might be the problem. Maybe a publisher only wants to modify one aspect of the scene, so it just sets one field in the message. With this patch, wouldn't the sky be deleted whenever a scene message is received that doesn't have sky information?

  6. Jose Luis Rivero

    That makes sense yes. In the very simple case, only one scene msg is being transferred. So the destruction it is never called. I did the test, and removing that part of the code, does not help, gzclient still have no sky.

  7. Jose Luis Rivero

    That make it works! Awesome, thanks Ian.

    However I've noticed a problem. Do:

    • Open gzserver worlds/empty_sky.world
    • Open gzclient
    • Add a camera in the gzclient
    • Close gzclient -> Segmentation fault
    • Open a new gzclient -> no render, black GUI.

    I can continue with the gdb if useful.

  8. Ian Chen

    oh good catch. I was able to fix the segfault but then I noticed that black screen doesn't always happen. After some debugging, I found that delaying the creation of skyx changed the start up timing which causes the scene_info request message not to be sent somehow (the server side sometimes just doesn't get the request msg). I didn't go into the transport layer to debug this.

    So I changed the approach a little. I create the sky in Init and only start updating skyx when the scene has a sky - see changes here.

  9. Jose Luis Rivero

    Umm interesting, yes I've checked that not always the black screen appears.

    Your approach works for me! nice. valgrind reports some activity on skyx (higher than the other approach) but expected and not really big. I believe that we are ready to pull request. And open a new issue for the problem with the transport layer.

    Thanks Ian! I will pay the beers.

  10. Log in to comment