Filter the simulation state takes hours

Issue #40 new
Martin Dlouhy
created an issue

Yes, I know that you mentioned on https://bitbucket.org/osrf/subt/wiki/tutorials/qual that "This process can take many minutes." ... but hours? Is it really expected? The original simulation was faster (!) and the files are relatively small (original 259MB and the "filtered" after 3hours 27MB). I think that I just missed dead-line even for the west coast midnight ...

Also is it worth it? I understand that smaller the better but the "price" is maybe too high?

I am referring to command: gz log -f ~/.gazebo/log/2018-12-22T040945.151018/gzserver/state.log --filter .pose/.pose -z 60 -o ~/.gazebo/log/2018-12-22T040945.151018/gzserver/subt_tunnel_qual_sim_state.log

Comments (3)

  1. Martin Dlouhy reporter

    OK, submitted "raw" status was gziped to 229MB and filtered version 22MB, but the conversion took 8 hours! The simulation itself was around 50 minutes.

    It is really necessary to re-sample original poses to 60Hz?

  2. Martin Dlouhy reporter

    p.s. simple reading original gazebo log file with Python sax, concatenating blocks, zlib.uncompress and dump sizes of each chunk took 32.520 seconds ... so if "filtering" should just drop some iterations it should be much much faster. Also I am not sure if it is necessary to dump pose of each tile for every iteration? In other words the filtered poses file could be probably much smaller.

    (example how to extract readable XML is here: https://github.com/robotika/osgar/commit/bbd26b54cd7425ace084006e3d65912ab23c71db)

  3. Log in to comment