Qualification ZIP file - 2 or 3 files?

Issue #39 closed
Martin Dlouhy created an issue

The https://bitbucket.org/osrf/subt/wiki/tutorials/qual states that qualification ZIP file should contain 2 files, while SubT_Qualification_Guide.pdf says: "A submission will consist of a single zip (.zip) file that contains three files: simulation.log, subt.log, and narrative.md." ... so which one is correct?

Comments (12)

  1. Martin Dlouhy reporter

    Thank you Nathan for update - could you maybe please also update the command in "3. Create the final zip file", i.e. to add on the command line "narrative.md"?

    For me it is still not clear if we should submit improvements and what is the current threshold for qualification? Is there any "score board" with other teams? I did not realize that it is manual work so maybe multiple submissions are discouraged? thanks a lot Martin

  2. Nate Koenig

    The following text has been added to the wiki:

    There are sixteen artifacts in the tunnel qualification world. In order to pass qualifications, you will need to find at least half of these artifacts. For each successful artifact found, you can receive a minimum score of 3 and a maximum score of 9. The more accurate, in Euclidean distance, your artifact report is the higher the score. A minimum overall score of 24 is required, which is 8 artifacts each with a value of 3. Keep in mind that a score of 24 is not the only requirement, you must find at least 8 artifacts.

  3. Nate Koenig

    A leaderboard that will display team scores is under development. We are not discouraging multiple submissions from teams, but it would be good to refrain from inundating us with numerous submissions.

  4. Martin Dlouhy reporter

    Thank you Nathan for update. Is there a change in scoring since 21st Dec 2018? At that time the point for artifact was multiplied by up to 3 times for artifact precision and the same how fast you report it. Our first attempt was in 20 minutes and it was 1 point only. The second attempt (the same artifact) was 4 points:

    465 1000000 new_artifact_reported
    465 1000000 distance_bonus_x2
    465 1000000 time_bonus_x2
    465 1000000 modified_score 4
    465 1000000 new_total_score 4 

    Note, that draft "SubT_Challenge_Competition_Rules_Tunnel_Circuit.pdf" section 11.1 talks only about 1 point for 5 meter precision.

  5. Nate Koenig

    Our original scoring algorithm predated the competition rules. Pull request #107 will bring the scoring algorithm inline with the competition rules. Once merged, I'll update the wiki page to match.

  6. Log in to comment