Download Metadata and Media errors

[John Walshaw]

Launchbox 11.10 with full set of MAME 0320 (38,199 files). Upon running Download Metadata and Media there are errors. This results in taking approx 15 hours to run. to completion Options selected are as follows:

• Search for game information and emdia from the Launchbox Games Database = enabled
• Would you like to download media fro your games = Select None
• Remove roughly-matched duplicate images = enabled
• Would you like to download media from EmuMovies = Video - Theme selected
• Prioritize images over LaunchBox Games Database = disabled
• Would you like to update games with existing metadata and/or media = Yes, but do not replace any existing fields or media

Approx 85 error messages in the following format:

Could not download image http://api.gamesdbase.com/<long filename>.mp4 for <name of game>: The remote server returned an error: (404) Session has expired. Please login..
Could not download image http://api.gamesdbase.com/<long filename>.mp4 for <name of game>: The remote server returned an error: (404) Session has expired. Please login..
Could not download image http://api.gamesdbase.com/<long filename>.mp4 for <name of game>: The remote server returned an error: (404) Session has expired. Please login..
Could not download image http://api.gamesdbase.com/<long filename>.mp4 for <name of game>: The remote server returned an error: (404) Session has expired. Please login..
Could not download image http://api.gamesdbase.com/<long filename>.mp4 for <name of game>: The remote server returned an error: (404) Session has expired. Please login..

Would you like to keep the metadata updates despite these erorrs? = Yes

Appears that the timeout value on the session could be tuned to a lower number, combined with a retry count. Thank you.

‌

[Christian]

These are all related to EmuMovies. They recently did a server update/restart and had connectivity issues because of it. I would assume if you tried again you would see less errors. Because of that restart however you may need to reset your password as EmuMovies forced reset a very large number of users passwords. I’d make sure to vet out the password will still work by using the test button in the options menu before trying to download a large collection of media.

[John Walshaw]

Thank you and yes I agree that it’s EmuMovies. I already took care of the password and tested to verify connectivity. What are your thoughts about the timeout and potentially a retry mechanism to address session instability issues? I’m basically re-running until there aren’t any errors, which appears will take another few days.

[John Walshaw]

I repeated and it ran in under 5 hours, which I believe is accounted for the bulk of mp4 files already having been downloaded. Interestingly, the errors are identical. I’ll look to open a ticket with EmuMovies on the files. Interested to hear your thoughts on the question on tuning timeout and retries logic?

[Christian]

Not entirely sure what the question is if I’m honest. Are you asking the likelihood of placing a “try this file link x times before moving on” functionality into the tool?

[John Walshaw]

Yes, and possibly looking at the time-out to verify it’s appropriately tuned. Watching LaunchBox status, it would appear to be hanging for many minutes.

[Christian]

I’m fairly certain we do have time outs and I’m pretty sure Jason not too long ago shortened them already. I’d have to look and confirm though. The retry thing I’m not as sold on but we can leave that open to see how popular a request it is. My main concern with it is that the download tool isn’t passive and is transactional. Because of that we want it to do its job the best it can and then return control as soon as possible. Adding a retry to a link that could simply be dead would increase the time it would take to download with no additional files netted so not sure I’m quite on board with that part of the request, something to consider though.

[John Walshaw]

I agree that the retry logic may not be helpful. Seeing these are likely dead links, it doesn’t make sense. I created the ticket on EmuMovies (#7203) to see if that can be looked at. For the timeout, what I observed at times was LaunchBox status would appear to hang. I can’t say it’s related to these dead links for sure and would need to correlate. I would often find myself thinking that I needed to exit and relaunch. Leaving it alone would eventually (many minutes) status messages would update. The supposed hung state would be download related, hence my thinking related to this and raising the topic of timeout. I’d be curious to know what the timeout is that Jason tweaked.

[Jason Carr]

This particular issue was caused by a problem that EmuMovies was having recently. All EmuMovies users need to reset their passwords on EmuMovies now. Because of this, all the EmuMovies downloads were failing because the login was no longer working. Unfortunately retrying downloads wouldn't have done anything to help in this situation.

[John Walshaw]

Thank you for confirming this detail. Christian and I covered that and the password reset had been addressed before creating this ticket. The links are dead and I’ve created a ticket with EmuMovies for that also.

However, I am seeing periods of time in LaunchBox where it hangs on a download and is what lead to my questioning the timeout.