Bulk download of media (Similar to google drive)

[grupoapunte]

Hey,

This idea simply came up because my drive containing all my media failed and I basically lost everything, so I had to redownload for over 40k roms. This process takes a lot of time and it feels as it’s doing it one by one, but when I ran the process to download everything it was for all the games. So my proposal, similar to how google drive works is, when the server is asked to download more than X number of media files at once, that instead of having the client asking one by one, it builds a request to the server of all the media, and the server builds a zip like file (not necessarily compressed depending on what is less expensive CPU vs Network+Temp Storage in case the server is in AWS or GCloud). And a single file is sent to the client… this would certainly save a lot of time and resources but probably bandwith may be higher during transfer given that it would be a single file.

In short:

• Client request media for 2k roms to the server
• The server builds a zip with all the media
• The media gets transfered when ready
• Once in the client it is unpacked and added to the library

Other ideas to futher improve this:

• Spliting the file into pieces in order to avoid the download to fail and having to restart the download from scratch
• Maybe using P2P transfer so it can be paused (the zip file would have to eventually timeout)
• Standard zips could be already in the server with full media for each platform so no zipping is required, then once the zip is downloaded the client does the job of only adding to the library what the client requested (images/games) and the rest is deleted. This would require that the request to the server is of at least X% of the total contents of the zip (ie: 80% of the contents will be used).
• If the implementation is actually more expensive than to continue using the current method perhaps this could be for premium users

Thanks for reading and sorry for the long explanation, the platform is getting better and better since I originally started using it several years ago, keep it up!

Thanks!

Best, Damian.