RetroAchievements scan inconsistent/spotty for PS2 games

[retroNUC]

Out of 213 PS2 games that I had on a playlist that were all manually verified as correct RA hashes, only 63 correctly generated hashes when scanned by RetroAchievements. The rest ended up in metadata post-scan as COULDNTFILEHASH.

Unsure if the problem is that these games are CHD format, or whether there’s something wrong with your identification implementation - https://docs.retroachievements.org/Game-Identification/#playstation-2

White trophy icons in image below are my manually set Achievement badges, yellow trophy icons are the new official implementation:

‌

[retroNUC]

Code algorithm here, if you want to compare. Not sure if you’re embedding rcheevos library or rolling your own:

https://github.com/RetroAchievements/rcheevos/blob/31f8788fe0e694e99db7ce138d45a655c556fa96/src/rhash/hash.c#L1390

‌

[Christian]

We are using that repo’s release. A hash value of COULDNTFILEHASH means their program returned an error when attempting to hash your file. I have noticed this happens quite a bit more with PS2 CHD files than any other filetype. Unfortunately there isn’t much we can do on our end until the release is updated to correct the issue.

[retroNUC]

Hmm. Given that PCSX2 uses the same library (albeit an older version) and manages to recognize these CHDs 100%, that would suggest either:

• Something broke with PS2 identification between the library release that PCSX2 uses and their latest release, in which case I’d raise an issue with Jamiras on the RA GitHub page
• Something’s slightly off with the LB library integration

If you locally roll back to the older library release, that could help narrow it down.

‌

[retroNUC]

Huh, maybe ignore me, PCSX2 isn’t actually using rc_hash_ps2()

https://github.com/PCSX2/pcsx2/blob/master/pcsx2/Frontend/Achievements.cpp#L1380

[retroNUC]

https://github.com/RetroAchievements/rcheevos/issues/225

[Christian]

Thanks for looking into this so thoroughly

[Christian]

Believe you misunderstood what Jamiras was telling you. We aren’t doing any of the logic whatsoever and ARE using the application from the repo he posted. We simply pass your file to THEIR application and apply the result they give when it successfully hashes the file.

[retroNUC]

Apologies, deleted my previous post, rahasher does support CHD as part of the scan.

Looking through the games in my database, I noticed that I had two game entries that were separate (an original and an undub) and two different files on my drive, but both result in the same RA hash because the ELF and partial hash is the same. However, one was picked up correctly in the LB scan, and the other resulted in was COULDNTFILEHASH.

Manually deleting the RA-related data from the platform XML and redoing the LB scan, both were picked up correctly the second time.

Perhaps it could be the case you mentioned where if you’re multithreading the hash calls and saturating the CPU, then the timeout for the hash response gets hit on some of these?

‌

[retroNUC]

Here’s a video of what the subthreads appear to be doing, see if this matches your expectations - https://youtu.be/m9MF22UF2LA

This set is made up of both files on a local drive and a mounted network drive.

‌

[retroNUC]

Oh, interesting, I am finding some CHD files that rahasher is failing on!

When one of the threads does fail, does it kill all other in-progress rahasher threads? Perhaps that’s why we're seeing files that do hash fine come up with COULDNTFILEHASH?

[Christian]

There is no communication between processes. I think what is happening is that we have a 1000ms timeout set in app and you have some files that CAN be hashed but take longer than that. We may need to increase that timeout period to assist with that. On my machine failures can take upwards of 5-6s and that just makes scanning this platform unbearable given how often you see it happen.

[retroNUC]

An update on this one:

• RAHasher was failing on some files. Narrowed this down to it using a new version of libchdr that it was using, which seems to be broken. RetroArch, PCSX2, and most other emulators use a much older release.

• Opened an issue on the RALibretro repo here, but the issue probably needs to be fixed on the libchdr side:

  • https://github.com/RetroAchievements/RALibretro/issues/368

• I uploaded a new RAHasher exec to the LB forums, but you could always build it from my libchdr downgrade branch for inclusion in LB releases:

  • https://forums.launchbox-app.com/topic/71514-132-beta-thread/page/9/#comment-433284
  • https://github.com/retroNUC/RALibretro/tree/libchdr_update

• It still fails when hashing a few titles, but haven’t looked into it further, and this now seems to work for 99% of files

‌