Issues With Database Migration From v1.4 of CaVE to v1.5

[Dark Akuma]

This happens with various types of NSO database, but largely with SNES.

It should only happen with NES, SNES, and Genesis databases as those used to use the xtx.z format for images, but no longer do in modern versions.

It can also happen with SNES due to the change in the file naming convention for the sfroms.

…

The gist of the issue I understand, I think. Its because v1.4.6 corrupted the databases as it tried to adjust to those file changes. It had a mix of xtx.z and png images. And as long as the file worked and the path was valid, NSO did not care if the format was correct to that version.

The issue is that CaVE v1.5 tries to fix this database stuff… but in the process makes it worse. It tries to make all images use the proper format of the the specified version of the app. To that end, it remakes image files, and renames sfrom as appropriate… but fails to change the img format (instead just changing the extension), or fails to change the paths to the images/sfroms in the titlesdb.

I have tried to address this issue several times… but it hasent worked, and I dont know why.

It’s very hard for me to reproduce as I have to manufacture a corrupt database. There is no procedure to make a v1.4 database like this that will help test the issue. And everything I have tried to manufacture one… works in the end.

…

The issue presents differently depending on image or sfrom based. Loading screen freeze or failure to load a game (as if the full unlock was not working). Of course, if both are in effect at the same time you wouldn't get past the loading screen to notice games not loading. And naturally… the image one relates to the loading screen freeze, and the sfrom name relates to the game loading not happening.

…

Additionally, a important part of this issue is that these are peoples normal databases that they have been updating through various updates, both with the v1.4 branch, and then in the v1.5 branch (at least 1 version update in v1.5, likely a large one to a more recent, then latest version of the NSO app). In particular, their databases creation preceeds the versions in which these formats changes. When they updated their databases between versions in CaVE v1.4, they did so to versions in which these format changes happened. For that reason I should include when these changes were made.

• NES_XTX_2_PNG_CHANGE = Version "6.4.0.0"
• SNES_XTX_2_PNG_CHANGE = Version "3.6.0.0"
• GEN_XTX_2_PNG_CHANGE = Version "2.5.0.0"
• SNES_ROM_PREFIX_CHANGE = Version "3.3.0.0"

I believe v1.4.6 tried to adjust how it handled these files with the version change, but off the top of my head I forget in what way. I just remember that it was not as through as I tried with v1.5.

[iiTzSamurai]

ok so I wanted to test the migration of database from v1.4.6 to v1.5 I made a fresh SNES database in CaVE v1.4.6, Imported 1 rom and added Custom cover art and Saved (Ignore what I chose as a test image lol) Opened the database in CaVE v1.5 and it asked what version database it was, I selected v3.6 I think and it instantly asked If I wanted to update the database to v3.9 (NICE) Updated the database with no error's and it even saved the image properly through migration

database loaded fine in Yuzu after migration also

‌

[Dark Akuma]

I posted about this in Discord, then completely forgot I was working on this issue. So I’m cleaning up and reposting what I said, here.

I went and made a database with CaVE v1.4 for testing when importing into v1.5. I found the result insufficient as... we tend to forget. CaVE v1.4 evolved over time. In its last state it doesn't make SNES databases exactly like previous versions did. In particular in regards to the sfrom naming convention. v1.4.6 includes the S- prefix with sfrom file names.

So I went and edited some the custom added games to not have that prefix, as if the database was originally made with a older version that did that. This put the database into a position to actually show the issue people encounter. And its not where I thought it would happen. Its not part of the update process. Its more part of either the import process (selecting the version for the first time and such) or just when you make your first save.

What happens is... the file names stay the same. But the prefix is added to the references in the titlesdb.

Now that I am aware of the exact issue, I just need to remember it… and get around to addressing it.

I recall looking into it a little deeper after that, and the issue happens because the database version is checked when accessing a variable for the rom path, and depending on what version it is, it returns that rom path with a sfrom prefix or not. It does not return the actual value, it constructs one.

Of course, this is only 1 of the 2 issues with Database migration. The other being images. But as this above issue is now currently understood, any testing should focus on that. And to dial in on what bugged result we are looking for with the image issue, its somehow ending up with a xtx.z that simply has its extension renamed to .png.

Looking over my code… I'm just not sure how that could happen, and I’m leaning towards it having happened with a earlier version of v1.5 only. I know I have revisited the relevant code a couple times, and may have fixed it. Just the ones reporting the issues more recently may have been using older versions of v1.5.

Actually… I should look over the v1.4.6 code too. And see how that might do it. Ive been assuming its a issue with the update process, since if v1.4.6 did that… the database wouldn't work. No v1.5 needed. But the issue has been presented as if it only occurred after converting the v1.4.6 database to v1.5.