Support working with apps on iOS 8

[S. Zeid]

iOS 8 changes the location of saved app data. I need to support this change.

TL;DR

Data is now stored in /var/mobile/Containers/Data, and apps themselves are stored in /var/mobile/Containers/Bundle, and both still contain GUID-named directories as before, but the GUIDs are not the same, and I am not aware of any database that keeps a mapping. However, the GUID directories each contain a plist file called .com.apple.mobile_container_manager.metadata.plist that contains the reverse-domain-name bundle identifier, and I can use that to link the two together internally.

General layout of /var/mobile/Containers

• /Bundle
  • /Application
    • /<GUID>
      • /<app>.app
      • .com.apple.mobile_container_manager.metadata.plist
  • /VPNPlugin
    • (unknown)
• /Data
  • /Application
    • /<GUID>
      • /Documents
      • /Library
      • /<app>.app (sometimes; irrelevant)
      • /tmp
      • .com.apple.mobile_container_manager.metadata.plist
  • /PluginKitPlugin
    • (unknown; irrelevant)
  • /VPNPlugin
    • (unknown; irrelevant)
• /Shared
  • (unknown)

In short, /var/mobile/Applications → /var/mobile/Containers/Data/Application, but the GUIDs in Bundle and Data don't match. I need a new way to enumerate apps.

About .com.apple.mobile_container_manager.metadata.plist

Here are a couple of specimens:

• $ plutil -p ~mobile/Containers/Bundle/Application/B9DCED24-703C-4B08-B04D-3AE2EF0F6B71/.com.apple.mobile_container_manager.metadata.plist

  {
    "MCMMetadataIdentifier" => "example.Test-App--Please-Ignore"
    "MCMMetadataInfo" => {
      "com.apple.MobileInstallation.BundleMaxLinkedSDKVersion" => 524544
      "com.apple.MobileInstallation.StaticDiskUsage" => 53248
    }
    "MCMMetadataContentClass" => 1
    "MCMMetadataUUID" => "A8A12AAF-8DEB-4A7A-9C07-4BEAE9E4D58F"
  }
  

• $ plutil -p ~mobile/Containers/Data/Application/67E32582-EB57-48B0-B5F8-9C6375E71641/.com.apple.mobile_container_manager.metadata.plist

  {
    "MCMMetadataContentClass" => 2
    "MCMMetadataIdentifier" => "example.Test-App--Please-Ignore"
    "MCMMetadataUUID" => "4D475624-0527-484B-9125-4E933270CF60"
  }
  

Fortunately, both classes of this file happen to contain the bundle identifier. The UUIDs contained in the metadata plists are junk for my purposes; I only need the UUIDs of the containers' directories.

I will need to look at both metadata files for each app, because I need to read the app bundle's Info.plist to get things like the app name to display to the user.

Changelog

2014-12-12:

• Learned about .com.apple.mobile_container_manager.metadata.plist

[S. Zeid]

I need access to iOS 8 somehow for discovery purposes. (The listing of /var/mobile/Containers was provided by a user, but that's not all I need to know.)

[S. Zeid]

Reopening for visibility in the issue list (without having to change the filter to "All").

[Yohan Irani]

Hi Scott, I extensively use your app and I was surprised to know it isn't working on iOS 8.1 So can you please tell me what would be an approx ETA for iOS 8.1 support? Also I was trying to manually restore WhatsApp settings in iOS 8.1 which I backed up on iOS 7.0.4 I found the location and restored the Documents and Library folders. But when I start WhatsApp after the restore, it crashes and I have to delete the app and redownload it. Can you please help?

[S. Zeid]

There is absolutely no ETA. I have no idea when I'll have access to an iOS 8 installation to help me determine what needs to be done.

As for your WhatsApp issue, please make sure that when you restore the files, they are owned by the user and group mobile and not root, and that the permissions allow the owner to read and write the files (and read/write/execute the folders). If that isn't the issue, then I won't be able to help further.

[Yohan Irani]

Hi Scott,

Thanks for replying. Your solution to WhatsApp was very helpful and I could successfully restore its backup without the app crashing. I have done the same for other apps like a couple of games and Viber. All restored successfully. But it goes without mentioning that restoring backups using your app was super easy whereas this becomes very difficult, especially manually finding the UID of the AppStore app. Anywayz, I won't do the rest as they aren't that important right now. So will wait for your AppBackup to be compatible with iOS 8.x and will then use it to restore the backups of the rest of the apps. Thanks again and best of luck!

[S. Zeid]

@yohu123 You're welcome, and thanks for letting me know that manually restoring the data files works (after fixing the permissions).

@Anyone So I've figured out what needs to be done: it turns out that the Bundle/<GUID> and Data/<GUID> directories each have a plist file called .com.apple.mobile_container_manager.metadata.plist which contains (among other things) a MCMMetadataIdentifier string containing the corresponding app's reverse-domain-name ID. So all I need to do is update the app scanning code to look at these plist files and keep track of which GUIDs belong to which apps, and which ones are Bundle or Data containers. (I do need to look at both containers, because I have to look at the app bundle to see what the app's name, etc. is, but I only need to modify the Data container.)

Still no ETA for when it will be done, but it's not too hard at this point, so hopefully sooner rather than later. At this point it depends more on my personal schedule.

[S. Zeid]

[python] UNTESTED refactor and begin adding support for iOS 8 (see issue #2)

→ <<cset 2c289e43952a>>

[S. Zeid]

App listing has been supported for about two weeks now; however I've left this open to not confuse people about iOS 8 readiness since it was just called "Add support for iOS 8".

Currently, a release compatible with iOS 8 is blocked by #6 (Fix sporadic crashes on iOS 8).

[mai ling]

this need a bit of additional work to be 100% compatible with ios8. crossposting to https://github.com/autopear/ipainstaller/issues/14

Some apps store data in ~mobile/Containers/Shared/AppGroup/UUID.

They need to be backed up and restored, too.

for example, WhatsApp stores like this:

iPhone:/var/mobile/Containers root# plutil -key MCMMetadataInfo $(ipainstaller -i net.whatsapp.WhatsApp|grep ^Data:|cut -d\  -f2)/.com.apple.mobile_container_manager.metadata.plist
{
    "com.apple.MobileInstallation.ContentProtectionClass" = 0;
    "com.apple.MobileInstallation.GroupContainerIDs" =     (
        "group.net.whatsapp.WhatsApp.shared"
    );
}

take each value of "com.apple.MobileInstallation.GroupContainerIDs" and search it in every folder

iPhone:/var/mobile/Containers root# for file in $(find Shared/AppGroup/ -type f -name .com.apple.mobile_container_manager.metadata.plist); do grep -- 'group.net.whatsapp.WhatsApp.shared' $file; done

Binary file Shared/AppGroup/B0A2102D-3E87-4767-961F-24336E04B066/.com.apple.mobile_container_manager.metadata.plist matches

We found the folder, include it in the backup.

Restore:

1. backup with ipainstaller -B
2. make a copy of AppGroup/UUID
3. delete app
4. install app with ipainstaller
5. lookup the data directory with ipainstaller -i, unzip the IPA, copy files from inside the IPA file in the Container directory to Data/uuid directory BUT EXCLUDE THE FILE .com.apple.mobile_container_manager.metadata.plist, adjust permissions mobile:mobile
6. lookup the AppGroup directory, restore data from step 2, EXCLUDE .com.apple.mobile_container_manager.metadata.plist file

Probably there are some API calls to directly find UUID and skip searching, but I'm a sysadmin, not a programmer. You could possibly find more here?

https://developer.apple.com/library/prerelease/ios/documentation/General/Conceptual/ExtensibilityPG/ExtensionScenarios.html#//apple_ref/doc/uid/TP40014214-CH21-SW6 http://www.atomicbird.com/blog/sharing-with-app-extensions https://invasivecode.com/weblog/sharing-data-between-apps-and-their-extenstions/ http://stackoverflow.com/questions/24015506/communicating-and-persisting-data-between-apps-with-app-groups/24066826#24066826 http://webcache.googleusercontent.com/search?q=cache:Jc7tC01ar7IJ:https://groups.google.com/d/topic/rubymotion/qDsNSf8NtXM+&cd=9&hl=en&ct=clnk&gl=ro

Note: I have modified WhatsApp Info.plist to allow it to backup with Shared documents:

plutil -key UIFileSharingEnabled -type bool -value true /var/mobile/Containers/Bundle/Application/UUID/WhatsApp.app/Info.plist

[S. Zeid]

Thank you so much for telling me that! I never would have known otherwise. Reopening; I'll work on this soon.

[mai ling]

hey there, any update on this? cheers!