Metadata pulled from iTunes for TV shows not matching US catalog, missing metadata.
Issue #531
resolved
I am importing many episodes of Fairy Tail, seasons 1-6, episodes 1-150. Despite being named appropriately, and confirming that iTunes does indeed have the relevant episode in the store, Subler is not finding the metadata either when manually searching or when putting the file in the queue for some episodes. Most episodes are OK. Normally, I’d just switch to TVDB, but the naming convention in TVDB is wrong and doesn’t match my files. Season 4, Episode 107 & 108, for example. Mac OS Mojave, Subler 1.5.16, no error message or dialog.
“Fairy Tail Season 4 Episode 107.mp4”, etc.
Marked as major because it’s core functionality, please edit as you see fit. Thank you for a wonderful application.
Comments (6)
-
repo owner
-
reporter
I see. In this case, it’s a “season 4, pt. 1”, pt. 2 example in the iTunes Store.
If I have time I’ll see if I can help.
-
reporter
Found one potential issue: Line 300 in iTunesStore.swift
let url = URL(string: "https://itunes.apple.com/search?country=\(store.country2)&lang=\(store.language2.lowercased())&term=\(searchTerm)&attribute=showTerm&entity=tvShow&limit=250"),Per the .json in the root of the project, store.language2 would return the 2 character language (EN), which is then lowercased. Per the documentation [https://affiliate.itunes.apple.com/resources/documentation/itunes-store-web-service-search-api/](https://affiliate.itunes.apple.com/resources/documentation/itunes-store-web-service-search-api/)
“The language, English or Japanese, you want to use when returning search results. Specify the language using the five-letter codename. For example: en_us.The default is en_us (English).”
Similarly, the documentation states that the “limit” variable’s accepted range is from 1 to 200. I’m trying some test cases, and so far this seems to reduce the volatility of the search results.
An example, I’m using this as the query url: http://itunes.apple.com/search?term=Fairy+Tail+Season+5&media=tvShow&entity=tvEpisode&attribute=tvSeasonTerm&limit=200&lang=en_us
In this example, I’ve changed the media, entity, attribute, limit, and language.
There are 22 episodes in Season 5 (11 in part 1, 11 in part 2) according to the iTunes store, and this returns 22 results. I am working on parsing those results more carefully to determine whether or not all episodes are returned correctly.
-
reporter
UPDATE: These changes returned all of the appropriate episodes, and none from other seasons. I’ll run some more test cases and report back.
-
- Returned 22 results from season 6 as expected.
-
- Returned 53 episodes as expected.
-
- Returned all episodes from season 4 as expected.
-
Also returned all episodes from season 7, pt. 4 and season 8, pt. 4.
-
Behavior exists due to “Pt. 4” in collectionName
- Next test case will be season 3 to confirm that seasons with 3 parts are also included. Could be significant if more than 200 results exist for season 1, given that all seasons have a “Pt. 1”
-
-
- Same behavior as case 4, with 61 total results
-
- Same behavior as cases 4 & 5, with 107 total results
-
- Same behavior as cases 4, 5, & 6, with 131 total results. Returns all of season 1, and all episodes with “1” in the collectionName
Based on these improved results, I recommend the following replacement for line 300 (I’ve not used SWIFT before, so please forgive me if I make some syntax errors):
let url = URL(string: "https://itunes.apple.com/search?country=\(store.country2)&lang=\(store.language2.lowercased())_\(store.country2.lowercased())&term=\(searchTerm)&media=tvShow&entity=tvEpisode&attribute=tvSeasonTerm&limit=200"),//This updated query string uses the correct media type, so that the API is searching TV show information, uses the tvEpisode “entity” such that the API is returning TV episodes, and the tvSeasonTerm attribute so that the term is searching in the collectionName, rather than the trackName. It also updates the “lang” query to use the 5 character code, per API documentation, and limits the results to 200, also per the API documentation.
In the test cases I’ve used, this has improved the search results, and prevented omissions of episodes in a season relative to the original query string. Filtering the results to exclude episodes not in the correct season is still necessary, as I have not figured out how to search more strictly.
-
-
reporter
Based on these improved results, I recommend the following replacement for line 300 (I’ve not used SWIFT before, so please forgive me if I make some syntax errors):
let url = URL(string: "https://itunes.apple.com/search?country=\(store.country2)&lang=\(store.language2.lowercased())_\(store.country2.lowercased())&term=\(searchTerm)&media=tvShow&entity=tvEpisode&attribute=tvSeasonTerm&limit=200"), -
reporter
- changed status to resolved
I created a pull request that modifies the query strings and other relevant lines in the /Classes/MetaDataImporter/iTunesStore.swift file that was merged into the project. The new code solves my issue.
- Log in to comment
TV Show metadata from iTunes Store are a mess. There is no metadata for the season number, you have to parse the show name after “season” or “book” or whatever, and ofter iTunes Store has the same show two times with a different name and a different episode numbers.
And the iTunes Store API sometimes doesn’t return all the data.
I’ll look in your specific issue in the next few days.