1. Damiano Galassi Avatar Damiano Galassi
  2. Subler
  3. Subler
  4. Issues

Support for new tags (HDR, 4K, Dolby Vision, Dolby Atmos)

Issue #451 closed
Kyle Rove created an issue

At least in tvOS, there appear to be tags for these. Would be nice to support them. Of course, finding documentation for them anywhere on the web has proved fruitless. Maybe someone knows more?

Comments (63)

  1. Priit Varik

    I know, how to make Atmos to be recognised by tvOS 12. Would be nice to understand the effort it would take to implement into Subler. EAC3+Atmos, HDR10 and Dolby Vision bitstreams are already passed on by Subler so that they are played back correctly. Atmos requires the update to dec3 atom, though.

  2. Damiano Galassi repo owner

    Probably you would need to modify mp4v2. In the contrib there is only a prebuilt version. If you need to rebuild it yourself, you need the version that was available on google code (probably you can find it on GitHub) and apply this diff: https://subler.org/downloads/mp42.diff

  3. Priit Varik

    So, I managed to create a quick fix to my primary objective - to inject correct dec3 atom into e-ac3 track. Basically, it allows me to both fix existing MP4 files and convert DD+Atmos tracks from anything Subler is willing to open and convert to MP4 container. The place I found to do this is CFDataRef createCookie_EAC3(void *context) in MP42FormatUtilites.mm To make it proper and universal, several steps are still missing:

    1. understand the semantics of those bytes
    2. parse E-AC3 syncframe for Atmos identification in int analyze_EAC3(void **context ,uint8_t *frame, uint32_t size).
    3. add Atmos flag to struct eac3_info {} ??

    These snippets may be of help:

    https://github.com/mbunkus/mkvtoolnix/commit/498c097bb8389bdd09de76b96aeba358382f7071

    https://github.com/MediaArea/MediaInfoLib/blob/master/Source/MediaInfo/Audio/File_Ac3.cpp

  4. Damiano Galassi repo owner

    I can't find any public available specs on the structure of the eac3 + atmos dec3 atom.

  5. Priit Varik

    Hi Damiano,

    It seems that the MediaInfo app does it best at the moment. Luckily enough, they are also open source. So the startingpoint is hidden here somewhere (am just studying the code) https://github.com/MediaArea/MediaInfoLib/blob/master/Source/MediaInfo/Audio/File_Ac3.cpp

    I already can see that they indeed read the EMDF section in DD+ stream (See ETSI TS 102 366 for details) and determine object count from JOC payload in it (see ETSI TS 103 420).

    The dec3 atom needs to have this: Bytes 0-4 - standard meaning (TS 102 366) Byte 5 needs to be 0 - no dependent substreams Byte 6 needs to be 1 - i have named it Atmos version (current atmos version is defined as 1.0 see the ETSI Revision to RDD 29:2014 Dolby Atmos® Bitstream Specification Byte 7 seems to indicate object count - deductively observed from mp4 demo/test clips from Dolby Labs and comparing to MediaInfo object count display. The value here is always 1 larger than MI display. Eg if MI counts 11 objects, cookie has 0x0c, if 13 objects then 0x0e and if 15 objects then 0x10.

    Also tested: having byte 5 !=0, byte 6 =0 or byte 7 =0 makes tvOS think the stream is just E-AC-3 and play it in PCM either 5.1 or 7.1 (depending on dependent substream and channel location definition in dec3).

    PS is there an easy way to make a diff of my code changes to your git master? Quick look around in the Arlassian Sourcetree app offered only external diff and I’d need to do it manually file-by-file.

    Lugupidamisega/Regards, Priit Varik

  6. Damiano Galassi repo owner

    I usually use the git cli in Terminal. You can move to the project folder and type 'git diff'.

  8. Priit Varik

    Hi Damiano,

    These are the changes I've made so far: MP42Foundation_PV.diff Basically I just hardcode 16 objects and write dec3 atom with 00 01 10 as last 3 bytes. The main thing I can not say just looking at your code yet, is how many AC-3 syncframes do you let Subler scan to build up the necessary attributes?

    I am asking because at this moment, I have no idea, how many syncframes do we need to scan to discover the first JOC description in the stream and whether current Subler code even looks that far. I also think, that it would make sense to write just 1 universal routine to detect Atmos and # of objects directly from the stream (ie 0x0B77 syncframes) so that it could be used in MPEG-TS, MKV and MP4 importers. I am not sure, how easily could this be achieved in current code (just haven't spent time studying it enough).

  10. Priit Varik

    Hi,

    I basically now have the code that determines from bitstream the object count that indicates Atmos presence in E-AC-3 and is needed for dec3 magicCookie's last, 7th byte. It is the object_count /indicates the total number of objects in the bitstream/ element from OAMD payload in EMDF container. Now the main question becomes - as this data may appear quite late in the stream - how to make sure enough frames are scanned in e-ac-3 bitstreams, be it imported from whatever container format?

    Damiano, I hope you can help me out with this?

    ScreenShot.jpg

  11. Damiano Galassi repo owner

    How late? Right now when you open a mkv it reads 1/64 of the eac3 track (line 731 of MP42MKVImporter.m). The more you read the more it will take to open the file, so you have to find a balance between the two.

  12. Priit Varik

    Hi Damiano,

    In the e-ac-3 track of Atmos Amaze trailer i find the first OAMD payload in the 89’th syncframe. On the screenshot attached in my previous post, it appeared only in 427’th syncframe. Considering the total length, it may well fit into 1/64 of the file. I will check the code you ponted at.

    Lugupidamisega/Regards, Priit Varik

  13. Priit Varik

    Hi Damiano,

    Unfortunately my vacation is over and so is my spare time to deal with this in detail. I copy the code that I used to parse the OAMD object count from the e-ac-3 files. I looked at the MP42MKVImporter.m like you suggested, but I do not understand, why not build up all the relevant MP42AudioTrack parameters already while importing? That magicCookieForTrack is only executed, when I save the new file. But because of this, the audio track info is also missing the Atmos info in the FileImportController view.

    Likewise, I could not understand why, using mkv_ReadFrame() and readMkvPacket(), do I see only frames of 6656 bytes in size, consisting of 2 AC-3 syncframes, 2560 and 4096 bytes in size, respectively? Is this MKV-specific?

    I also tried to see, if importing a raw bitstream from .eac3 file behaves any differently, but there I discovered, that it is handled in CoreAudio and does not recognise the e-ac-3 frames embedded inbetween ac-3 frames, so it constructs the parameters and magicCookie as if it were a regular AC-3 file.

  16. Priit Varik

    Damiano,

    Never mind! That code worked for me just by accident! Upon closer inspection, I can not find any auxdata in these frames, and the "hit by accident" occurred always somewhere mid-frame, so I am now trying to dig down to skipfields, which appear in the audio blocks. At least their presence is flagged in the stream, so it seems.

    @ Johnny - the tags are easy part, to discover Atmos' presence in the stream, that is the hard part.

    PS - TrueHD is not supported inside MP4 container yet, so we are talking only about Dolby Digital Plus (E-AC-3 here). That is also the only Atmos stream tvOS 12 supports.

    Next best bet would be to wait until someone (Infuse, Plex, VLC or someone else) manages to bitstream into HDMI, then all of this becomes less important as it is left to your AVR or soundbar to recognise Atmos.

  17. Priit Varik

    Hi Damiano,

    Please check my code and compiled app. I've also added some test clips to test with. You are welcome to test with your own as well. http://varik.servehttp.com/AST/PV_Subler.zip (678MB in size!). If you find this new functionality can be permanently implemented in next Subler release, feel free to use the code. A little credit in the about box would not hurt, if it makes that far :)

  18. Damiano Galassi repo owner

    I changed it a bit and committed it. There were a lot of static analyser warnings in the parser code, I tried to fix them, hopefully it still works and doesn't crash on invalid files.

  19. Priit Varik

    Strange, for me there were only the 2 warnings of unused variables. Unless you mean some additional static analyzer besides what the pre-indexer does.

    PS your project does not compile with Xcode 10 beta 4, tonight I will be able to test if it now does with beta 5. Otherwise I will need to stay on beta 3 until the code is adapted. Beta 4 complained about one class override, if I am not mistaken.

  21. Priit Varik

    Nevermind my beta 4 nagging, just noticed you've commited that change 3 days ago. Will check the analyzer output later in the evening. I left beta 5 downloading this morning, so have not gotten the chance to test it yet. In a nutshell, what my code does is this:

    It carries on at the end of AC3 header analysis with the rest of the bitstream until it finds the skipfield in first audio block, then continues to look for 0x5838 signature from there. I did not want to do the full analysis of audio blocks, as they are of unknown length and above average complexity. Should check if ffmpeg has complete code for that.

    After the skipfield anaysis returns (with whatever results), the auxdata field is also parsed, as this is very deterministic and can be done reading backwards from the end of frame. In all current files I do not see auxdata being used at all. So the EMDF must be located in later audioblocks than the first. More observations - the Amaze Trailer has different encoding than the rest, meaning e-ac-3 frames are embedded as dependent frames inside classic ac-3 frames. Also, the one sample file of mine, without video track, is converted nicely by Subler, but tvOS 12 does not recognise audio-only clips as movies, so it does not consider Atmos at all. It will play as a music track, without Atmos, without drop-down HUD.

  22. Priit Varik

    Damiano,

    First quick tests with a source pulled from your repository gave good results. Will deal with the analyzer warnings in coming days. It would be good to try the behaviour with invalid input files, if I only knew, where to find them. Or do you think I should simply destroy some bytes in existing good files?

    The file import window does not let adjust all columns freely, is it so by design? And does it need to have so strict limits as they are now?

    Screen Shot 2018-08-07 at 20.54.56.jpg

    I also reconsidered the format string and replaced phrase + Atmos with number of objects like this:

    Screen Shot 2018-08-07 at 21.01.56.jpg

    Do you think this is a good idea for average user? Or would they not care and not understand that it is an Atmos track, if not explicitly written out? PS Thank you for the credits!

  23. Damiano Galassi repo owner

    I don't think the average user knows what objects in atmos are… By the way, if you have time can you run it with the address sanitizer too, it was crashing with a out of bound read/write with a normal eac3 audio track.

  24. Priit Varik

    Hi, yes, I will take time to double-check that.

    I will need to look for a longer regular E-AC-3 track than my short test clips.

    BTW what import file did it crash with, MKV or MP4?

  25. Priit Varik

    Hi,

    By the way, if you have time can you run it with the address sanitizer too, it was crashing with a out of bound read/write with a normal eac3 audio track.

    Yes, I could reproduce the issue. It appears that inside audblk parser one iteration limiter ncplsubnd computes as negative (-2 or 0xFE) in this loop trying to access illegal index in cplbndstrc[i] array:

            if (ecplinu == 0) {                         /* standard coupling in use */
            //4.4.3.13 cplbndstrc[sbnd] - Coupling band structure - 1 bit
            //ncplbnd = (ncplsubnd - (cplbndstrc[1] + ... + cplbndstrc[ncplsubnd - 1]))
            int ncplbnd = 0;
            for (int i = 1; i < ncplsubnd; i++) {
                ncplbnd += cplbndstrc[i]; <<Out of bounds index exception!
            }
  26. Priit Varik

    Hi Damiano,

    I actually got rid of both audblk and audfrm parsing altogether and now the code snippet that failed above, is not there at all :)

    You will find the new diff to your latest git commit in this file: http://varik.servehttp.com/AST/MP42Foundation_PV_2.diff

    Also, I cut down the mkv packets read in during file opening to just 5. According to spec, the OAMD and JOC metadata must be present once per frame, so in each and every frame. And they indeed are.

    So I tested with number of atmos and non-atmos E-AC-3 files and could not crash it any more.

    Please try with your own source material as well.

    PS What about the narrow Info column width in the import view? Actually, just removing the spaces on both sides of the + will win space for 2 more letters to be displayed.

  27. Damiano Galassi repo owner

    Thanks, I committed the new patch. Yes I will make the max column width a big larger.

  28. Priit Varik

    Hi Damiano,

    Do you have a proposal on how to extend the Atmos identification beyond mkv files? As far as I can see, the other 3 major possible soirces - .ac3 , .mp4 , .ts - are all implemented in MP42AVFileImporter. And that one uses CoreMedia framework to get main characteristics of a file. One hope could be that Apple will include Atmos recognition in its framework. But if we need to do it ourselves, then how to do it best? PS does CM framework provide hooks to intercept the file processing?

  29. Damiano Galassi repo owner

    You would need to recreate part of the demux method, use it to extract some packets like you do for mkv.

  30. Priit Varik

    Hi Damiano,

    Thanks for releasing the 1.5!

    I found a way to import track from .eac3 file. And one memory leak in previous code eac3_info *context was not freed in MP42KmvImporter,numObjectsForAudioTrack()

    Please find my changes in a diff file. http://varik.servehttp.com/AST/MP42Foundation_PV_3.diff

    I have an impression, that current macOS AVFoundation does not allow to do better than that. It incorrectly characterises AC-3-embedded E-AC-3 substream (like Atmos Amaze trailer) as 'ac-3' and this ruins further processing downstream completely.

    I also do not think trying to create workarounds to Apple framework is a good idea.

    So we need to wait and see if and to what extent will Apple implement Atmos support in AVFoundation or AudioToolbox.

    Current most foolproof workaround - to mux whatever you might have, into MKV and use that as Subler input.

  33. Damiano Galassi repo owner

    I changed it a bit so now QuickTime reports the right channel count, can you check if it's still recognised as atmos on the AppleTV?

  34. Damiano Galassi repo owner

    _Amaze Trailer (16obj-ac3-embedded).mkv num_dep_sub is 32, but in the dec3 atom we have only 4 bit for that. I think either the file is broken or something is wrong in the parser code.

  35. Priit Varik

    Hi Damiano,

    Sorry for late answer, was on business trip for long time, so had no way to concentrate on the issues.

    Does the dec3 creation code needs to be updated?

    Yes, it will make sense to adapt it to official specification. Currently the Atmos-related additions are purely empirical.

    I see you set bsdi to 16 and num_dep_sub to 0 for Atmos. Where did you find this info?

    What do you mean by ‘bsdi’ ? I do not have such field in dec3 magic cookie. But both values have also been just observed in the dolby-created demo mp4 files downloaded from their site. bsid needs to be 0x10 (16) to denote e-ac-3 bitstream, right?

    After the Atmos changes this file is recognised as 5.1 in Subler, before it was 7.1: https://subler.org/downloads/Minions_part.mkv I changed it a bit so now QuickTime reports the right channel count, can you check if it's still recognised as atmos on the AppleTV?

    Let me check, please. It may well be that I will need to correct the num_dep_sub key in dec3 atom. However, I seem to recall from tests, that if they are >0, appleTV does not recognise the stream as Atmos any more. I will have to re-test. The actual channel count within stream remains at 8. Only magic cookie does not contain the dep_sub information and so Subler displays the channel count from that cookie as having 6ch. I will test, if I add these bits and atmos complexity index to dec3 cookie, if tvOS still recognises the track as Atmos track.

    Untitled.jpg

    UPDATE: So, I have tested, and indeed the clip plays in Atmos. Unfortunately my receiver does not indicate the exact channel locations being received.

    _Amaze Trailer (16obj-ac3-embedded).mkv num_dep_sub is 32, but in the dec3 atom we have only 4 bit for that. I think either the file is broken or something is wrong in the parser code.

    This one is the ac-3 embedded flavor of e-ac-3 and I think they all come from the Dolby Atmos 2014 demo disc (https://www.demo-world.eu/2014/09/15/dolby-atmos-demonstration-disc-august-2014/). They may be early encodes, or really broken. But the only conversion I’ve done on my side, was to use MKVToolNix to wrap the streams from m2ts into mkv. As you can see below, MediaInfo does not recognise as a E-AC-3 JOC stream either.

    PastedGraphic-1.jpg

  37. Priit Varik

    Hi Damiano,

    I did a major revision to my code. Now it

    • follows the ETSI TS 103 420 V1.2.1 (2018-10) standard
    • does not look for EMDF stream elements. All necessary info is found in addbsi field, as per spec
    • imports DD+ Atmos tracks correctly also from .MP4 and .EAC3 files, via MP42AVFImporter class
    • retains the 8ch configuration in your test clip Minions_part.mkv

    Please find my changes in a diff file. http://varik.servehttp.com/AST/MP42Foundation_PV_4.diff

    There is also one hack I am not especially proud of - the current (macOS Mojave) CoreMedia Framework creates magic cookie for pure E-AC3 track very nicely. However, it does not understand E-AC3 frames embedded into AC3 frames as substream. For this special case, I created a workaround. in the MP42AVFImporter class' methods initWithURL, magicCookieForTrack and fourCCoverrideForAtmos.

    Please try it out and let me know if it can be committed into git.

  42. Damiano Galassi repo owner

    Thanks for the patch, I committed it. I changed it a bit to avoid having a lot of eac3 specific properties in MP42AudioTrack, hopefully I didn't break anything.

    One thing we could do is to keep each track magic cookie in memory in MP42Track so we don't have to recreate them two times.

  43. Priit Varik

    Hi!

    I notice one strange thing - the 1.5.7 I pulled from your update server, does loose JOC data on the AC3 embedded EAC3 track. After muxing into MP4, it returns to AC3 track with 6ch. If I pull the source from GIT and build it myself - it still works. So I wonder what makes this difference?

    In general, the memory-stored magic cookies sounds as a reasonable idea. However, the Dolby Audio tracks carry minuscule cookies, so the overhead to create them is negligible. Some other tracks, esp video, carry much bigger cookies. There it may pay off.

  48. Jean-Baptiste Joret

    What about tags such as 4K and HDR ? I have an Apple TV 4K and actually make 4K movies, I was hoping to be able to select 4K (maybe 2160p) or something like that in the Tag "HD Video", currently the available options are 720p, 1080p and No. Does Apple m4v not have a standard tag for 4K and HDR ?

  49. Damiano Galassi repo owner

    Apple didn't add such tags yet. Even so I see no reason to add those tags, they can just read the info from the file directly.

  50. Former user Account Deleted

    So you mean if the audio file in the container is Dolby Atmos or the video track is HDR, Apple player will automatically add these tags?

  51. Priit Varik

    Yes, but they appear only in the player HUD, like seen on the screenshot here: https://forums.macrumors.com/threads/dolby-atmos-content-to-test.2124806/page-2#post-26220484 Apple did not add the tags to be shown in your own library's metadata. Probably because at the moment, from their perspective, 4K is only available as iTunes Store streaming and you can not store 4K content from their servers in your library. Obviously, you can add 4K and HDR and Atmos content to your library from other sources, but iTunes Store.

  52. Former user Account Deleted

    Understood. Thank you so much on behalf of all of us for your work and amazing software you made. I really enjoy using it to catogrize my library. Thank you so much brother.

  53. Raphaël Tintine

    Hi everyone!

    Is there something new? (sorry for my English I'm French).

    I would like to know if you can now add HDR tag on 4K movies? And can we now read Dolby Vision movies on the Apple TV 4K through the “Home Sharing” option on Mac.

    if someone know something and can help it will be very helpful.

    Thanks! 🙏

  54. Priit Varik

    Do not know about HDR tag, I think aTV 4K reads it from movie headers.
    Home Sharing still excludes all movies that are in Dolby Vision from the view on aTV 4K. You need to use Infuse client for them.

  55. Raphaël Tintine

    thanks man for you answer (very quick btw)! Ok I think I'm gonna stop looking for Dolby Vision on Home Sharing. But for the HDR tag I remember I saw something but I didn’t find anything on internet, and that is pissed me off! I hope find something one day!

  56. Priit Varik

    Where would you find DoVi content (profile 5, streaming services, useable on appleTV) anyway? Most of it comes from bluray discs and that flavour is not tvOS compatible anyway. So not much use anyway.
    But it is true, that Apple’s iTunes/TV apps hide all content whose HD type has been set to 4K or that does contain dvhe encoded (Dolby Vision) video.

  57. Raphaël Tintine

    To have my 4K movies on my aTV I set the HD video type on 1080p with Subler. And the App Apple TV recognize them as HD movies, but read them in 4K. And with that technique I can see my movies on my aTV using Home Sharing. And then the aTV can also read Dolby Atmos movies with the Home Sharing.

    I have some content in DoVi Profile 5 in mp4. But I think the aTV can only use this codec with the HLS. (use by Apple for the movie and series on Apple TV+). When I download a movie that I bought one iTunes in HDR on my Mac, the file is a .movpkg file.

  58. Edoardo Volpi Kellermann

    Buongiorno Damiano, ho una difficoltà con la applicazione TV - Ex iTunes su Mac OS Big Sur. Uso Subler per integrare i dati e la copertina dei film da me inseriti in TV (da fonti di mia proprietà come DVD o BluRay). Le info vengono regolarmente registrate, così come la cover. Però le cover non vengono mostrare regolarmente: se chiedo le informazioni del film mi mostra la cover che ho inserito ma nell’elenco generale - e sulla mia Apple TV 4K - continua ad apparire la cover vecchia. Capisco che questo è un bug della applicazione TV, ma esiste un modo per ovviare? P.S. ho notato che se le cover le aggiorno manualmente, il problema non sussiste.

  59. Edoardo Volpi Kellermann

    Hello Damiano, I have a difficulty with the TV application - Ex iTunes on Mac OS Big Sur. I use Subler to integrate the data and the cover of the films I put on TV (from sources I own such as DVD or BluRay). The info is regularly recorded, as well as the cover. However, the covers are not shown regularly: if I ask for information about the film, it shows me the cover I entered but in the general list - and on my Apple TV 4K - the old cover continues to appear. I understand this is a bug in the TV application, but is there any way around it? P.S. I noticed that if I update the covers manually, the problem does not exist.

  60. Steven Mancera

    Home Sharing still excludes all movies that are in Dolby Vision from the view on aTV 4K. You need to use Infuse client for them.

    @Priit Varik What do you mean exclude? Like straight up yank it from ever showing on the TV macOS app, even if you drag it / import it to be included on your Library? And on Home Sharing app on Apple TV 4K? Like don’t display it? Or only not show the Dolby Vision tag on the Library?

  61. Priit Varik

    Hi Steven

    You will see them on your Mac and the iTunes/TV app will ingest them happily.

    It is the Computers app on tvOS, that will still hide them.

    Incidentally, now that HD mode also has finally value to indicate (4K) in addition to already known 1080p and 720p. Just this value HD=4K is sufficient for Computers app to hide your clip. So I tag all my 4K movies as 1080p to keep them visible on apple TV.

  63. Joseph Sayegh

    I know that the Computers app is still hiding content tagged as 4K. As a workaround, I’m marking all my 4K content as 5 stars in iTunes/TV app. Then I’m creating a smart playlist that shows all my content which is rated as 5 stars. That shows me all my 4K content on the Apple TV computes app.

  64. Log in to comment
Assignee
–
Type
enhancement
Priority
minor
Status
closed
Votes
1
Watchers
8
Jira: the preferred issue tracker for Bitbucket. Join the team!