MP4Box incompatibility,...

[Selur]

Trying to create some HRD content I ran into an issue with MP4Box and from the gist of it, it seems to be a problem with the stream produced by the encoder. see: https://github.com/gpac/gpac/issues/705

Would be nice if this could be fixed.

[Pradeep Ramachandran]

If I understand the issue, this is the issue that you are facing?

The file is imported as hev1 because a PPS with the same id but different is encountered. Did you try to contact the x265 dev? I know there is also another open issue about initial data being sent by the encoder being set at default values - this may be related.

We use id 0 for the VPS, SPS, and PPSes that we generate. Are you recommending a change in this?

[Selur]

I can't really recommend anything. :) I guess it would be a good idea to exchange with the GPAC team.

[rbouqueau]

Hi, GPAC maintainer here. Our HEVC parser parses two SPS with ID 0 and respective sizes 8 and 9 bytes (bitstream position in byte resp. 103 and 308966) on the data provided by @4selur on our tracker.

SPS[0@103]:44 01 c1 72 8a 55 62 40 SPS[1@308966]: 44 01 c1 72 8a 55 62 40 00

This is not an issue per-se for MP4Box. But it prevents some type of storage ('hvc1'). Let me know what you think.

[Bertrand LORMEAU]

Hi, After trying different builds X265 10bits, it seems that regression appears as of 2016/09/29. Builds after this have the bug.

x265-64bit-10bit-2016-09-28.exe --version x265 [info]: HEVC encoder version 2.1+5-9373283232a0 x265 [info]: build info [Windows][GCC 5.3.1][64 bit] 10bit x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX

Hopefully this will help

[Selur]

any news on this? thought it would be easy and fast to fix since it seemed like a regression,...

[Pradeep Ramachandran]

Sorry, got caught up with several other issues. Lets re-open this and fix before we tag 2.2 which should be happening imminently.

If I understand the last couple of posts, the issue is that the size of the SPS from one SPS to another? And this issue happens on builds on the default branch after changeset 9373283232a0? We haven't made any changes to change the content of the SPS afaik.

[Selur]

rbouqueau can probably elaborate on that. I ran into this issue when I encoded a bunch of files and tried to mux each of them using MP4Box. From what I got reading the posts in the gpac tracker the issue was that two pps are identical,...

[Pradeep Ramachandran]

Can you try adding the following options to your command-line: --no-opt-qp-pps --no-opt-ref-list-length-pps?

[Selur]

using:

x265 --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 5.2 --limit-modes --no-open-gop --lookahead-slices 0 --bitrate 15000 --crf-min 0.00 --crf-max 0.00 --cbqpoffs -2 --crqpoffs -2 --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 15.00 --aq-mode 2 --cu-lossless --vbv-maxrate 60000 --vbv-bufsize 60000 --hrd --aud --repeat-headers --range limited --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --max-cll "1000,200" --output "H:\Temp\08_48_54_7410_01.265"

crashes

adding '--no-opt-qp-pps':

x265 --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 5.2 --limit-modes --no-open-gop --lookahead-slices 0 --bitrate 15000 --crf-min 0.00 --crf-max 0.00 --no-opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 15.00 --aq-mode 2 --cu-lossless --vbv-maxrate 60000 --vbv-bufsize 60000 --hrd --aud --repeat-headers --range limited --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --max-cll "1000,200" --output "H:\Temp\08_53_03_6010_01.265"

crashes

adding '--no-opt-ref-list-length-pps':

x265 --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 5.2 --limit-modes --no-open-gop --no-opt-ref-list-length-pps --lookahead-slices 0 --bitrate 15000 --crf-min 0.00 --crf-max 0.00 --cbqpoffs -2 --crqpoffs -2 --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 15.00 --aq-mode 2 --cu-lossless --vbv-maxrate 60000 --vbv-bufsize 60000 --hrd --aud --repeat-headers --range limited --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --max-cll "1000,200" --output "H:\Temp\08_59_10_5710_01.265"

crashes

adding '--no-opt-qp-pps' and '--no-opt-ref-list-length-pps':

x265 --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 5.2 --limit-modes --no-open-gop --no-opt-ref-list-length-pps --lookahead-slices 0 --bitrate 15000 --crf-min 0.00 --crf-max 0.00 --no-opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 15.00 --aq-mode 2 --cu-lossless --vbv-maxrate 60000 --vbv-bufsize 60000 --hrd --aud --repeat-headers --range limited --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --max-cll "1000,200" --output "H:\Temp\08_57_03_6510_01.265"

works. (btw. if it is of any help I could upload the source file I use to my google drive and share it)

[Pradeep Ramachandran]

When you say crashes, do you mean that the encode is crashing, or the encoded HEVC file isn't working with MP4Box? I am hoping it is crashing with MP4Box and not an x265 crash :-).

And if using these two options works, then the issue is with your PPS. Maybe MP4Box is expecting 'PPS'es of identical content every GOP. These two options modify the contents of the PPS based on measurements of QP and ref-list-length done in the previous GOP.

[Selur]

the encoded HEVC file isn't working with MP4Box. the raw output of x265 can be played back. Can't really comment on what MP4Box is expecting, but rbouqueau probably can. :)

[Pradeep Ramachandran]

If with these two cli options, the HEVC can be viewed with MP4Box, can we close this issue? Having optimized PPS for every GOP isn't a bug, but a feature :-).

[Selur]

before closing it might be good to wait for a reply from the MP4Box team to be sure that adding that feature isn't breaking something,... especially since that feature is enabled by default. (I'm not that knowledgeable with the PPS structure to tell)

[rbouqueau]

@pramach2 I think our projects already works well together :)

1) For me the root of https://github.com/gpac/gpac/issues/705 is that MP4Box is not able to understand if it is the same SPS or not (as there is only an additional 0 byte and same SPS ID). Any advice here on a more advanced check is welcome ; since you use the same SPS ID, may we ignore it safely? Any streams are also welcome for our regression test base.

2) The use case for end users is: if we can factor SPSs, then SPSs are not store inband (i.e. with the frames) but outband (i.e. once in a header). When once in a header, we allow some modifications such as the PAR that we rewrite in the SPS. These operations are quite common and popular within the community. At this point, users such as @4selur see random failures for that.

I think we all care about our users. What do 1) and 2) inspire you?

[Selur]

Yes, the randomness is an annoying thing, for me and other. (took me a while to reproduce the issue since it was first reported to me from a user which couldn't provide a sample and the tests I ran here all didn't allow me to reproduce the issue. So when another user reported the issue and mentioned that he only encounters the problems on some files and could provide a sample I finally could trace this at least a bit) For some files when they are reencoded with the above settings, the x265 output can't be handled with MP4Box and with others the x265 output works fine. So atm. as user it seems that one has to use --no-opt-qp-pps --no-opt-ref-list-length-pps since without it one seemingly randomly gets output which can't be multiplexed with MP4Box. :( -> hoping that either gpac or x265 can be 'fixed' :)

[Pradeep Ramachandran]

@rbouqueau - The two options that I listed to @4selur that is making the generated streams work with MP4Box modify only the PPS and not the SPS; the SPS will be identical for every GOP. (We send SPS and PPS NALs at the start of every GOP if --repeat-headers is enabled, as with @4selur 's command-lines. Are you sure that the issue you're seeing is to do with the SPS and not the PPS?

@4selur - is adding these two options for all HEVC streams created from x265 to be used with MP4Box an option? For regular encodes, we see bitrate savings from enabling these two options by default and therefore don't want to disable them.

[Selur]

is adding these two options for all HEVC streams created from x265 to be used with MP4Box an option?

I could live with it, though I would prefer if either MP4Box or x265 could be adjusted so one could use those options together with MP4Box to get the bitrate savings. ;)

[rbouqueau]

@4selur Is there a specific reason for adding --repeat-headers when encoding? IIUC without this option we wouldn't have any problem.

[Selur]

I normally add it so that I can cut the stream easier if need be. :)

[rbouqueau]

Ok, that raises a lot of new questions for me :)

Does it mean that you cut raw streams? What's the use-case? Why not put in a MP4 and then manipulate the data?

[Selur]

Cutting and anlysing raw streams is the reason for the '--repeat-headers' option for me. Select start&end frame, calculate the GOP at the start&end of the cut, extract both, analyse the extracted GOPs, reencode the GOPs if need be,... is a part of the process I do for frame accurate cutting. :) I assumed '--repeat-headers' is basically the same as x264s '--stitchable',...

So you say that: dropping the repeat headers option would be an alternative to using '--no-opt-qp-pps --no-opt-ref-list-length-pps' ?

[Selur]

Just tested:

x265 --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 5.2 --limit-modes --no-open-gop --lookahead-slices 0 --bitrate 15000 --crf-min 0.00 --crf-max 0.00 --cbqpoffs -2 --crqpoffs -2 --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 15.00 --aq-mode 2 --cu-lossless --vbv-maxrate 60000 --vbv-bufsize 60000 --hrd --aud --range limited --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt709 --max-cll "1000,200" --output "H:\Temp\test.265"

and indeed MP4Box had no problem with the output.

[rbouqueau]

So you say that: dropping the repeat headers option would be an alternative to using '--no-opt-qp-pps --no-opt-ref-list-length-pps' ?

Probably, I would need a stream to confirm. Could you encode the stream that caused the issue with the different settings? PS: seen your test, thank you :)

Cutting and anlysing raw streams is the reason for the '--repeat-headers' option for me. Select start&end frame, calculate the GOP at the start&end of the cut, extract both, analyse the extracted GOPs, reencode the GOPs if need be,... is a part of the process I do for frame accurate cutting. :) I assumed '--repeat-headers' is basically the same as x264s '--stitchable',...

The MP4 fie format explicitly deals with different xPSs of same ID. So in theory it is the responsibility of the muxer to ensure xPSs with the same ID are identical. However that's insanely difficult to achieve this semantically so serious muxers only check they are syntactically identical. It seems like a reasonable assumption, what do you think?

Obviously we could add a flag in MP4Box to ignore the next SPSs/PPSs but what good would that do? It would be a x265-only flag and wouldn't solve the issue for all the other muxers (mp4, mkv, etc.). I hope @pramach2 or any x265 contributor would come up with a better idea. I understand the optimization but it may break much more than expected.

[Selur]

For future tests, I uploaded the source I used to my google drive: https://drive.google.com/drive/folders/0B_WxUS1XGCPASUZibG5XZkRfeTg?usp=sharing (HDR-Rheinschiff_Exc01_ProResHQ.mov) Also here's a complete call, like I would use. (this is the working one without the --repeat-headers)

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\ProRes\HDR-Rheinschiff_Exc01_ProResHQ.mov" -map 0:0 -an -sn -vsync 0 -strict -1 -pix_fmt yuv420p10 -f yuv4mpegpipe - | x265 --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 5.2 --limit-modes --no-open-gop --lookahead-slices 0 --bitrate 15000 --crf-min 0.00 --crf-max 0.00 --cbqpoffs -2 --crqpoffs -2 --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 15.00 --aq-mode 2 --cu-lossless --vbv-maxrate 60000 --vbv-bufsize 60000 --hrd --aud --range limited --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt709 --max-cll "1000,200" --output "H:\Temp\19_04_44_2610_01.265"

and an example of the muxing call:

MP4Box -par 1=1:1 -add "H:\Temp\19_04_44_2610_01.265"#video:fps=25 -brand hvc1 -tmp "H:\Temp" -new "H:\Output\19_04_44_2610__02.mp4"

Would be nice if one wouldn't have to keep in mind that one should use either '--opt-qp-pps --opt-ref-list-length-pps' or '--repeat-headers' to avoid problems. :) Personally I'm fine with dropping one of them for my normal processing passes, but to avoid that other users run into this problem too, a 'fix' of some sort would be nice.

[Pradeep Ramachandran]

@rbouqueau - according to the HEVC spec, xPSes with the same ID are said to be identical only if the no_parameter_set_update_flag is set in the active parameter sets SEI message (D.3.20 in the spec). Else, newer xPSs (VPS/SPS/PPS) with the same ID over-ride older xPSes with the same ID. x265 is therefore spec compliant.

I think you guys have to fix this in your muxer.

[rbouqueau]

MP4Box is also compliant since it pushes the xPSs inband. I'm just trying to find a workable issue for our users :)

My best option right now, if users see this optimization by default, is to add a warning and a flag to explicitly ignore xPS updates. @4selur What do you think?

[Selur]

As long as MP4Box doesn't abort the muxing I would be happy with a warning and if I would have to add an additional muxing option, that wouldn't be an issue either. :)

[Pradeep Ramachandran]

@rbouqueau If these options are on (--opt-qp-pps and --opt-ref-list-length-pps), then ignoring the new PPS isn't correct although it has the same id as the updated values in the PPS are required for functional correctness.

We are also working on a patch to update ids of xPS when the flags change the contents of the PPS. It may be a few days before this patch comes in though - please stay tuned.

[Pradeep Ramachandran]

@4selur / @rbouqueau - We implemented changing the set_id for PPS and SPS whenever the contents of set_id are changed and it looks like MP4Box doesn't support warp-around when we over-run the 4 bits allowed in the spec for their ids! Also, we exchanged notes with other integrations of x265, and it looks like their application is able to handle the changed xPSes the way they are right now. Given this, I would recommend that you fix this in your MP4Box integration; ensuring to add --no-opt-qp-pps --no-opt-ref-list-length-pps and emitting a warning if it isn't added might be the simplest for you.

[rbouqueau]

Thank you for your efforts. We are getting closer to an issue on this :) However your last message raises some questions for me:

We implemented changing the set_id for PPS and SPS whenever the contents of set_id are changed

Have you changed the default behaviour of x265, or is it related to an option?

we exchanged notes with other integrations of x265, and it looks like their application is able to handle the changed xPSes the way they are right now.

2 questions: - Are you talking about the same or some changed set_id? - How do they handle that? By any chance do you have any stream? It would be ok for us to mimic what's already available if it can help the community.

it looks like MP4Box doesn't support warp-around when we over-run the 4 bits allowed in the spec for their ids!

Thanks for reporting but I don't understand what it means. What do you see? Could you provide me a way for us to reproduce?