x265_encoder_reconfig: output out of spec if modifying bEnableAMP

[Martin Belleau]

When decoding the output if ffmpeg, it gives out errors such as:

[hevc @ 000001fa26f388c0] The cu_qp_delta 66 is outside the valid range [-26, 25]

but with different values for cu_qp_delta. Visually, there is heavy blocking and frame re-ordering when played back in VLC.

This happens when reconfiguring the settings while encoding, from slow preset, to slower preset, and back to slow preset. This makes bEnableAmp go from 0, to 1, and back to 0. If I modify the code to not change this value when doing x265_encoder_reconfig (and changing all other preset related values), then the output has no artifact.

This happens with the 3.2.1 version (not sure why I can’t select it here - latest version if 1.9) and have been happening since 3.0.

I don’t have an easy way to reproduce, since x265.exe doesn’t have a way to use x265_encoder_reconfig. I am seeing this is our own application. Hopefully you have a test harness for x265_encoder_reconfig and can reproduce there.

‌