Poor fluidity / loss of details when transitioning between dark and light scenes

[Former user]

[Please excuse the rather feeble attempt at describing this from the title; I'm certain there is a much better word for what I am experiencing. I also did quite a number of searches to see whether something similar had been reported before, but came up empty.]

First of all, x265 is shaping up to be an awesome codec; I've seen siginificant reductions at high crf values (18-19) when compared to x264 counterparts using crf 17-19, while retaining quality in most cases. Many files which were uncompressable (at decent quality settings) with x264 achieve hugely respectable 50-60% reductions with x265.

However, an issue that has persevered up until the most recent git build tested (1.6+381-f32e6464225afa02) is that certain scenes which transition from dimly-lit to better-lit (or from dark to light, i.e, when flicking a switch) make the areas that have light and those who have not more pronounced (i.e introducing an artifical boundary)

I have included two clips, the first is the full-quality h264 original, and the second is a re-encoded x265 clip that should explain this better than what I tried to do above. ;)

x265 encoding settings used (different settings, particularly for deblock, have also been tested): wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=3 / subme=3 / merange=57 / rect / no-amp / max-merge=3 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=25 / lookahead-slices=0 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=3 / weightp / no-weightb / aq-mode=1 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=4 / psy-rd=0.30 / rdoq-level=2 / psy-rdoq=1.00 / signhide / deblock=-3:-3 / sao / no-sao-non-deblock / b-pyramid / cutree / rc=crf / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

Original sequence: https://www.dropbox.com/s/fjlwgllry6vanvb/mulholland.drive.test.mkv?dl=0 (x264 encode is also unaffected by the x265-introduced artifacts)

X265: https://www.dropbox.com/s/zl932m52saoak1i/mulholland.drive.test-crf18-slow-hevc.mkv?dl=0

[Deepthi Nandakumar]

Hi,

Sorry for the late response. This is a grainy video, and using tune grain will help significantly. In fact, we're testing to see if turning off AQ altogether lends to better visual quality compared to x264.

[Former user]

Thanks for looking into this.

I did try setting -tune grain and -tune fastdecode after I posted the issue (in addition to trying various deblock settings), but the result unfortunately was the same. Notice the rather extreme difference between the x264 encode and this. To better see the artifacts, switch the HDMI color range to limited/16-235 (or whichever setting is the brightest.)

btw., I have encoded significantly grainier films than this using x265, which for the most part looked excellent. But some of these also had flaws in scenes where the light transitioned from dark to light (and vice-versa), so it appears that the light fluctuations is not handled well currently by the encoder (or there is something very wrong with the default settings)

I can post another clip later (from a different source) if it is of interest.

[Former user]

Do you have any findings to report from the tests? I looked at the logs for the post-1.7 development releases, but did not see changes which appeared to be relevant for this issue.

[Deepthi Nandakumar]

Hi, we'd done some testing on this, and setting --aq-strength 0.3 alone, seemed to improve the quality considerably. Can you check and see if you can replicate this?

[Former user]

Unsure if my previous comment was noticed, but I did test aq-strength=0.3 earlier (as it's provided with the -tune grain setting). It had no effect on this issue. To verify I did two new encodes, one of which only modified the aq-strength parameter (with the other settings left at default values) to re-test with the 1.7 branch, but the result was the same.

Since you are describing the quality gains as 'considerable' I wonder if we are looking at the same thing? The issues I am having are the blockiness in areas in-between the lit and unlit scenes. They are very prominent. Some monitors hide these effects, which is why I suggested to adjust the color space to the brightest mode.

Full encoding settings used, for reference:

Writing library : x265 1.7+2-d7b100e51e828833:[Mac OS X][clang 6.0.0][64 bit] Encoding settings : wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=3 / subme=3 / merange=57 / rect / no-amp / max-merge=3 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=25 / lookahead-slices=0 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / weightp / no-weightb / aq-mode=1 / aq-strength=0.30 / cbqpoffs=0 / crqpoffs=0 / rd=4 / psy-rd=0.30 / rdoq-level=2 / psy-rdoq=1.00 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / rc=crf / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

[Former user]

Hi everyone,

First off I don't have anything to do with x265 in a professional way. I'm just a guy who's quite interested in keeping tabs on its development.

(Dear) Drag0nFly, I have a feeling the setting --qcomp might be of interest to you. Could you perhaps run a test to see if raising this value (to say, 0.70) helps to do away with the artifacts you see?

Sorry to waste your time if this setting would turn out to be of no help, but of course I hope it will be.

Kind regards, Steven de Jong

[Former user]

Not a problem Steven; I'll run a lest with qcomp=0.7 later today to compare and post the results. Thanks for your interest in this issue.

[Former user]

@Steven–Not much difference, sorry. :( Might have had a marginal impact, but the artifacts are very much still there. It did increase encoding time quite significantly, though ;) (from ~2.8fps to ~0.8); initially leading me to believe it accomplished something more than what the prior encodes did.

[Former user]

Aw, that's a shame..

I assume the encoding time rose so much because a higher qcomp significantly increases the file size (at the same crf), meaning more bytes that need to be written (but now I'm just repeating what I read in the forums :p )

Thank you for trying this and I hope you guys manage to find the real solution ;)

[Deepthi Nandakumar]

Drag0nFly,

We started over on this, and with the monitor set to maximum brightness level, but the difference between the original and x265 encode is reduced grain level and very slight blurring. I was not able to see any blockiness problems between lit and un-lit areas. In the clips where "Mulholland Dr." comes up, there is patchiness between the 2 D's and beneath lho - but this is present in the source as well. In short, we're not seeing the same things. Exactly, which frame did you see a problem?

Also, the resolution of your encode has been changed from 1080p to 1904 x 1024?

[Former user]

@nandaku2 Just to clarify, the brightness setting referred to the HDMI color space (normal vs. expanded / video vs. pc-level), just so that we're on the same page. :)

Also, which player (+ GPU) are you testing this with? I am experiencing issues throughout the entire encode when using recent git versions of VLC, mplayer, MplayerX and Kodi–all of which play other HEVC material fine. Same issue on my projector and two lcd monitors as well, and on two different OS-es (OSX & Linux). This is using Intel drivers (HD4000 & HD5100.)

It is probably easier for me to post a video clip of the playback in the hopes that it can better illustrate the issues I am seeing. I can do this later today, and make it available if it indeed is able to capture the artifacts.

Regarding the resolution change: the movie is cropped to its original aspect ratio (fairly common practice), so that the resulting file is 1.85:1 vs. 16:9/1.77:1

[Deepthi Nandakumar]

Yes, the brightness is set to the max HDMI color space (on mine, it's called 32-bit true color) though I can't claim to be an expert in this area so maybe there's something missing here.

I used an in-house developed HEVC player + Intel HD 4600, and also verified this using a YUV player (after decoding your x265 encode).

Thanks, the video clip should be useful.

[Former user]

Sorry for the delay, I did not have time to capture this yesterday – but I've now made two clips. The quailty is not the best, but it does show the artifacts. Also excuse the angle/framing, I did this in a hurry and had issues with focus unless it captured parts of the monitor edge and I needed to tilt the camera slightly.

1. Camera capture of VLC (Version 3.0.0-git Vetinari (Intel 64bit)) on OSX 10.9 playing an x264 encode of the same segment I provided earlier:

=> https://www.dropbox.com/s/j8bmvqvfsxvrblt/mulholland.drive.camera.x264.mkv?dl=0

2. Camera capture of the x265 encode from the same system in an identical setup (i.e, using VLC as the player):

=> https://www.dropbox.com/s/kcvqb8fpf0zh44v/mulholland.drive.camera.x265.mkv?dl=0

As mentioned before, various other players have also been tested, and they all show the same issues (under OSX & Linux), with different drivers & versions on two monitors and a projector.

Unfortunately I did not have time to capture the x265 encode from the projector, but can do so over the weekend. It is running on a completely different setup, off of a HD4600 GPU (Intel(R) Ivybridge Desktop (GT2)) running Linux and Kodi.

Hope the videos prove useful.