vbv bugs: mosaic blocky effect in some frames

[shinchiro]

I run quality test with this sample: http://a.pomf.hummingbird.moe/rbfahx.mkv

Result:

--crf 20  --vbv-maxrate 2000 --vbv-bufsize 2000 --preset medium --rd 6

^All encoded frames are good

--crf 20 --ctu 32 --max-tu-size 16 --vbv-maxrate 2000 --vbv-bufsize 2000 --preset medium

^Only frame 32 showed weird mosaic block

--crf 20 --ctu 32 --max-tu-size 16 --vbv-maxrate 2000 --vbv-bufsize 2000 --preset medium --rd 6

^Both frame 32 & 33 showed more weird mosaic block. Strange the only different is it just using rd 6

here's the screenshot comparison: http://screenshotcomparison.com/comparison/153662

[shinchiro]

The quality is a bit better after I apply this patch: https://mailman.videolan.org/pipermail/x265-devel/2015-December/009903.html

[Deepthi Nandakumar]

Thanks for the report. What is the average bitrate when you use crf 20, without VBV?

The reason why I ask is, without VBV, if CRF 20 gives an average of, say 4000 kbps, then crf 20 + VBV-2000 will give you significantly lower quality. You may be better off, raising the CRF (to roughly match an average bitrate of 2000) and then applying the vbv-2000 parameters.

[shinchiro]

Ok, I re-run with --crf 25 since it give avg bitrate ~2300kbps. The result is there is no visible block. But when I increase --rc-lookahead to 90, the ugly blocks appear again.

[shinchiro]

After a few more tests, I guess this is because the vbv buffer size is too small to receive sudden bitrate spike and overflow it. Increasing --vbv-bufsize to 4000 solved the problem

[Deepthi Nandakumar]

As a developer, I always assume that the user knows what VBV constraints they need to meet (dictated by hardware, software, transmission bandwidth etc etc).

In case you do not have any VBV constraints at all (offline encoding for movie storage etc), then the best quality will be achieved with CRF and no VBV. If you're flexible about VBV params, but do not have fixed numbers in mind, then larger VBV buffer sizes/maxrates will always give you better ratecontrol/quality.