2pass encode worse than just the 1st pass or crf encoding,...

Issue #262 resolved
Selur created an issue

got a source with 2334 frames and a frame rate of 60,1fps Using 2pass encoding and a target bit rate of 60kbit/s the output end up with an average quantizer of 49.07 and an average bit rate of 48.37kBit/s

for the decoding I used:

ffmpeg -y -loglevel fatal -threads 8 -i "C:\Users\Selur\Desktop\ng2.avi" -map 0:0 -an -sn  -vf crop=256:224:0:8 -r 6010000/100000  -strict -2 -pix_fmt yuv420p10  -strict -2 -f yuv4mpegpipe - 

First Pass:

"G:\Hybrid\x265.exe" --pmode --pme --input - --output-depth 10 --y4m --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --merange 100 --max-merge 5 --no-open-gop --keyint -1 --bframes 16 --weightb --b-intra --rc-lookahead 80 --pass 1 --bitrate 60 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --stats "H:\Temp\test_19_38_40_1110_01.stats" --output NUL
y4m  [info]: 256x224 fps 601/10 i420p10 unknown frame count
raw  [info]: output file: NUL
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97c7ec
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [warning]: Limit reference options 2 and 3 are not supported with pmode. Disabling limit reference
x265 [info]: Main 10 profile, Level-2 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(7 rows)+pmode+pme
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : dia / 100 / 2 / 1
x265 [info]: Keyframe min / max / scenecut       : disabled
x265 [info]: Lookahead / bframes / badapt        : 80 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 1 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.5 / 32 / 1
x265 [info]: Rate Control / qCompress            : ABR-60 kbps / 0.60
x265 [info]: tools: rd=2 psy-rd=2.00 rdoq=2 psy-rdoq=25.00 early-skip signhide
x265 [info]: tools: tmvp b-intra fast-intra strong-intra-smoothing deblock sao
x265 [info]: tools: stats-write
x265 [info]: frame I:      1, Avg QP:31.99  kb/s: 530.80
x265 [info]: frame P:    312, Avg QP:49.83  kb/s: 107.88
x265 [info]: frame B:   2021, Avg QP:50.71  kb/s: 50.34
x265 [info]: Weighted P-Frames: Y:0.6% UV:0.3%
x265 [info]: Weighted B-Frames: Y:0.9% UV:0.6%
x265 [info]: consecutive B-frames: 2.6% 4.2% 1.0% 6.7% 7.3% 31.3% 8.3% 2.2% 6.7% 18.8% 1.6% 0.3% 1.0% 4.5% 0.3% 0.0% 3.2%
encoded 2334 frames in 7.29s (320.30 fps), 58.23 kb/s, Avg QP:50.59
process finished with exitcode: 0 and exitStatue: 0
finished after 00:00:07.386

2nd pass:

G:\Hybrid\x265.exe" --pmode --pme --input - --output-depth 10 --y4m --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --me 3 --subme 5 --merange 100 --rect --amp --max-merge 5 --no-open-gop --keyint -1 --bframes 16 --weightb --b-intra --ref 16 --rc-lookahead 80 --pass 2 --bitrate 60 --rd 6 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --stats "H:\Temp\test_19_38_40_1110_01.stats" --output "H:\Temp\19_38_40_1110_02.265"
y4m  [info]: 256x224 fps 601/10 i420p10 unknown frame count
raw  [info]: output file: H:\Temp\19_38_40_1110_02.265
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97c7ec
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [warning]: Limit reference options 2 and 3 are not supported with pmode. Disabling limit reference
x265 [warning]: level 2.1 detected, but NumPocTotalCurr (total references) is non-compliant
x265 [info]: NONE profile, Level-NONE (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(7 rows)+pmode+pme
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 100 / 5 / 5
x265 [info]: Keyframe min / max / scenecut       : disabled
x265 [info]: Lookahead / bframes / badapt        : 80 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 16 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.5 / 32 / 1
x265 [info]: Rate Control / qCompress            : ABR-60 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=25.00 signhide
x265 [info]: tools: tmvp b-intra strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I:      1, Avg QP:47.84  kb/s: 70.20
x265 [info]: frame P:    312, Avg QP:47.18  kb/s: 129.95
x265 [info]: frame B:   2021, Avg QP:49.37  kb/s: 35.76
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: Weighted B-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 2.6% 4.2% 1.0% 6.7% 7.3% 31.3% 8.3% 2.2% 6.7% 18.8% 1.6% 0.3% 1.0% 4.5% 0.3% 0.0% 3.2%
encoded 2334 frames in 167.47s (13.94 fps), 48.37 kb/s, Avg QP:49.07

Just seeing that I could accept that x265 thinks it can't hit the desired average bit rate, but now comes the 'funny' thing: Using basically the same settings, but with -crf 38 instead of --pass 2 --bitrate 60, I end up with:

"G:\Hybrid\x265.exe" --pmode --pme --input - --output-depth 10 --y4m --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --me 3 --subme 5 --merange 100 --rect --amp --max-merge 5 --no-open-gop --keyint -1 --bframes 16 --weightb --b-intra --ref 16 --rc-lookahead 80 --crf 38.00 --rd 6 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --output "H:\Temp\19_51_35_4910_01.265"
y4m  [info]: 256x224 fps 601/10 i420p10 unknown frame count
raw  [info]: output file: H:\Temp\19_51_35_4910_01.265
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97c7ec
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [warning]: Limit reference options 2 and 3 are not supported with pmode. Disabling limit reference
x265 [warning]: level 2.1 detected, but NumPocTotalCurr (total references) is non-compliant
x265 [info]: NONE profile, Level-NONE (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(7 rows)+pmode+pme
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 100 / 5 / 5
x265 [info]: Keyframe min / max / scenecut       : disabled
x265 [info]: Lookahead / bframes / badapt        : 80 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 16 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.5 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-38.0 / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=25.00 signhide
x265 [info]: tools: tmvp b-intra strong-intra-smoothing deblock sao
x265 [info]: frame I:      1, Avg QP:31.87  kb/s: 308.19
x265 [info]: frame P:    312, Avg QP:44.06  kb/s: 164.51
x265 [info]: frame B:   2021, Avg QP:48.57  kb/s: 43.10
x265 [info]: Weighted P-Frames: Y:0.6% UV:0.3%
x265 [info]: Weighted B-Frames: Y:0.9% UV:0.6%
x265 [info]: consecutive B-frames: 2.6% 4.2% 1.0% 6.7% 7.3% 31.3% 8.3% 2.2% 6.7% 18.8% 1.6% 0.3% 1.0% 4.5% 0.3% 0.0% 3.2%
encoded 2334 frames in 150.85s (15.47 fps), 59.44 kb/s, Avg QP:47.96
process finished with exitcode: 0 and exitStatue: 0
finished after 00:02:30.949
Created H:\Temp\19_51_35_4910_01.265 (0.28511 MB)

=> seems like there is something amiss with 2pass rate control.

Okay, there are two things a bit uncommon in my calls: a. --ref 16, which makes the output not accepable to any current H.265 level b. --keyint -1

So using --ref 6 and --keyint 5000, I still end up with: First pass:

"G:\Hybrid\x265.exe" --pmode --pme --input - --output-depth 10 --y4m --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --me 3 --subme 5 --merange 100 --rect --amp --max-merge 5 --no-open-gop --keyint 5000 --bframes 16 --weightb --b-intra --ref 6 --rc-lookahead 80 --pass 2 --bitrate 60 --rd 6 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --stats "H:\Temp\test_19_59_48_0510_01.stats" --output "H:\Temp\19_59_48_0510_02.265"
y4m  [info]: 256x224 fps 601/10 i420p10 unknown frame count
raw  [info]: output file: H:\Temp\19_59_48_0510_02.265
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97c7ec
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [warning]: Limit reference options 2 and 3 are not supported with pmode. Disabling limit reference
x265 [info]: Main 10 profile, Level-2 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(7 rows)+pmode+pme
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 100 / 5 / 5
x265 [info]: Keyframe min / max / scenecut       : 60 / 5000 / 40
x265 [info]: Lookahead / bframes / badapt        : 80 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 6 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.5 / 32 / 1
x265 [info]: Rate Control / qCompress            : ABR-60 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=25.00 signhide
x265 [info]: tools: tmvp b-intra strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I:      1, Avg QP:47.84  kb/s: 70.20
x265 [info]: frame P:    312, Avg QP:46.46  kb/s: 143.55
x265 [info]: frame B:   2021, Avg QP:49.10  kb/s: 34.05
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: Weighted B-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 2.6% 4.2% 1.0% 6.7% 7.3% 31.3% 8.3% 2.2% 6.7% 18.8% 1.6% 0.3% 1.0% 4.5% 0.3% 0.0% 3.2%
encoded 2334 frames in 75.98s (30.72 fps), 48.70 kb/s, Avg QP:48.75
process finished with exitcode: 0 and exitStatue: 0
finished after 00:01:16.146

and 2nd pass:

"G:\Hybrid\x265.exe" --pmode --pme --input - --output-depth 10 --y4m --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --me 3 --subme 5 --merange 100 --rect --amp --max-merge 5 --no-open-gop --keyint 5000 --bframes 16 --weightb --b-intra --ref 6 --rc-lookahead 80 --pass 2 --bitrate 60 --rd 6 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --stats "H:\Temp\test_19_59_48_0510_01.stats" --output "H:\Temp\19_59_48_0510_02.265"
y4m  [info]: 256x224 fps 601/10 i420p10 unknown frame count
raw  [info]: output file: H:\Temp\19_59_48_0510_02.265
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97c7ec
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [warning]: Limit reference options 2 and 3 are not supported with pmode. Disabling limit reference
x265 [info]: Main 10 profile, Level-2 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(7 rows)+pmode+pme
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 100 / 5 / 5
x265 [info]: Keyframe min / max / scenecut       : 60 / 5000 / 40
x265 [info]: Lookahead / bframes / badapt        : 80 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 6 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.5 / 32 / 1
x265 [info]: Rate Control / qCompress            : ABR-60 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=25.00 signhide
x265 [info]: tools: tmvp b-intra strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I:      1, Avg QP:47.84  kb/s: 70.20
x265 [info]: frame P:    312, Avg QP:46.46  kb/s: 143.55
x265 [info]: frame B:   2021, Avg QP:49.10  kb/s: 34.05
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: Weighted B-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 2.6% 4.2% 1.0% 6.7% 7.3% 31.3% 8.3% 2.2% 6.7% 18.8% 1.6% 0.3% 1.0% 4.5% 0.3% 0.0% 3.2%
encoded 2334 frames in 75.98s (30.72 fps), 48.70 kb/s, Avg QP:48.75
process finished with exitcode: 0 and exitStatue: 0
finished after 00:01:16.146

(crf also return nearly the same output as above, so keyint and ref count probably aren't the issue)

Throwing out --no-open-gop, didn't help either,... Also the crf encode looks way better than the 2pass encode,... (yes, they are both really ugly, but the 2pass encode is just junk)

Uploaded the input file, 2pass encode and a 1pass encode to my google drive (x265_2pass.7z)

I'm not really concerned that the result isn't stunning, but seeing that the 1st pass of the 2pass encoding is nearer at the target bit rate I aimed for than the full 2pass encoding seems like a bug, which is why I wanted to report the problem.


As a side note: the Version-dropDown doesn't offer 1.9, last entry there is 1.7

Comments (4)

  1. Divya Manivannan

    We tried to reproduce this issue with the given avi file, but we got ~51 kb/s for pass 1 and ~61 kb/s for pass 2. The command lines and the output are given below:

    Pass 1: ffmpeg.exe -i ng2.avi -pix_fmt yuv420p -r 60.10 -f yuv4mpegpipe - | x265.exe --pmode --pme --output-depth 10 --y4m - --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --merange 100 --max-merge 5 --no-open-gop --keyint -1 --bframes 16 --weightb --b-intra --rc-lookahead 80 --pass 1 --bitrate 60 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --stats pass1try.stats --output try.hevc

    encoded 2334 frames in 9.07s (257.33 fps), 51.75 kb/s, Avg QP:50.53

    Pass 2: ffmpeg.exe -i ng2.avi -pix_fmt yuv420p -r 60.10 -f yuv4mpegpipe - | x265.exe --pmode --pme --output-depth 10 --y4m - --allow-non-conformance --ctu 32 --tu-intra-depth 3 --tu-inter-depth 3 --me 3 --subme 5 --merange 100 --rect --amp --max-merge 5 --no-open-gop --keyint -1 --bframes 16 --weightb --b-intra --ref 16 --rc-lookahead 80 --pass 2 --bitrate 60 --rd 6 --rdoq-level 2 --psy-rdoq 25.00 --aq-strength 1.50 --range full --colormatrix bt470bg --stats pass1try.stats --output try.hevc

    encoded 2334 frames in 147.85s (15.79 fps), 61.07 kb/s, Avg QP:47.61

    We are unable to reproduce this issue.

  2. Selur reporter

    Strange have you also tried: a. feeding x265 with 10bit input and b. crop the content like I did?

  3. Log in to comment