encoder crash with --no-wpp

[Former user]

crash started at this tip,

Changeset:11190 (c6fd2839b9aa) deblock: row deblocking is now performed within the encode loop User:Min Chen [e-mail]

and encoded fine with --no-deblock --no-sao

Tested at latest tip: e951ab673b1c

[Ma0]

Confirmed:

$ gdb x265.exe
GNU gdb (GDB) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from x265.exe...done.
(gdb) r --no-wpp --preset veryslow 720p50_parkrun_ter.y4m -o w.hevc
Starting program: i:\x265\ma\x265.exe --no-wpp --preset veryslow 720p50_parkrun_ter.y4m -o w.hevc
[New Thread 4492.0x1134]
warning: Can not parse XML library list; XML support was disabled at compile time
y4m  [info]: 1280x720 fps 50/1 i420p8 sar 1:1 frames 0 - 503 of 504
[New Thread 4492.0xdbc]
raw  [info]: output file: w.hevc
x265 [info]: HEVC encoder version 1.8+167-e951ab673b1c
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: frame threads / pool features       : 4 / none
[New Thread 4492.0x1318]
[New Thread 4492.0xdfc]
[New Thread 4492.0x8e4]
[New Thread 4492.0xc98]
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 57 / 4 / 4
x265 [info]: Keyframe min / max / scenecut       : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 5 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 signhide
x265 [info]: tools: tmvp b-intra strong-intra-smoothing deblock sao

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 4492.0x8e4]
0x00000000004f0c32 in x265::ThreadPool::tryAcquireSleepingThread (this=0x0, firstTryBitmap=0, secondTryBitmap=0)
    at i:/x265/source/common/threadpool.cpp:181
181         sleepbitmap_t masked = m_sleepBitmap & firstTryBitmap;
(gdb) bt
#0  0x00000000004f0c32 in x265::ThreadPool::tryAcquireSleepingThread (this=0x0, firstTryBitmap=0, secondTryBitmap=0)
    at i:/x265/source/common/threadpool.cpp:181
#1  0x00000000004f0d74 in x265::ThreadPool::tryBondPeers (this=0x0, maxPeers=1, peerBitmap=0, master=...)
    at i:/x265/source/common/threadpool.cpp:213
#2  0x00000000007ac846 in x265::BondedTaskGroup::tryBondPeers (this=0x3414fa8, jp=..., maxPeers=1)
    at I:/x265/source/common/threadpool.h:143
#3  0x000000000050eb14 in x265::FrameEncoder::processRowEncoder (this=0xfda250, intRow=1, tld=...)
    at i:/x265/source/encoder/frameencoder.cpp:1119
#4  0x000000000050b258 in x265::FrameEncoder::compressFrame (this=0xfda250) at i:/x265/source/encoder/frameencoder.cpp:577
#5  0x0000000000509a1d in x265::FrameEncoder::threadMain (this=0xfda250) at i:/x265/source/encoder/frameencoder.cpp:284
#6  0x00000000004609c0 in x265::stackAlignMain (instance=0xfda290) at i:/x265/source/common/threading.cpp:91
#7  0x00000000004609dc in x265::ThreadShim (instance=0xfda290) at i:/x265/source/common/threading.cpp:98
#8  0x0000000077725a4d in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x4f0c12 to 0x4f0c52:
   0x00000000004f0c12 <x265::JobProvider::tryWakeOne()+210>:    add    %dl,0x40c48348(%rax)
   0x00000000004f0c18 <x265::JobProvider::tryWakeOne()+216>:    pop    %rbp
   0x00000000004f0c19 <x265::JobProvider::tryWakeOne()+217>:    retq
   0x00000000004f0c1a <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+0>:   push   %rbp
   0x00000000004f0c1b <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+1>:   mov    %rsp,%rbp

   0x00000000004f0c1e <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+4>:   sub    $0x20,%rs
p
   0x00000000004f0c22 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+8>:   mov    %rcx,0x10
(%rbp)
   0x00000000004f0c26 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+12>:  mov    %rdx,0x18
(%rbp)
   0x00000000004f0c2a <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+16>:  mov    %r8,0x20(
%rbp)
   0x00000000004f0c2e <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+20>:  mov    0x10(%rbp
),%rax
=> 0x00000000004f0c32 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+24>:  mov    (%rax),%r
ax
   0x00000000004f0c35 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+27>:  and    0x18(%rbp
),%rax
   0x00000000004f0c39 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+31>:  mov    %rax,-0x8
(%rbp)
   0x00000000004f0c3d <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+35>:  cmpq   $0x0,-0x8
(%rbp)
   0x00000000004f0c42 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+40>:  je     0x4f0cb3
<x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+153>
   0x00000000004f0c44 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+42>:  xor    %eax,%eax

   0x00000000004f0c46 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+44>:  tzcnt  -0x8(%rbp
),%rax
   0x00000000004f0c4c <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+50>:  mov    %eax,-0xc
(%rbp)
   0x00000000004f0c4f <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+53>:  mov    -0xc(%rbp
),%eax
End of assembler dump.
(gdb) info all-registers
rax            0x0      0
rbx            0xffffffffffffffff       -1
rcx            0x0      0
rdx            0x0      0
rsi            0x383ff70        58982256
rdi            0x383ff98        58982296
rbp            0x463f780        0x463f780
rsp            0x463f760        0x463f760
r8             0x0      0
r9             0x3414fa8        54611880
r10            0xef60   61280
r11            0x0      0
r12            0x0      0
r13            0x0      0
r14            0x0      0
r15            0x0      0
rip            0x4f0c32 0x4f0c32 <x265::ThreadPool::tryAcquireSleepingThread(unsigned long long, unsigned long long)+24>
eflags         0x10206  [ PF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x2b     43
es             0x2b     43
fs             0x53     83
gs             0x2b     43
st0            -nan(0x12002200000022)   (raw 0xffff0012002200000022)
st1            -nan(0x12001200000010)   (raw 0xffff0012001200000010)
st2            -nan(0x500000008)        (raw 0xffff0000000500000008)
st3            -nan(0x600000009)        (raw 0xffff0000000600000009)
st4            -nan(0x100000005)        (raw 0xffff0000000100000005)
st5            -nan(0x07f7e7e80)        (raw 0xffff000000007f7e7e80)
st6            -nan(0x07f807f7e)        (raw 0xffff000000007f807f7e)
st7            -nan(0xff01ff01ff01ff01) (raw 0xffffff01ff01ff01ff01)
fctrl          0x20027f 2097791
fstat          0x20     32
ftag           0x0      0
fiseg          0x33     51
fioff          0x79c83f 7981119
foseg          0x2b     43
fooff          0x463fba8        73661352
fop            0x0      0
xmm0           {v4_float = {0x0, 0x5, 0x0, 0x0}, v2_double = {0x1e00, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbe,
    0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x40be, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0,
    0x40be0000, 0x0, 0x0}, v2_int64 = {0x40be000000000000, 0x0}, uint128 = 0x000000000000000040be000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
---Type <return> to continue, or q <return> to quit---
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {
    0xff <repeats 16 times>}, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {
    0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff},
  uint128 = 0xffffffffffffffffffffffffffffffff}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x80 <repeats 16 times>}, v8_int16 = {
    0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080}, v4_int32 = {0x80808080, 0x80808080, 0x80808080,
    0x80808080}, v2_int64 = {0x8080808080808080, 0x8080808080808080}, uint128 = 0x80808080808080808080808080808080}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x80 <repeats 16 times>}, v8_int16 = {
    0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080}, v4_int32 = {0x80808080, 0x80808080, 0x80808080,
    0x80808080}, v2_int64 = {0x8080808080808080, 0x8080808080808080}, uint128 = 0x80808080808080808080808080808080}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x80 <repeats 16 times>}, v8_int16 = {
    0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080, 0x8080}, v4_int32 = {0x80808080, 0x80808080, 0x80808080,
    0x80808080}, v2_int64 = {0x8080808080808080, 0x8080808080808080}, uint128 = 0x80808080808080808080808080808080}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm8           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm9           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm12          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
mxcsr          0x1fa0   [ PE IM DM ZM OM UM PM ]

[Mahesh Pittala]

These encoder crashes exists on latest tip - 1471e4e433cc

1) no-wpp crashing only at veryslow preset.

CLI: x265.exe BasketballDrive_1920x1080_50.y4m -o b.hevc --no-wpp -p veryslow -f 10

full encoder log - http://pastie.org/10660130

2) combination of no-wpp & lookahead-slices 0 crashing at all presets.

CLI: x265.exe BasketballDrive_1920x1080_50.y4m -o b.hevc --no-wpp --lookahead-slices 0 -p ultrafast --crf 40 -f 10

full encoder log - http://pastie.org/10660139

[Ma0]

Maybe this helps:

diff -r 375ce77b8c35 source/encoder/encoder.cpp
--- a/source/encoder/encoder.cpp    Mon Dec 28 16:06:55 2015 -0600
+++ b/source/encoder/encoder.cpp    Thu Dec 31 21:30:30 2015 +0100
@@ -104,8 +104,8 @@
     bool allowPools = !p->numaPools || strcmp(p->numaPools, "none");

     // Trim the thread pool if --wpp, --pme, and --pmode are disabled
-    if (!p->bEnableWavefront && !p->bDistributeModeAnalysis && !p->bDistributeMotionEstimation && !p->lookaheadSlices)
-        allowPools = false;
+//    if (!p->bEnableWavefront && !p->bDistributeModeAnalysis && !p->bDistributeMotionEstimation && !p->lookaheadSlices)
+//        allowPools = false;

     if (!p->frameNumThreads)
     { 

[M CHEN]

Could you try my patch in the mailing-list?

[Ma0]

In my tests this patch is OK. Thanks!