ffmpeg 7.1 and x265 4.1 built with alpine/musl segfaults

Issue #969 resolved
Mattias Wadman created an issue

Hello, i’m trying to build x265 4.1 with ffmpeg 7.1 with alpine linux (uses musl libc). x265 4.0 build from same build env seems to run fine. I haven’t tried same versions with something glibc based yet.

Reproduction dockerfile:

ARG ALPINE_VERSION=alpine:3.20.3
FROM $ALPINE_VERSION AS builder

RUN apk add --no-cache \
  coreutils \
  wget \
  ca-certificates \
  bash \
  tar \
  build-base \
  autoconf automake \
  libtool \
  diffutils \
  cmake meson ninja \
  git \
  yasm nasm \
  texinfo \
  jq \
  xxd \
  python3 py3-packaging \
  curl \
  pkgconfig

# retry dns and some http codes that might be transient errors
ARG WGET_OPTS="--retry-on-host-error --retry-on-http-error=429,500,502,503"

# --no-same-owner as we don't care about uid/gid even if we run as root. fixes invalid gid/uid issue.
ARG TAR_OPTS="--no-same-owner --extract --file"

ARG X265_VERSION=4.1
ARG X265_URL="https://bitbucket.org/multicoreware/x265_git/downloads/x265_$X265_VERSION.tar.gz"
RUN \
  wget $WGET_OPTS -O x265_git.tar.bz2 "$X265_URL" && \
  tar $TAR_OPTS x265_git.tar.bz2 && cd x265_*/build/linux && \
  cmake -G "Unix Makefiles" -D CMAKE_BUILD_TYPE=Debug ../../source && \
  make -j$(nproc) && \
  make install

ARG FFMPEG_VERSION=7.1
ARG FFMPEG_URL="https://ffmpeg.org/releases/ffmpeg-$FFMPEG_VERSION.tar.bz2"
RUN \
  wget $WGET_OPTS -O ffmpeg.tar.bz2 "$FFMPEG_URL" && \
  tar $TAR_OPTS ffmpeg.tar.bz2 && cd ffmpeg* && \
  CFLAGS="-O0 -ggdb" ./configure \
  --enable-gpl \
  --enable-version3 \
  --disable-ffplay \
  --enable-libx265 \
  || (cat ffbuild/config.log ; false) \
  && make -j$(nproc)

#RUN cd ffmpeg* && ./ffmpeg_g -f lavfi -i testsrc -c:v libx265 -t 100ms -f null -

Reproduction:

$ docker build -t dev --progress=plain .
<log cut>

$ docker run -ti --privileged dev
/ # apk add gdb
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/aarch64/APKINDEX.tar.gz
(1/3) Installing musl-dbg (1.2.5-r0)
(2/3) Installing gdb (14.2-r1)
(3/3) Installing gcc-gdb (13.2.1_git20240309-r0)
Executing busybox-1.36.1-r29.trigger
OK: 423 MiB in 105 packages

/ # cd ffmpeg-7.1/

/ffmpeg-7.1 # ./ffmpeg_g -f lavfi -i testsrc -c:v libx265 -t 100ms -f null -
ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
  configuration: --enable-gpl --enable-version3 --disable-ffplay --enable-libx265
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, lavfi, from 'testsrc':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (libx265))
Press [q] to stop, [?] for help
x265 [info]: HEVC encoder version 4.1+1-1d117be
x265 [info]: build info [Linux][GCC 13.2.1][64 bit] 8bit
x265 [info]: using cpu capabilities: NEON Neon_DotProd
x265 [warning]: halving the quality when psy-rd is enabled for 444 input. Setting cbQpOffset = 6 and crQpOffset = 6
x265 [info]: Main 4:4:4 profile, Level-2 (Main tier)
x265 [info]: Thread pool created using 11 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 3 / wpp(4 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00
x265 [info]: Cb/Cr QP Offset                     : 6 / 6
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing deblock sao
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: hevc, gbrp(pc, gbr/unknown/unknown, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.19.100 libx265
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Segmentation fault


/ffmpeg-7.1 # gdb --args ./ffmpeg_g -f lavfi -i testsrc -c:v libx265 -t 100ms -f null -
GNU gdb (GDB) 14.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./ffmpeg_g...
(gdb)
Starting program: /ffmpeg-7.1/ffmpeg_g -f lavfi -i testsrc -c:v libx265 -t 100ms -f null -
ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
  configuration: --enable-gpl --enable-version3 --disable-ffplay --enable-libx265
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[New LWP 78]
[New LWP 79]
[New LWP 80]
[New LWP 81]
[New LWP 82]
[New LWP 83]
[New LWP 84]
[New LWP 85]
[New LWP 86]
[New LWP 87]
[New LWP 88]
Input #0, lavfi, from 'testsrc':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (libx265))
[New LWP 89]
[New LWP 90]
[New LWP 91]
[New LWP 92]
Press [q] to stop, [?] for help
[New LWP 93]
[New LWP 94]
[New LWP 95]
[New LWP 96]
[New LWP 97]
[New LWP 98]
[New LWP 99]
[New LWP 100]
[New LWP 101]
[New LWP 102]
[New LWP 103]
[New LWP 104]
[New LWP 105]
[New LWP 106]
[New LWP 107]
[New LWP 108]
[New LWP 109]
[New LWP 110]
[New LWP 111]
[New LWP 112]
[New LWP 113]
[New LWP 114]
x265 [info]: HEVC encoder version 4.1+1-1d117be
x265 [info]: build info [Linux][GCC 13.2.1][64 bit] 8bit
x265 [info]: using cpu capabilities: NEON Neon_DotProd
x265 [warning]: halving the quality when psy-rd is enabled for 444 input. Setting cbQpOffset = 6 and crQpOffset = 6
x265 [info]: Main 4:4:4 profile, Level-2 (Main tier)
x265 [info]: Thread pool created using 11 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 3 / wpp(4 rows)
[New LWP 115]
[New LWP 116]
[New LWP 117]
[New LWP 118]
[New LWP 119]
[New LWP 120]
[New LWP 121]
[New LWP 122]
[New LWP 123]
[New LWP 124]
[New LWP 125]
x265 [warning]: Source height < 720p; disabling lookahead-slices
[New LWP 126]
[New LWP 127]
[New LWP 128]
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00
x265 [info]: Cb/Cr QP Offset                     : 6 / 6
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing deblock sao
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: hevc, gbrp(pc, gbr/unknown/unknown, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.19.100 libx265
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[New LWP 129]
[LWP 114 exited]
[LWP 113 exited]
[LWP 112 exited]
[LWP 111 exited]
[LWP 109 exited]
[LWP 108 exited]
[LWP 107 exited]
[LWP 106 exited]
[LWP 105 exited]
[LWP 104 exited]
[LWP 110 exited]
[LWP 102 exited]
[LWP 101 exited]
[LWP 100 exited]
[LWP 99 exited]
[LWP 97 exited]
[LWP 95 exited]
[LWP 94 exited]
[LWP 93 exited]
[LWP 96 exited]
[LWP 103 exited]
[LWP 98 exited]
[LWP 90 exited]
[LWP 92 exited]
[LWP 91 exited]

Thread 13 "enc0:0:libx265" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 89]
0x0000fffff7f66214 in a_crash () at ./src/internal/atomic.h:250
warning: 250    ./src/internal/atomic.h: No such file or directory

(gdb)
#0  0x0000fffff7f66214 in a_crash () at ./src/internal/atomic.h:250
#1  get_meta (p=p@entry=0x4040563333333333 <error: Cannot access memory at address 0x4040563333333333>) at src/malloc/mallocng/meta.h:154
#2  0x0000fffff7f6669c in __libc_free (p=0x4040563333333333) at src/malloc/mallocng/free.c:105
#3  0x0000fffff7f65cc4 in free (p=<optimized out>) at src/malloc/free.c:5
#4  0x0000aaaaab8e6b74 in av_free (ptr=<optimized out>) at libavutil/mem.c:243
#5  0x0000aaaaab8e6d30 in av_freep (arg=arg@entry=0xfffff76e0fe8) at libavutil/mem.c:253
#6  0x0000aaaaab27f044 in libx265_encode_frame (avctx=0xfffff7fe3c10, pkt=0xfffff7227410, pic=<optimized out>, got_packet=0xfffff76e56d4)
    at libavcodec/libx265.c:821
#7  0x0000aaaaab17dfbc in ff_encode_encode_cb (avctx=avctx@entry=0xfffff7fe3c10, avpkt=avpkt@entry=0xfffff7227410, frame=0x0,
    got_packet=got_packet@entry=0xfffff76e56d4) at libavcodec/encode.c:254
#8  0x0000aaaaab17e2dc in encode_simple_internal (avpkt=0xfffff7227410, avctx=0xfffff7fe3c10) at libavcodec/encode.c:340
#9  encode_simple_receive_packet (avpkt=<optimized out>, avctx=<optimized out>) at libavcodec/encode.c:354
#10 encode_receive_packet_internal (avctx=avctx@entry=0xfffff7fe3c10, avpkt=0xfffff7227410) at libavcodec/encode.c:388
#11 0x0000aaaaab17e5ec in avcodec_send_frame (avctx=avctx@entry=0xfffff7fe3c10, frame=frame@entry=0x0) at libavcodec/encode.c:531
#12 0x0000aaaaaac473d4 in encode_frame (of=0xfffff7fe3920, ost=ost@entry=0xfffff78d18b0, frame=frame@entry=0x0, pkt=0xfffff76e6170)
    at fftools/ffmpeg_enc.c:635
#13 0x0000aaaaaac477e0 in frame_encode (pkt=<optimized out>, frame=0x0, ost=0xfffff78d18b0) at fftools/ffmpeg_enc.c:803
#14 encoder_thread (arg=0xfffff78d18b0) at fftools/ffmpeg_enc.c:906
#15 0x0000aaaaaac5bce8 in task_wrapper (arg=0xfffff7ffc518) at fftools/ffmpeg_sched.c:2514
#16 0x0000fffff7f9f3e0 in start (p=0xfffff76e5ad8) at src/thread/pthread_create.c:207
#17 0x0000fffff7f9d790 in __clone () at src/thread/aarch64/clone.s:28
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

Comments (5)

  1. Uthayakumar Muthu

    Hi Mattias Wadman,

    Thanks for reporting this issue. We were able to reproduce it with FFmpeg 7.1. The issue happens because of API changes in x265 that were reverted after version 4.0 (commit: 78e5b703b186fe184bf91bb37df82f64059b3f61).

    • FFmpeg 7.1 works fine with x265-4.0, but fails with x265-4.1 because of these changes.
    • We tested the FFmpeg master branch (commit: ac3f69a4b91700075cc3b3379b428e62fbb68606) with x265-4.1, and it works without any issues.

    Both musl-based and glibc-based builds are working fine with the FFmpeg master branch.

    FFmpeg$ ./ffmpeg_g -f lavfi -i testsrc -c:v libx265 -t 100ms -f null -
    ffmpeg version N-117917-gac3f69a4b9 Copyright (c) 2000-2024 the FFmpeg developers
      built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
      configuration: --enable-gpl --enable-version3 --disable-ffplay --enable-libx265
      libavutil      59. 47.101 / 59. 47.101
      libavcodec     61. 26.100 / 61. 26.100
      libavformat    61.  9.100 / 61.  9.100
      libavdevice    61.  4.100 / 61.  4.100
      libavfilter    10.  6.101 / 10.  6.101
      libswscale      8. 12.100 /  8. 12.100
      libswresample   5.  4.100 /  5.  4.100
      libpostproc    58.  4.100 / 58.  4.100
    Input #0, lavfi, from 'testsrc':
      Duration: N/A, start: 0.000000, bitrate: N/A
      Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
    Stream mapping:
      Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (libx265))
    Press [q] to stop, [?] for help
    x265 [info]: HEVC encoder version 4.1+1-1d117be
    x265 [info]: build info [Linux][GCC 13.2.1][64 bit] 8bit
    x265 [info]: using cpu capabilities: NEON SVE Neon_DotProd Neon_I8MM
    x265 [warning]: halving the quality when psy-rd is enabled for 444 input. Setting cbQpOffset = 6 and crQpOffset = 6
    x265 [info]: Main 4:4:4 profile, Level-2 (Main tier)
    x265 [info]: Thread pool created using 8 threads
    x265 [info]: Slices                              : 1
    x265 [info]: frame threads / pool features       : 3 / wpp(4 rows)
    x265 [warning]: Source height < 720p; disabling lookahead-slices
    x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
    x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
    x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
    x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00
    x265 [info]: Cb/Cr QP Offset                     : 6 / 6
    x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
    x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
    x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
    x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
    x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
    x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
    x265 [info]: tools: b-intra strong-intra-smoothing deblock sao
    Output #0, null, to 'pipe:':
      Metadata:
        encoder         : Lavf61.9.100
      Stream #0:0: Video: hevc, gbrp(pc, gbr/unknown/unknown, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbn
        Metadata:
          encoder         : Lavc61.26.100 libx265
        Side data:
          cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    [out#0/null @ 0xffff2f701240] video:5KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
    frame=    3 fps=0.0 q=36.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.695x
    x265 [info]: frame I:      1, Avg QP:32.67  kb/s: 480.20
    x265 [info]: frame P:      1, Avg QP:34.24  kb/s: 26.80
    x265 [info]: frame B:      1, Avg QP:36.00  kb/s: 5.20
    x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
    
    encoded 3 frames in 0.04s (66.69 fps), 170.73 kb/s, Avg QP:34.30
    

    If you face any further issues or need help adapting to the changes, feel free to reach out—we’d be happy to assist.

  2. Mattias Wadman reporter

    Ok thanks. So x265 4.1 (and later) will only be compatible with the next ffmpeg stable release based on master? would be great if such breaking API change could be communicated using semver etc to make configure/build fail instead of segfauling at runtime.

  3. Log in to comment