yuv444p lossless roundtrip for 1080p videos

Issue #197 new
Peter Steinbach created an issue

Hi,

I am testing the yuv444p roundtrip quality of the x265 lossless (ultrafast) mode on some 1080p imagery on todays "tip" (changeset: 11031:f4c267f28487). http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_30fps_420_8bit_AVC_MP4.mp4

I am doing the following:

$ ffmpeg -i Bosphorus_1920x1080_30fps_420_8bit_AVC_MP4.mp4 -pix_fmt yuv444p Bosphorus_1920x1080_30fps_444_8bit_AVC_MP4.yuv
$ x265 --lossless --preset=ultrafast --pme --pmode --input-depth 8 --input-res 256x128 --fps 30 --input-csp i444 --input Bosphorus_1920x1080_30fps_444_8bit_AVC_MP4.yuv Bosphorus_1920x1080_30fps_444_8bit_AVC_MP4_lrt.hevc
$  ffmpeg -i Bosphorus_1920x1080_30fps_444_8bit_AVC_MP4_lrt.hevc -pix_fmt yuv444p Bosphorus_1920x1080_30fps_444_8bit_AVC_MP4_lrt.yuv
$ md5sum Bosphorus*yuv
a5218f68414d8956e6e69bdb48301228  Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv
bc55e7201dc65f6cc542dabd4dc50ba9  Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.yuv

The funny thing, with smaller crops (made with ffmpeg) of the same video, lossless compression works perfectly and yields identical md5 hashes for the input and output *.yuv files.

I am unclear if this is a ffmpeg thing or just x265 misbehaving?

Best, P

Comments (14)

  1. Deepthi Nandakumar

    Hi,

    x265 is certainly expected to be fully lossless for all resolutions. Are you saying that for 1080p resolutions this does not work?

    You've displayed the md5sums of Jockey**.yuv

  2. Peter Steinbach reporter

    yes, this is my question. I guess I copy-and-pasted the wrong commands. Apologies for that. here is the stuff I'm doing with the x265 tip from Oct 7 from stable branch and using ffmpeg 2.8 linked to it.

    $ ffmpeg -i Jockey_1920x1080_30fps_420_8bit_AVC_MP4.mp4 -pix_fmt yuv444p Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv
    $ x265 --lossless --preset=ultrafast --input-depth 8 --input-res 1920x1080 --fps 30 --input-csp i444 Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.hevc
    $ ffmpeg -i Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.hevc -pix_fmt yuv444p Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.yuv 
    $ md5sum Jockey_1920x1080_30fps_444*yuv
    bc55e7201dc65f6cc542dabd4dc50ba9  Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.yuv
    a5218f68414d8956e6e69bdb48301228  Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv
    

    Strange, isn't it?

    PS. for completeness:

    $ hg tip
    changeset:   11037:a85011719ea1
    branch:      stable
    tag:         tip
    parent:      11033:98ac099a766f
    user:        Deepthi Nandakumar <deepthi@multicorewareinc.com>
    date:        Wed Oct 07 09:22:44 2015 +0530
    summary:     scenecut: fix an obvious bug in scenecut thresholds
    
  3. Peter Steinbach reporter

    ping? this is a show breaker for our application of x265. we have gray16 progressive data and are looking into ways to compress it with x265. yuv444 was one option. I'll try the above again with the current tip.

    $ hg tip
    changeset:   11089:04575a459a16
    tag:         tip
    user:        Ramya Sriraman <ramya@multicorewareinc.com>
    date:        Wed Sep 30 11:22:16 2015 +0530
    summary:     asm: Add sse_ss for [16x16],[32x32] & [64x64] for 8bpp avx2
    

    and the result is:

    $ md5sum Jockey_1920x1080_30fps_444*yuv
    bc55e7201dc65f6cc542dabd4dc50ba9  Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.yuv
    a5218f68414d8956e6e69bdb48301228  Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv
    

    So whatever it is, it's still there. Any help would be appreciated!

  4. Peter Steinbach reporter

    one more piece of information, I just encoded the same file through ffmpeg/libx265, and the error prevails (so its unlikely due to some bogous i/o while writing the yuv "container"):

    $ ffmpeg -pix_fmt yuv444p -s 1920x1080 -r 30 -i Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv -c:v libx265 -preset ultrafast -x265-params lossless=1 -strict experimental Jockey_1920x1080_30fps_444_8bit_AVC_MP4_ffmpegencoded.hevc
    $ ffmpeg -i Jockey_1920x1080_30fps_444_8bit_AVC_MP4_ffmpegencoded.hevc -pix_fmt yuv444p Jockey_1920x1080_30fps_444_8bit_AVC_MP4_ffmpegencoded.yuv
    $ md5sum Jockey_1920x1080_30fps_4*yuv
    bc55e7201dc65f6cc542dabd4dc50ba9  Jockey_1920x1080_30fps_444_8bit_AVC_MP4_ffmpegencoded.yuv
    bc55e7201dc65f6cc542dabd4dc50ba9  Jockey_1920x1080_30fps_444_8bit_AVC_MP4_lrt.yuv
    a5218f68414d8956e6e69bdb48301228  Jockey_1920x1080_30fps_444_8bit_AVC_MP4.yuv
    
  5. Deepthi Nandakumar

    Sorry for the delay, I tested this out on the x265 commandline with a couple of testcases (crowdrun, duckstakeoff - 8bit 444, 1080p) and it worked fine. Our automated regression test results have also not complained.

    Can you use x265.exe and see if you're seeing the same results?

    One more thing to check, are the csp parameters being passed from ffmpeg to x265? This was my standalone x265 commandline: D:\YUVClips\CrowdRun_1920x1080_50_8bit_444.yuv --input-res 1920x1080 --fps 25 --input-csp i444 -o crowdrun.hevc --psnr --ssim -p ultrafast --lossless --hash 1 -f 100

  6. Peter Steinbach reporter

    mmmh, my roundtrips with all frames (not just 100) still fail. I used the duckstakeoff 1080p50 sample from here. but I found something odd:

    $ x265 ducks_take_off_1080p50_fps50_444.yuv --input-res 1920x1080 --fps 50 --input-csp i444 -o ducks.hevc --psnr --ssim -p ultrafast --lossless
    ...
    $$ ffprobe ducks.hevc
    ffprobe version n2.8 Copyright (c) 2007-2015 the FFmpeg developers
      built with gcc 4.8.3 (GCC) 20140911 (Red Hat 4.8.3-9)
      configuration: --prefix=/home/steinbac/software/ffmpeg/2.8-x265master --enable-libx265 --enable-libx264 --enable-gpl --enable-shared
      libavutil      54. 31.100 / 54. 31.100
      libavcodec     56. 60.100 / 56. 60.100
      libavformat    56. 40.101 / 56. 40.101
      libavdevice    56.  4.100 / 56.  4.100
      libavfilter     5. 40.101 /  5. 40.101
      libswscale      3.  1.101 /  3.  1.101
      libswresample   1.  2.101 /  1.  2.101
      libpostproc    53.  3.100 / 53.  3.100
    [hevc @ 0xa5a600] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, hevc, from 'ducks.hevc':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: hevc (Rext), yuv444p(tv), 1920x1072, 50 tbr, 1200k tbn, 50 tbc
    

    if I then decode this with ffmpeg to say y4m and look at the header I again encounter 1920x1072 not 1920x1080 as it should be! So is the ffmpeg 2.8 decoder wrong then?

    $ ffmepg -i ducks.hevc ducks.y4m
    $ strings ducks.y4m|head -n1
    YUV4MPEG2 W1920 H1072 F50:1 Ip A0:0 C444 XYSCSS=444
    
  7. Peter Steinbach reporter

    submitted a ffmpeg bug report for now: https://trac.ffmpeg.org/ticket/4980

    It apparently is a problem with the ultrafast versus fast preset:

     $ ffmpeg -f lavfi -i testsrc=s=320x248 -vframes 10 -pix_fmt yuv444p
     out_10.yuv
     $ ffmpeg -f lavfi -i testsrc=s=320x248 -vframes 10 -strict experimental
     -pix_fmt yuv444p -vcodec libx265 -preset ultrafast -x265-params lossless=1
     out_10_lossless.hevc
     $ ffmpeg -i out_10_lossless.hevc -pix_fmt yuv444p out_10_lrt.yuv
     $ md5sum out_10*.yuv
     cd5e06d1217f093c4265a94b30e11e53  out_10_lrt.yuv
     bbc307efc496e074b177b51f555f4437  out_10.yuv
     $ ffmpeg -f lavfi -i testsrc=s=320x248 -vframes 10 -strict experimental
     -pix_fmt yuv444p -vcodec libx265 -preset fast -x265-params lossless=1
     out_10_lossless_fast.hevc
     $ ffmpeg -i out_10_lossless_fast.hevc -pix_fmt yuv444p out_10_lrt_fast.yuv
     $ md5sum out_10*.yuv
     bbc307efc496e074b177b51f555f4437  out_10_lrt_fast.yuv
     cd5e06d1217f093c4265a94b30e11e53  out_10_lrt.yuv
     bbc307efc496e074b177b51f555f4437  out_10.yuv
    

    btw, the decoding without the lossless flag in ultrafast also produces the wrong resolution! so it seems not only limited to that ... I used x265 tip and ffmpeg 2.9-dev

  8. Ashok Kumar Mishra

    This issue is with ffmpeg decoder and that is for 444 colorspcae format only. I tested with HM decoder(HM 16.4) which is giving the correct output. ffmpeg decoder with 420 colorspace format is working fine.

  9. Ashok Kumar Mishra

    Make sure you are using the correct input color space format to test and check with HM decoder (latest 16.4) for conformance. As I found the video you pasted "Bosphorus_1920x1080_30fps_420_8bit_AVC_MP4" is 420 video. But you have given the command line option --input-csp i444

  10. Peter Steinbach reporter

    Ok, I took the yuv444p ducks_take_off sample from here: https://media.xiph.org/video/derf/ (using the x265 tip and ffmpeg master pulled/checkedout on Nov 17, 1:40 CET)

    $ x265 --lossless --preset=ultrafast --input-depth 8 --input-res 1200x720 --fps 50 --input-csp i444 ducks_take_off_444_1200x720p50.yuv ducks_take_off_444_1200x720p50_lossless_ultrafast.hevc
    $ ffmpeg -i ducks_take_off_444_1200x720p50_lossless_ultrafast.hevc -pix_fmt yuv444p ducks_take_off_444_1200x720p50_lossless_ultrafast.yuv
    $ ${HOME}/software/jctvc-hm/repo/bin/./TAppDecoderStatic -b ducks_take_off_444_1200x720p50_lossless_ultrafast.hevc -o ducks_take_off_444_1200x720p50_lossless_ultrafast_HM.yuv --OutputColourSpaceConvert=UNCHANGED
    $ md5sum  ducks_take_off_444_1200x720p50.yuv ducks_take_off_444_1200x720p50_lossless_ultrafast.yuv ducks_take_off_444_1200x720p50_lossless_ultrafast_HM.yuv
    c1d2b822af36d5bd84f3f1a516851b45  ducks_take_off_444_1200x720p50.yuv
    022c44ad7e5623144e0e7082db243fd4  ducks_take_off_444_1200x720p50_lossless_ultrafast.yuv
    2ea9da8efee359221599b502413e957b  ducks_take_off_444_1200x720p50_lossless_ultrafast_HM.yuv
    

    I'll ping pong this back to the ffmpeg people then! thanks so far.

  11. Log in to comment