Y4M and very large files

[Former user]

I ran into a bug recently when using x265 and very large files (at least under x86_64).

My input Y4m file has 2960 frames at 1080 P, but when running x265 it would only convert 198 frames.

the problem seems to be in Y4MInput::guessFrameCount()

at this line

return (int)((size - cur) / (frameSize + strlen(header) + 1));

it seems to convert size._Fpos (a 64 bit int) to a 32 bit int, losing the upper 32 bit values.

I've patched it to do this instead

return (int)((size.seekpos() - cur.seekpos()) / (frameSize + strlen(header) + 1));

which seems to work.

[Steve Borho]

what compiler version are you using? I believe this was a misfeature of some older versions of MSVC.

seekpos() is documented as an internal function that should not be used, in fact on Mac this function is not exposed, so I'm hesitant to use it here.

btw: a workaround for this is to use stdin: x265 - --y4m o.bin < foo.y4m

[Former user]

Hey Steve,

I'm using Visual Studio 2010 x64, the cl.exe returns a version of 16.00.40219.01

Thanks for the workaround, I'll give that a try.

[Steve Borho]

Does VC10 work any better if you cast size and cur to uint64_t?

[Former user]

casting to int64 doesn't make any difference, I changed the code to this

__int64 u64size = (__int64)size;

return (int)((u64size - cur) / (frameSize + strlen(header) + 1));

In the debugger I can see this

    size._Fpos   0x0000000224c4a59c __int64
    u64size      0x0000000024c4a59c __int64

the cast generates these assembly instructions (in debug)

movsxd      rax,dword ptr [rsp+60h]  
mov         rcx,qword ptr [size]  
add         rcx,rax  
mov         rax,rcx  
mov         qword ptr [u64size],rax

[Steve Borho]

input: workaround MSVC runtime bugs in older compilers (fixes #27)

Prior to VC11, ifstream::pos_type could not handle 64bit sizes correctly

→ <<cset a4c676d671d6>>