FFMPEG Motion Compensation and Search

[Former user]

I'm trying to modify the motion detection part of FFMPEG. What I want to do is to extend the search space, so that whenever the macroblock hit the right most edge of the frame, I need it to still move the block towards the left-most as if they are connected (in my example videos, the right edge is actually a continue of the left edge). Can someone help me to point where exactly I can modify it within FFMPEG source code or x265, or x264?

I took H265 as an example from here. It has a motion.c file which nicely specifies the possible block sizes as below. But I can't find the specific loop that traverses the frame. A help is highly appreciated.

#define SETUP_SCALE(W, H) \
    sizeScale[LUMA_ ## W ## x ## H] = (H * H) >> 4;
    SETUP_SCALE(4, 4);
    SETUP_SCALE(8, 8);
    SETUP_SCALE(8, 4);
    SETUP_SCALE(4, 8);
    SETUP_SCALE(16, 16);
    SETUP_SCALE(16, 8);
    SETUP_SCALE(8, 16);
    SETUP_SCALE(16, 12);
    SETUP_SCALE(12, 16);
    SETUP_SCALE(4, 16);
    SETUP_SCALE(16, 4);
    SETUP_SCALE(32, 32);
    SETUP_SCALE(32, 16);
    SETUP_SCALE(16, 32);
    SETUP_SCALE(32, 24);
    SETUP_SCALE(24, 32);
    SETUP_SCALE(32, 8);
    SETUP_SCALE(8, 32);
    SETUP_SCALE(64, 64);
    SETUP_SCALE(64, 32);
    SETUP_SCALE(32, 64);
    SETUP_SCALE(64, 48);
    SETUP_SCALE(48, 64);
    SETUP_SCALE(64, 16);
    SETUP_SCALE(16, 64);
#undef SETUP_SCALE

[Former user]

I wanted to email you separately Steve. I'm glad I found you here.

[Deepthi Nandakumar]

Hi,

Not sure how often Steve checks in here - so chipping in. Search::setSearchRange generates mvmin and mvmax, which are then passed through to MotionEstimate and used as bounds for clipping.

To clarify what you need to do, you need the motion vector of the rightmost block to be able to point to the leftmost block? With a large enough merange, this is theoretically possible - if all the rightmost blocks end up pointing to the leftmost, the cost in bits for mvd can be kept low, I suppose.

[Former user]

Hi Deepthi, Yes. The rightmost block should be able to point to the leftmost until the block ends (until block.left <= frame.width). Theoretically it should help when my frames are like cylinder. Can you help me with this modification? The code base is huge; I was just looking through it all over today and I'm just lost.

[Deepthi Nandakumar]

I would start with an merange = width, and then modify MV::clipped to do a mod-operation, instead of min, max. I'm pretty sure you will run into more issues though - you can ping us if you need more clarifications. All the best!

[Former user]

merange is the motion estimation range? What does it mean exactly when merange = width? I think if I add a % somewhere as a mod it should solve the problem. Is it correct that each block starts from block_width to frame.width?

block.rightX ==block.width; // a block initiates from left part of the frame
while(block.leftX <=frame.width) //until it hits the right most edge (assuming it moves from left to right for search)
for (int i=0;i<block.width();i++)
    for(int j=0;j< i<block.height();j++)
        block[i][j] = frame_pixel [(block.leftX+i) % frame.width] [block.leftY+j] //if the block's width passes frame's width, we just take the mod.

Is my logic correct? This is what I need to do.

[Former user]

One way to do that is to modify the edge extension area (which is already in the code, in the frameFilter.cpp), and do that for rightmost and fill blocks with leftmost pixels. I identified the piece of code here. Can someone help me to add this feature for right-to-left extension? I will take that as example to add the top-to-bottom extension.

if ((col == 0) | (col == m_frameFilter->m_numCols - 1))
    {
        // TODO: improve by process on Left or Right only
        primitives.extendRowBorder(reconPic->getLumaAddr(m_rowAddr), stride, reconPic->m_picWidth, realH, reconPic->m_lumaMarginX);

        if (m_frameFilter->m_param->internalCsp != X265_CSP_I400)
        {
            primitives.extendRowBorder(reconPic->getCbAddr(m_rowAddr), strideC, reconPic->m_picWidth >> hChromaShift, realH >> vChromaShift, reconPic->m_chromaMarginX);
            primitives.extendRowBorder(reconPic->getCrAddr(m_rowAddr), strideC, reconPic->m_picWidth >> hChromaShift, realH >> vChromaShift, reconPic->m_chromaMarginX);
        }
    }

    // Extra Left and Right border on first and last CU
    if ((col == 0) | (col == m_frameFilter->m_numCols - 1))
    {
        copySizeY += lumaMarginX;
        copySizeC += chromaMarginX;
    }

    // First column need extension left padding area and first CU
    if (col == 0)
    {
        pixY -= lumaMarginX;
        pixU -= chromaMarginX;
        pixV -= chromaMarginX;
    }