What is this ?

This is a library addressing on Video processing using Microsoft Media Foundation (MF). It provide easy-to-understand video processing functions based on MF with new APIs (OpenCV friendly).

Motivation ?

Previously, I prefer to use OpenCV to handle video processing. Unfortunately, OpenCV cannot fulfil accurate video processing, e.g. OpenCV cannot accurately obtain a frame from a video based on the frame time (it does return a frame but not the correct one). This is the main reason to use Microsoft Media Foundation (MF). Also, it is quite convenient to use it; all you need to do is to install Windows SDK, where it is integrated.

There ain't no such thing as a free lunch. The MF's interfaces are a little bit obscure. In order to use it, lots of time maybe needed to fully understand these. Fortunately, one of my project colleges recently created a easy-to-understand media IO library based on MF. I modified some interfaces to make it more friendly to use and created this library (mediaio) so you can link it in your code.

How to use it ?

Just follow the mediaio-test project and its sample code.

Highlights ?

  1. Load a video:

    mediaioAPI mediaio_API("-your-video-file-.avi");
  2. Retrieve video properties:

    int width = mediaio_API.get_width();                            // get frame width
    int height = mediaio_API.get_height();                          // get frame height
    double fps = mediaio_API.get_FPS();                             // get fps
    unsigned long long frame_span = mediaio_API.get_frame_span();   // get frame span (in 10^-7s)
    unsigned long long num_frames = mediaio_API.get_num_frames();   // get total # of frames
    unsigned long long duration = mediaio_API.get_duration();       // get total duration (in 10^-7s)
  3. Read given frame (CV_BGR) given time or index:

    Mat frame_time  = mediaio_API.get_frame_at_time(time);          // given time (in 10^-7s)
    Mat frame_index = mediaio_API.get_frame_at_index(index);        // given frame index

Remarks ?

  • This library is originally designed for the situation that one camera has multiple clips. When only processing a single video, you just need to add the video to the single clip, like in above sample code.

  • Video Type: The types of supported videos depend on the types that are registered on your PC. Installing DivX Plus can enable you PC natively support .wmv/.asf/.avi/.divx/.mkv/.mp4/.mov and extended support (requires external component installed) .mpg/.mpe/.mpeg/.m1v/.vob/.ts/.m2v/.rmvb/.m2ts/.mts videos. NOTE: currently only .avi is fully tested; .mkv seems works badly even after installing DivX Plus.

  • Time Unit: All the time used in the library are in 100 ns (10^-7s), e.g. return from mediaioAPI::get_duration().