Embedded Metadata Extractor for Audio/Video

Issue #665 resolved
Former user created an issue

The ability to scrape local video files (MKV, MP4) for embedded metadata and to select embedded as a metadata source over TMDB. This will allow user to use custom metadata instead of relying on TMDB.

ffmpeg -i of MP4 with metadata written by MetaX

C:\Program Files\Serviio\lib>ffmpeg -i "D:\ServerFolders\Movies\21 Jump Street (
2012).mp4"
ffmpeg version 1.1.1 Copyright (c) 2000-2013 the FFmpeg developers
  built on Feb  5 2013 21:59:54 with gcc 4.7.2 (GCC)
  configuration: --enable-static --disable-shared --disable-ffplay --disable-ffs
erver --enable-memalign-hack --enable-libmp3lame --enable-libass --enable-librtm
p --enable-fontconfig --enable-libfreetype --enable-zlib --extra-libs='-lrtmp -l
polarssl -lws2_32 -lwinmm -lexpat -lfreetype -lfribidi -lz' --arch=x86 --enable-
runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross-prefix=i686-w64-
mingw32- --pkg-config=pkg-config
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 86.100 / 54. 86.100
  libavformat    54. 59.106 / 54. 59.106
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.100 /  3. 32.100
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
[mov,mp4,m4a,3gp,3g2,mj2 @ 030B5AE0] stream 0, timescale not set
[mov,mp4,m4a,3gp,3g2,mj2 @ 030B5AE0] max_analyze_duration 5000000 reached at 500
5025
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\ServerFolders\Movies\21 Jump Street
(2012).mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isomavc1
    creation_time   : 2012-06-26 20:05:16
    title           : 21 Jump Street
    artist          : Channing Tatum, Jonah Hill, Brie Larson, Ellie Kemper, Ice
 Cube, Dave Franco, Nick Offerman, Lindsey Broad, Rob Riggle, Johnny Depp
    genre           : Action, Comedy
    date            : 2012-03-16
    track           : 0
    season_number   : 0
    episode_sort    : 0
    description     : They thought the streets were mean. Then they went back to
 high school.
    synopsis        : An undercover police unit consisting of young looking offi
cers infiltrate high schools to control youth crime.
    encoder         : HandBrake 0.9.6 2012022800
    hd_video        : 0
    media_type      : 9
  Duration: 01:49:38.30, start: 0.000000, bitrate: 1139 kb/s
    Chapter #0.0: start 0.000000, end 537.787000
    Metadata:
      title           : Chapter 1
    Chapter #0.1: start 537.787000, end 1102.184000
    Metadata:
      title           : Chapter 2
    Chapter #0.2: start 1102.184000, end 1574.989000
    Metadata:
      title           : Chapter 3
    Chapter #0.3: start 1574.989000, end 2026.273000
    Metadata:
      title           : Chapter 4
    Chapter #0.4: start 2026.273000, end 2379.125000
    Metadata:
      title           : Chapter 5
    Chapter #0.5: start 2379.125000, end 2753.699000
    Metadata:
      title           : Chapter 6
    Chapter #0.6: start 2753.699000, end 3005.750000
    Metadata:
      title           : Chapter 7
    Chapter #0.7: start 3005.750000, end 3379.957000
    Metadata:
      title           : Chapter 8
    Chapter #0.8: start 3379.957000, end 3883.293000
    Metadata:
      title           : Chapter 9
    Chapter #0.9: start 3883.293000, end 4189.599000
    Metadata:
      title           : Chapter 10
    Chapter #0.10: start 4189.599000, end 4523.098000
    Metadata:
      title           : Chapter 11
    Chapter #0.11: start 4523.098000, end 4794.202000
    Metadata:
      title           : Chapter 12
    Chapter #0.12: start 4794.202000, end 5126.033000
    Metadata:
      title           : Chapter 13
    Chapter #0.13: start 5126.033000, end 5685.091000
    Metadata:
      title           : Chapter 14
    Chapter #0.14: start 5685.091000, end 6051.123000
    Metadata:
      title           : Chapter 15
    Chapter #0.15: start 6051.123000, end 6578.266000
    Metadata:
      title           : Chapter 16
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x352 [
SAR 1408:1215 DAR 64:27], 685 kb/s, 23.98 fps, 59.94 tbr, 90k tbn, 180k tbc
    Metadata:
      creation_time   : 2012-06-26 20:05:16
    Stream #0:1(eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp,
 448 kb/s
    Metadata:
      creation_time   : 2012-06-26 20:05:16
    Stream #0:2: Video: mjpeg, yuvj444p, 1000x1500 [SAR 72:72 DAR 2:3], 90k tbr,
 90k tbn, 90k tbc
    Stream #0:3(und): Subtitle: mov_text (text / 0x74786574)
    Metadata:
      creation_time   : 2012-06-26 22:28:36
At least one output file must be specified

C:\Program Files\Serviio\lib>


MediaInfo Text

General
Complete name                            : \\ROHTERTSERVER\Movies\21 Jump Street (2012).mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42
File size                                : 893 MiB
Duration                                 : 1h 49mn
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 139 Kbps
Movie name                               : 21 Jump Street
Performer                                : Channing Tatum, Jonah Hill, Brie Larson, Ellie Kemper, Ice Cube, Dave Franco, Nick Offerman, Lindsey Broad, Rob Riggle, Johnny Depp
Genre                                    : Action, Comedy
Recorded date                            : 2012-03-16
Encoded date                             : UTC 2012-06-26 20:05:16
Tagged date                              : UTC 2012-06-26 22:28:36
Writing application                      : HandBrake 0.9.6 2012022800
Cover                                    : Yes
iTunEXTC                                 : mpaa|R|400|
iTunMOVI                                 : <?xml version="1.0" encoding="UTF-8"?> / <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> / <plist version="1.0"> / <dict> /   <key>cast</key> /   <array> /       <dict> /            <key>name</key> /           <string>Channing Tatum</string> /       </dict> /       <dict> /            <key>name</key> /           <string>Jonah Hill</string> /       </dict> /       <dict> /            <key>name</key> /           <string>Brie Larson</string> /      </dict> /       <dict> /            <key>name</key> /           <string>Ellie Kemper</string> /         </dict> /       <dict> /            <key>name</key> /           <string>Ice Cube</string> /         </dict> /       <dict> /            <key>name</key> /           <string>Dave Franco</string> /      </dict> /       <dict> /            <key>name</key> /           <string>Nick Offerman</string> /        </dict> /       <dict> /            <key>name</key> /           <string>Lindsey Broad</string> /        </dict> /       <dict> /            <key>name</key> /           <string>Rob Riggle</string> /       </dict> /       <dict> /            <key>name</key> /           <string>Johnny Depp</string> /      </dict> /   </array> /  <key>directors</key> /  <array> /       <dict> /            <key>name</key> /           <string>Phil Lord</string> /        </dict> /       <dict> /            <key>name</key> /           <string>Chris Miller</string> /         </dict> /   </array> /  <key>screenwriters</key> /  <array> /       <dict> /            <key>name</key> /           <string>Michael Bacall</string> /       </dict> /   </array> / </dict> / </plist>
tvsn                                     : 0
tves                                     : 0
desc                                     : They thought the streets were mean. Then they went back to high school.
ldes                                     : An undercover police unit consisting of young looking officers infiltrate high schools to control youth crime.
hdvd                                     : 0
stik                                     : 9
cnID                                     : 0

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.0
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1h 49mn
Bit rate mode                            : Variable
Bit rate                                 : 685 Kbps
Width                                    : 720 pixels
Height                                   : 352 pixels
Display aspect ratio                     : 2.40:1
Original display aspect ratio            : 2.40:1
Frame rate mode                          : Variable
Frame rate                               : 23.976 fps
Minimum frame rate                       : 15.986 fps
Maximum frame rate                       : 29.970 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.113
Stream size                              : 537 MiB (60%)
Writing library                          : x264 core 120
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=3 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 2012-06-26 20:05:16
Tagged date                              : UTC 2012-06-26 20:24:51
Color primaries                          : BT.601 NTSC
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Audio
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Mode extension                           : CM (complete main)
Format settings, Endianness              : Big
Codec ID                                 : ac-3
Duration                                 : 1h 49mn
Bit rate mode                            : Constant
Bit rate                                 : 448 Kbps
Channel(s)                               : 6 channels
Channel positions                        : Front: L C R, Side: L R, LFE
Sampling rate                            : 48.0 KHz
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : 83ms
Stream size                              : 351 MiB (39%)
Language                                 : English
Encoded date                             : UTC 2012-06-26 20:05:16
Tagged date                              : UTC 2012-06-26 20:24:49

Text
ID                                       : 4
Format                                   : Apple text
Codec ID                                 : text
Duration                                 : 1h 49mn
Bit rate mode                            : Variable
Bit rate                                 : 0 bps
Delay relative to video                  : 83ms
Stream size                              : 375 Bytes (0%)
Encoded date                             : UTC 2012-06-26 22:28:36
Tagged date                              : UTC 2012-06-26 22:28:36

Comments (17)

  1. Petr Nejedly repo owner

    Another example for audio:

    #!
    
    
    d:\sample_media\video>ffmpeg -i d:\07-MemorySV8.mpc
    ffmpeg version 1.1.git Copyright (c) 2000-2013 the FFmpeg developers
      built on May 30 2013 23:10:31 with gcc 4.7.2 (GCC)
      configuration: --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memal
    ign-hack --enable-libmp3lame --enable-libass --enable-librtmp --enable-fontconfig --enable-libfreety
    pe --enable-zlib --enable-libx264 --extra-libs='-lrtmp -lpolarssl -lws2_32 -lwinmm -lexpat -lfreetyp
    e -lfribidi -lz' --arch=x86 --enable-runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross
    -prefix=i686-w64-mingw32- --enable-gpl --pkg-config=pkg-config
      libavutil      52. 34.100 / 52. 34.100
      libavcodec     55. 12.102 / 55. 12.102
      libavformat    55.  8.100 / 55.  8.100
      libavdevice    55.  2.100 / 55.  2.100
      libavfilter     3. 73.100 /  3. 73.100
      libswscale      2.  3.100 /  2.  3.100
      libswresample   0. 17.102 /  0. 17.102
      libpostproc    52.  3.100 / 52.  3.100
    [mpc8 @ 0037ECA0] max_analyze_duration 5000000 reached at 5015510 microseconds
    Input #0, mpc8, from 'd:\07-MemorySV8.mpc':
      Metadata:
        Track           : 7/17
        Artist          : Shy Keidar
        Title           : Memory
        Album           : Transient
        Year            : 2006
        Genre           : Electronic
        Composer        : Shy Keidar (1, 2, 4-15, 17)
                        : Unknown/Georgian folk (3)
                        : Johann Sebastian Bach (16)
        Comment         : www.steal-music.com
        Catalog number  : MPC001
        Engineer        : Shy Keidar
        Language        : Instrumental
                        : English
        License         : Creative Commons Music Sharing License
                        : http://creativecommons.org/licenses/by-nc-nd/2.0/deed-music
        Mastered by     : Shy Keidar
        Producer        : Shy Keidar
        Publisher       : Steal Music
        Release date    : 10 December 2006
      Duration: 00:03:49.04, start: 0.000000, bitrate: 262 kb/s
        Stream #0:0: Audio: musepack8, 44100 Hz, stereo, s16p
    At least one output file must be specified
    
  2. apr911

    I am happy to see this feature being implemented and look forward to seeing it released!

    I'd like to add to the request that the embedded metadata be used to assist searching for undefined metadata fields, basically reimplementing #668 and #839 using a field in the embedded metadata.

    1. Generate dynamic metadata search parameters from file name
    2. Override/add dynamic metadata search parameters from embedded metadata
    3. Retrieve dynamic metadata and apply.
    4. Override/reapply dynamic metadata with static metadata from embedded metadata

    Since some files might come with embedded metadata and users might not know how to clear it, I imagine a checkbox in the settings to enable/disable embedded metadata extraction would be helpful.

  3. Petr Nejedly repo owner

    mp4 episode example

    ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
      built with gcc 4.9.3 (GCC)
      configuration: --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-libass --enable-librtmp --enable-fontconfig --enable-libfreetype --enable-zlib --enable-libx264 --enable-libspeex --extra-libs='-lrtmp -lpolarssl -lws2_32 -lwinmm -lexpat -lfreetype -lfribidi -lz' --arch=x86 --enable-runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross-prefix=i686-w64-mingw32- --enable-gpl --pkg-config=pkg-config --bindir='/jenkins/workspace/FFmpeg - Win32/target'
      libavutil      55. 34.101 / 55. 34.101
      libavcodec     57. 64.101 / 57. 64.101
      libavformat    57. 56.101 / 57. 56.101
      libavdevice    57.  1.100 / 57.  1.100
      libavfilter     6. 65.100 /  6. 65.100
      libswscale      4.  2.100 /  4.  2.100
      libswresample   2.  3.100 /  2.  3.100
      libpostproc    54.  1.100 / 54.  1.100
    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '╬ù ╬┤╬┐¤à╬╗╬Á╬»╬▒ ╬▒¤ü¤ç╬»╬Â╬Á╬╣.mp4':
      Metadata:
        major_brand     : mp42
        minor_version   : 1
        compatible_brands: mp42mp41
        creation_time   : 2006-10-10T03:17:01.000000Z
        media_type      : 10
        genre           : TV Shows
        title           : Ted Mosby: Architect
        show            : How I Met Your Mother
        artist          : How I Met Your Mother
        season_number   : 2
        disc            : 2
        album           : How I Met Your Mother, Season 02
        episode_sort    : 4
        track           : 4
        date            : 2006
        episode_id      : 02x04 Ted Mosby: Architect
      Duration: 00:21:55.47, start: 0.000000, bitrate: 759 kb/s
        Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 320x176 [SAR 209:210 DAR 38:21], 628 kb/s, 23.98 fps, 23.98 tbr, 1199 tbn, 47.96 tbc (default)
        Metadata:
          creation_time   : 2006-10-10T03:17:01.000000Z
          handler_name    : Apple Video Media Handler
        Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
        Metadata:
          creation_time   : 2006-10-10T03:17:01.000000Z
          handler_name    : Apple Sound Media Handler
    
  4. Petr Nejedly repo owner

    wtv example:

    ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
      built with gcc 4.9.3 (GCC)
      configuration: --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-libass --enable-librtmp --enable-fontconfig --enable-libfreetype --enable-zlib --enable-libx264 --enable-libspeex --extra-libs='-lrtmp -lpolarssl -lws2_32 -lwinmm -lexpat -lfreetype -lfribidi -lz' --arch=x86 --enable-runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross-prefix=i686-w64-mingw32- --enable-gpl --pkg-config=pkg-config --bindir='/jenkins/workspace/FFmpeg - Win32/target'
      libavutil      55. 34.101 / 55. 34.101
      libavcodec     57. 64.101 / 57. 64.101
      libavformat    57. 56.101 / 57. 56.101
      libavdevice    57.  1.100 / 57.  1.100
      libavfilter     6. 65.100 /  6. 65.100
      libswscale      4.  2.100 /  4.  2.100
      libswresample   2.  3.100 /  2.  3.100
      libpostproc    54.  1.100 / 54.  1.100
    [mp2 @ 04202220] Header missing
    [mpeg2video @ 042015a0] Invalid frame dimensions 0x0.
        Last message repeated 13 times
    Input #0, wtv, from 'Top Chef_BRAVO_2011_01_19_21_58_00.wtv':
      Metadata:
        WM/MediaClassPrimaryID: db9830bd-3ab3-4fab-8a371a995f7ff74
        WM/MediaClassSecondaryID: ba7f258a-62f7-47a9-b21f4651c42a000
        Title           : Top Chef
        WM/SubTitle     : Restaurant Wars: One Night Only
        WM/SubTitleDescription: Fish fillet contest; chefs must open ``one night only'' restaurants in New York City.
        genre           : Cooking;Series;Reality
        WM/OriginalReleaseTime: 0
        WM/MediaCredits : ;;;
        service_provider: BRAVO
        service_name    : Bravo
        WM/MediaNetworkAffiliation: Satellite
        WM/MediaOriginalChannel: 33
        WM/MediaOriginalChannelSubNumber: 0
        WM/MediaOriginalBroadcastDateTime: 2011-01-19T05:00:00Z
        WM/MediaOriginalRunTime: 47961679000
        WM/MediaIsStereo: false
        WM/MediaIsRepeat: false
        WM/MediaIsLive  : false
        WM/MediaIsTape  : false
        WM/MediaIsDelay : false
        WM/MediaIsSubtitled: false
        WM/MediaIsMovie : false
        WM/MediaIsPremiere: false
        WM/MediaIsFinale: false
        WM/MediaIsSAP   : false
        WM/MediaIsSport : false
        WM/ParentalRating: TV-14
        WM/Provider     : MediaCenterDefault
        WM/VideoClosedCaptioning: true
        WM/WMRVEncodeTime: 2011-01-20 02:58:03
        WM/WMRVSeriesUID: !MCSeries!00812758
        WM/WMRVServiceID: !MCService!28455611
        WM/WMRVProgramID: !MCProgram!250807146
        WM/WMRVRequestID: 0
        WM/WMRVScheduleItemID: 0
        WM/WMRVQuality  : 3
        WM/WMRVOriginalSoftPrePadding: 420
        WM/WMRVOriginalSoftPostPadding: 180
        WM/WMRVHardPrePadding: 4294966996
        WM/WMRVHardPostPadding: 0
        WM/WMRVATSCContent: false
        WM/WMRVDTVContent: false
        WM/WMRVHDContent: false
        Duration        : 1720010000
        WM/WMRVEndTime  : 2011-01-20 03:00:55
        WM/WMRVBitrate  : 8.257536
        WM/WMRVKeepUntil: 4294967295
        WM/WMRVActualSoftPrePadding: 416
        WM/WMRVActualSoftPostPadding: 4294962852
        WM/WMRVContentProtected: false
        WM/WMRVContentProtectedPercent: 0
        WM/WMRVExpirationSpan: 9223372036854775807
        WM/WMRVInBandRatingSystem: 255
        WM/WMRVInBandRatingLevel: 255
        WM/WMRVInBandRatingAttributes: 0
        WM/WMRVWatched  : false
        WM/MediaThumbWidth: 0
        WM/MediaThumbHeight: 0
        WM/MediaThumbStride: 0
        WM/MediaThumbRet: 4
        WM/MediaThumbRatingSystem: 0
        WM/MediaThumbRatingLevel: 0
        WM/MediaThumbRatingAttributes: 0
        WM/MediaThumbAspectRatioX: 0
        WM/MediaThumbAspectRatioY: 0
        WM/MediaThumbTimeStamp: -4616189618054758400
      Duration: 00:02:49.97, start: 2.308662, bitrate: 8217 kb/s
        Stream #0:0[0x2c]: Subtitle: eia_608
        Stream #0:1[0x2d]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x480 [SAR 8:9 DAR 4:3], 29.97 fps, 29.97 tbr, 10000k tbn, 59.94 tbc
        Stream #0:2[0x2e]: Audio: mp2 (P[0][0][0] / 0x0050), 48000 Hz, stereo, s16p, 256 kb/s
        Stream #0:3[0xffffffff]: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 200x133 [SAR 96:96 DAR 200:133], 90k tbr, 90k tbn, 90k tbc
        Metadata:
          title           : TV Thumbnail
    
  5. Petr Nejedly repo owner

    MediaType no
    0 = Movie (old) 1 = Normal (Music) 2 = Audiobook 5 = Whacked Bookmark 6 = Music Video 9 = Movie 10 = TV Show 11 = Booklet 14 = Ringtone 21 = Podcast 23 = iTunes U

  6. Log in to comment