Support audio track selection for streaming

[Petr Nejedly]

These are the types of files we need to support:

• a file with a single audio track (native + transcoded) - already works
• a file with multiple audio tracks (one per language) (native + transcoded)
• a file with multiple audio tracks (multiple per language) (native + transcoded)

To deliver these we need to:

• have a profile setting whether the renderer can swap tracks in native files
• have a profile setting whether the renderer can swap tracks in transcoded files (maybe the same flag as above)
• have a console setting on whether all tracks should be delivered when audio needs transcoding (due to low CPU machines)
• have a console setting on the preferred audio language

Then we deliver it as follows:

• when the audio track is natively playable and on position 0, just natively deliver
• when the audio track is natively playable and not on position 1, but renderer can swap tracks, just natively deliver
• when the audio track is natively playable and not on position 1, and renderer can't swap tracks, deliver using remuxing, selected track first (and possibly all other tracks too, as per console setting and if profile supports swapping)
• when the audio track is not natively playable, deliver using transcoding on the first position (and possibly all other tracks too, as per console setting and if profile supports swapping)
• when there are multiple audio tracks for the same language, prefer the native one (if present), and deliver using one of the above scenarios
• the CDS API should return all available audio tracks, as per Will's suggestion bellow
• also see #570

http://forum.serviio.org/viewtopic.php?f=3&t=1104

http://forum.serviio.org/viewtopic.php?f=10&t=3161&start=10#p20664

http://forum.serviio.org/viewtopic.php?f=3&t=17307#p93029

[Former user]

As an alternative to this - transcode/remultiplex ALL audio tracks in the container.

Or, in the most flexible way (extension of the above), transcode/remultiplex a set of tracks that matches a configured criterion.

Reason: Language can be selected on the fly by the user (e.g. me operating the remote control of the TV - works with Panasonic VIERA).

A typical implementation would be

(Step 1) Probe file with ffmpeg -i: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'x.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf52.64.2 Duration: 02:27:31.24, start: 0.000000, bitrate: 3774 kb/s Stream #0.0(und): Video: h264, yuv420p, 720x576 [PAR 64:45 DAR 16:9], 3000 kb/s, 25 fps, 25 tbr, 50 tbn, 50 tbc Stream #0.1(eng): Audio: ac3, 48000 Hz, 2 channels, s16, 384 kb/s Stream #0.2(ger): Audio: ac3, 48000 Hz, 2 channels, s16, 384 kb/s

> One video stream, 0.0, and two audio streams, 0.1 & 0.2 available

(Step 2) Invoke transcoding ffmpeg -i <video options> -map 0.0 <first audio channel options> -map 0.1 <output file options> [ <audio channel options> -map 0.x -newaudio ]

The term in brackets [ ] at the end needs to be repeated for each extra audio channel. In the example above, it would be only one extra track. But there may be more.

Extra audio channel options incl. -newaudio really need to be at the end of the command line, otherwise ffmpeg doesn't work!

The audio transcoding options should be recalculated for each track, based on the source track format and the target audio format specified in the transcoding profile. I think the number of channels has to stay the same for each track for otherwise ffmpeg fails.

If the profile specifies no target acodec, -acodec copy all tracks... Seems reasonable.

[Former user]

Since not many renderers will support multiple audio tracks in the remuxed stream, and because it's not all that likely that people will really want to switch language once they are watching a film, would it not be better to do this PS3 Media Server-style whereby Serviio scans a file with FFMPEG as it's added to the library, sees multiple audio tracks and makes a note of this. When the DLNA client browses the containing folder it creates a new DLNA folder resource for this item, containing a separate M2TS remuxed entry per audio track, and suffixing each filename name with the label of that particular audio stream?

[Former user]

Personally, I much prefer having the first option... that is, Serviio should send ALL of the audio tracks at once and let the client choose. (I also have a Panasonic Viera device, and this works very nicely.)

The PS3-style solution of presenting a separate file for each audio track makes for a very cluttered up list. I suppose it's possible that some devices don't allow for choosing the audio track, though. Maybe what's need is the option to tell Serviio which way to do it when transcoding. :)

[patters]

I guess we'd need an additional attribute in the device profile called something like "MultipleAudioTrackAware" and set the behaviour accordingly between the two methods outlined above. As for clutter, that's why it would be best to have them in a subfolder per movie - that way you'd only see the list if you were interested in that particular item.

[Thomas Perschak]

I dont like this PS3MS style with a sub-folder - imagine having two languages and three subtitles - this would mean you have 6 films you need to choose from.

Initially I suggested to have a root control folder to change the language and select a prefered sub-title - this way you could do your setup via TV. You could also have there re-scan and shutdown server buttons - I think this would be nice(r).

[Former user]

I also strongly support this request. I think it should even be easy. ;) If I remember correctly, ffmpeg has an option to stream all audio tracks. I also agree with tombert that having a separate entry for each track is not a great idea.

Just selecting a preferred track for everything is not great either, though... especially if it's just for a specific language. I have lots of movies with an "commentary" audio track I'd like to be able to select... but it's also in English like the main track.

[patters]

Having recently bought gaming headphones with Dolby Headphone processing I'm interested in this improvement as I've noticed that quite a few of my MP4 files have both 2ch AAC and 5.1ch AC3 tracks which I can't hear.

[Former user]

...and with the upcoming ffmpeg/avconv command line syntax, "transwrapping" (bundling all audio tracks) should be even easier:

"-map 0:a" will copy/transcode & remux all audio channels of the first input file.

pleeease, it would increase usability a lot

[Former user]

bump for adding this ability

[Petr Nejedly]

Issue #547 was marked as a duplicate of this issue.

[patters]

I've been thinking some more about how this could be implemented to catch most scenarios and have written it up here: http://forum.serviio.org/viewtopic.php?f=11&t=6688&p=48290#p48290

[patters]

Updated that forum post above with a UI mockup

[Former user]

this feature is a must!

[Will Lunniss]

This should also allow CDS clients to dynamically select the audio track.

This could be implemented by returning a list of the available audio tracks during browse e.g.:

    <audioTracks>
       <audioTrack codec="dts" channels="5.1" language="Engligh" id="0" prefered="true">
       <audioTrack codec="aac" channels="5.1" language="Engligh" id="1">
       ...
    </audioTracks>

and then the client can optionally pass &audioTrack=[id] when requesting a stream.

[Former user]

Hi !

I looked forward to this option is added. I can not see my mkv because they are all multi-language, and my prefered audio track is not the first.

are there no way to make a plugin to force the selection of audio track corresponding to the desired country code (example: French 0x081)?

regards

David

[Former user]

I will pay again for this feature!

[Former user]

Hope this feature will be implemented soon, as many movies in MKV have several audio languages, and the first is not English in many cases, so the movie can not be watched..

[Former user]

Hi ! This feature is required imho ! I need it so much ! Thank you !

[Former user]

As a PRO user, let me ask you for any update/progress regarding to this issue? It's a little bit old topic but I believe for a quick solution. This solve my daily task of changing audio layers in the mkv files. Please take care of that. Thank you for your efforts.

[Giovanni Musto]

Any progress on this issue? My Samsung TV won't play anything beside the first audio track, and it's very annoying.

[Petr Nejedly]

Implemented for both, DLNA and CDS (API docs updated)

[Andrew]

Does this allow for the selection of the 5.1 audio track? All my recordings have 5.1 on stream 0:2 and mono or stereo on 0:1. I can't seem to figure out a way to force the 5.1 track? I tried sending all audio tracks, but it doesn't seem to be working. Thanks!

[Petr Nejedly]

it send all of them even when trancoding. Otherwise you can give a selected one a priority - but you have to identify the track with a (unique) language code and then use that language code on the Languages tab in the console.

[Andrew]

Even though the dvd player supports multiple tracks, pressing the audio select button shows only the one track. As for giving a track priority, here are the streams. There is no language. I've tried pasting the whole audio stream name/text to no avail. Thanks again.

Input #0, mpegts, from '1071_20160420010000.mpg': Duration: 00:59:59.41, start: 84190.617778, bitrate: 8488 kb/s Stream #0:0[0x31]: Video: mpeg2video (Main), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc Stream #0:10x34: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s Stream #0:20x35: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 96 kb/s