__TOC__

Channels, streaming modules, and seats

Linking channels to streaming modules

Channels are dynamically allocated to streaming modules for broadcasting their content. Multiple streaming modules can work together at the same time, you are not limited to having only one streamer per Shampoo instance.

Which Channels and how they are handled by one or several streaming modules is determined by the concept of Seats. Each Seat, formalized as a simple numeric value, represents a distinct and implicit "contract" between a Channel and a streaming module: whatever the streaming module used to broadcast content, a Channel is guaranteed to be handled in a unique, stable and predictable way by using a specific Seat number.

• The definition of the distinctive set of broadcasting features represented by a Seat is the responsibility of the streaming modules.
  • Those sets of distinctive features depend on the type of streaming involved. For example, if the streaming module broadcasts to Icecast or Shoutcast servers, they include the unique combination of server mountpoints, hosts and ports for both the final public streams and the live relay, if available.
• A streaming module can be linked to an unlimited number of different Seats. For practical reasons, the maximum number of Seats a single streaming module can handle at once is mostly limited by the bandwidth throughput and the CPU speed of the hosting machine.
  • A streaming module cannot be linked to the same Seat twice.
  • If two streaming modules are linked to a same Seat, the definition of this Seat must be identical.
• A Channel can only be bound to one Seat.
  • No two Channels can share the same Seat.
  • A Channel can change Seat at any time

Workflow

1. When a streaming module is ready for broadcasting, it advertises the Seats it can handle to Shampoo.

   1. Only the Seats not already taken care of by the streamer are advertised.
   2. The streaming module periodically polls Shampoo for Seats to handle, new Seats might become available from time to time.

2. Shampoo returns the list of unallocated Seats from those requested by the streaming module.

3. The streaming module notifies Shampoo that it will reserve those Seats for handling. Those Seats become then unavailable to other streamers.
4. The actual broadcasting can now start and Shampoo starts delivering content to the streaming module for each Seat independently.
5. If the streamer stops streaming a Seat, it signals it to Shampoo and the Seat becomes free again.
   1. If the streamer stops streaming for unexpected and uncontrolled reasons (network severed, program not responding, etc), then Shampoo will automatically detect the anomaly after a few moments and unilaterally free the Seat.
   2. Shampoo denies access to a streamer from handling a Seat if it didn't receive permission first. For example, if a Seat allocation has been switched from one streaming module to another, the first streamer cannot reclaim this Seat until it becomes free.

Examples of setups

The following diagram shows possible ways you can link Channels to Seats and streaming module instances. The first solution, "all-in-one", may be recommended if you don't need redundant fail-over streaming modules (i.e. a same Seat will be automatically handled by a second streaming module if the first one stops working), and you just have a handful of Channels to broadcast. It's the perfect solution when you start testing Shampoo. The last one, fully distributed, describes an extreme case as well, suitable only if you can spare lots of dedicated machines, and fail-over safety is a vital feature for you. In this example, Seats 1, 2 and 3 may be handled by any streaming module instances: the one that will handle the Channel bound to the Seat at any given time will depend on the availability of the streaming modules when first set up, or when a previous instance goes down.

The hybrid solution is perfectly indicated if you are a hosting company managing farms of radios. However, most of the time, when automatic fail-over is not required, you will have to adopt the mixed solution, allocating systems and resources on demand: plugging new Seats to the least crowded streaming module instances, until they're all full up and new streaming module hosts must be set up.

Using Liquidsoap as the streaming module

The Shampoo installation archive you've downloaded includes a Liquidsoap folder which contains all the configuration scripts and miscellaneous files needed to configure Liquidsoap as a streaming module for Shampoo.

Using these scripts, Liquidsoap will be able to talk to Shampoo and properly handle the streaming life-cycle of all its Channels, including, but not only:

• Broadcast multiple Channels content, in parallel, to Icecast or Shoutcast servers.
  • The same content can be streamed through different formats (codec and bitrate) to specific mountpoints or servers, simultaneously.
• Live performance relay using its internal Icecast/Shoutcast-compatible server.
• Signal processing: transcoding, stream pre-processing (silence detection, trimming, track crossfading, etc) and post-processing (volume normalization, dynamics compression, etc)

Technically, Shampoo and Liquidsoap communicate together via a JSON REST webservice. Liquidsoap gives orders through this webservice to Shampoo, which, in turn, answers to the request. The provided scripts provide all the plumbing for properly achieving this task. Shampoo will also give Liquidsoap access to its Datastore for streaming content, through FTP, HTTP or regular filesystem I/O.

Please share your own scripts if you improve upon the default ones or provide alternative features, for example, adding the ability to stream to other media than Icecast/Shoutcast servers.

Prerequisites

You need at least Liquidsoap 1.1.1 commit 8afa479 (May 22, 2014), or later, compiled with JSON support, libmagic binding and the following modules, to use the provided scripts. If you want to use older revisions of Liquidsoap, download older scripts shipped before 0.9.6. All scripts are incompatible with previous revisions of Liquidsoap.

Mandatory modules:

• ocaml-cry, for streaming to Icecast or Shoutcast servers.
• yojson, for reading and writing in JSON, not provided with Liquidsoap full source
• camomile, for converting charsets from and to UTF8, not provided with Liquidsoap full source
• ocaml-magic, for detecting file types, not provided with Liquidsoap full source

You might also need those optional modules, depending on your needs and how you configured Shampoo:

• ocaml-mad, for accepting MP3 files from Shampoo.
• ocaml-lame, if you want to stream in MP3 format.
• ocaml-ogg and ocaml-vorbis, for accepting Ogg Vorbis files from Shampoo, and streaming in this format. If you don't activate this option, you cannot use the bundled off-air media file offair.ogg which is encoded into that format. You must then either transcode it first, or provide your own media file.
• ocaml-faad, for accepting MP4 AAC files from Shampoo.
• ocaml-voaacenc and ocaml-aacplus, or ocaml-fdkaac, if you want to stream in AAC/AAC+ format.
• ocaml-flac, for accepting Flac files from Shampoo.
• ocaml-xmlplaylist and ocaml-lastfm, if you want to "scrobble" what your radios play.

This module is also highly recommended, but non-essential:

• ocaml-samplerate

You must also need the command-line tool wget if your Datastore serves its files through the HTTP or FTP protocol.

Note: libmagic version 5.18 and later is required if your Datastore serves MP3 files.

Checking plugin availability

• You can use Liquidsoap's --list-plugins switch to check for the presence of these plugins. For example, to check your build of Liquidsoap has JSON support:

$ liquidsoap --list-plugins | grep json
 + json_of
  Convert a value to a json string.
 + of_json
  Parse a json string into a liquidsoap value.

We can see here that this build of Liquidsoap has JSON support, else the command would've returned nothing. Simply substitute 'json' in the above example for 'samplerate', 'magic' or whichever plugin you wish to check for.

• To assess the version of your Liquidsoap build, use the --version switch. Example:

$ liquidsoap --version
Liquidsoap 1.1.1+scm (git://github.com/savonet/liquidsoap.git@3118f0299b894f4f24f92e14e868bd92d898adc9:20141019:171033)
Copyright (c) 2003-2014 Savonet team
Liquidsoap is open-source software, released under GNU General Public License.
See <http://liquidsoap.fm> for more information.

• For the system libmagic version, you can read the output of the --version switch used on the file program. Example:

$ file --version
file-5.18
magic file from /usr/share/misc/magic

Installation

• The installation of Liquidsoap is not covered here, please check the official documentation for guidance. Liquidsoap binaries and source can be downloaded from here. Your Linux or Unix distribution might also ship pre-packaged Liquidsoap versions, make sure the requirements are met though.

• You are not encouraged to install multiple instances of Liquidsoap per machine.

A single instance of Liquidsoap is virtually able to broadcast an almost unlimited number of Channels at once with the default scripts, but you're advised to install Liquidsoap on multiple remote machines as soon as the number of Channels or streams per Channel starts to grow. The actual number of Liquidsoap instances to install depends on the processing speed and the amount of memory you can allocate. For information, an Intel Atom N270 with 1gb of RAM can broadcast up to two transcoded MP3 streams of a single Channel, simultaneously. An Intel Atom 330 with 1gb or RAM, twice that number.

If you choose to install Liquidsoap on multiple machines, just repeat the following instructions for each instance of Liquidsoap. Each installation must have its own set of scripts and configuration files.

1. Compile Liquidsoap from source, or install a package that respects all the prerequisites above.
2. Extract the content of the Liquidsoap folder from the installation archive to a location where the Liquidsoap binary can access them, adjust their permissions if needed.
3. Take the time to properly edit the seat configuration file, according to your needs. You can use the provided example files as templates if needed. See the configuration section below.
4. Liquidsoap is now ready to run

Run

In order to launch Liquidsoap with the provided scripts, use the following command line syntax to invoke the program:

liquidsoap <SCRIPTS_PATH>/run.liq -- [-pid <PID_FILE>] [-socket <SOCKET_FILE>] [-log <LOG_FILE>] [-shampoo-host <SHAMPOO_URL>] [-offair-media <OFFAIR_FILE>] [-seat-conf <SEAT_CONFIGURATION_FILE>]

Where:

• SCRIPTS_PATH is the folder where you copied the provided script files. Example: /usr/local/etc/liquidsoap.
• PID_FILE is the full path name of the file referencing the PID number for the Liquidsoap running instance. Example: /var/run/liquidsoap/liquidsoap.pid. The default value if omitted will be a file called liquidsoap.pid and located within the default system "run" directory.
• SOCKET_FILE is the full path name of the socket file used by Liquidsoap for interacting with its own Telnet server. Internally, a few commands used by the provided scripts are only available through the Telnet server. Example: /var/run/liquidsoap/liquidsoap.sock. The default value if omitted will be a file called liquidsoap.sock and located within the default system "run" directory.
• LOG_FILE is the full path name of the file collecting all the program logs. Example: /var/log/liquidsoap/liquidsoap.log. The default value if omitted will be a file called liquidsoap.log and located within the default system "log" directory.
• SHAMPOO_URL is the full URL to the root of the running Shampoo instance. Example: http://beta2.radio.ddcr.biz. The default value if omitted will be http://localhost:8080/Shampoo.
• OFFAIR_FILE is the full path name of the sound file that will be played when no show is programmed on your radios. A default "offair" Ogg Vorbis file called offair.ogg is provided along with the scripts. Example: /usr/local/etc/liquidsoap/res/offair.mp3. The default value if omitted will be ''res/offair.ogg'.
• SEAT_CONFIGURATION_FILE is the full path name of the configuration file that is used for defining which Seats will be handled and how. An example, named shampoo_seats.json.example is provided along with the scripts. Example: /usr/local/etc/liquidsoap/configuration.json. The default value if omitted will be shampoo_seats.json. See section below for help on configuring this file.

It is recommended that Liquidsoap can automatically start when your system boots. For such a purpose, you can adapt, following the information written above, the Liquidsoap System V, or upstart, or whatever boot script that has been installed along with the program. Here's an example NetBSD rc.d script and a Debian init.d script and 'default' configuration file.

Configuration

Liquidsoap is unaware of the Channels created in Shampoo, and doesn't stream anything, unless you explicitly define which Seats it should handle.

Seat configuration file

This file represents a simplified JSON tree, written using a subset of this language. The differences with regular JSON are: keys and values are text only (no numeric values, no other objects, etc), and each leaf level of the tree must be of the same type.

1. The root node of the file (first level of the tree) is connected to a list of Seat identifiers, each of these identifiers corresponds to a specific Seat that must be handled by this instance of Liquidsoap: it is a numeric value formatted as a string.
2. The value for a each of these entries (second level) is composed of a set of alternative streams specific to the corresponding Seat, and a single entry for global parameters. You can stream the same content into different formats or toward different servers. Each entry is defined by a free-form key name. Only one entry is mandatory per Seat, and it must be named either "global" or "common": it defines the options shared by all streams. At least one entry beside the "common"/"global" node must be set.
3. Each value is then composed of a simple bag (third level) of key/value pairs. The list of keys is fixed, and all must be set

• Common options ("common"/"global" node):
  • "master_key" is the semi-secret passkey you've already defined for the private.ws.key option within the main Shampoo configuration file, shampoo.properties. It allows this instance of Liquidsoap to be able to talk to the Shampoo server. If this key is different than the one specified in the file, all communications are forbidden.
  • The three following parameters are related to how Lives can be hooked into Liquidsoap, and how Icecast or Shoutcast clients (like Mixxx, Virtual DJ, Traktor, Winamp with the Shoutcast DSP, and others) can connect. Of course, those settings must be revealed to anybody susceptible to perform Live sessions with such streaming clients. The access to this internal Shoutcast/Icecast-compatible server is protected. The credentials are Playlist-dependent. See the User guide for more info.
  • "live_port" is the port Liquidsoap should listen to, waiting for a streaming client to connect and establish a Live session.
  • "live_mountpoint" is the mountpoint to connect to. This parameter is only relevant for Icecast clients. leave it to "/" (slash) for Shoutcast clients.
  • "live_is_shoutcast" defines whether the internal relay server must be Shoutcast-compliant only. Set the value to "false" if your clients can speak Icecast, otherwise "true". You cannot accept both Icecast and Shoutcast clients for any given Seat at the same time.
  • "lastfm_login" and "lastfm_password" are the login and password of the last.fm account that will 'scrobble' what this Seat broadcasts. You must have build Liquidsoap with last.fm bindings to use this feature. Keep both fields blank ("") if you do not want to use it.
  • "queue_min_items" defines the minimum number of items to broadcast that should be present in the Channel internal queue at all times. "1" is the smallest accepted value and allows very reactive changes in playlist contents, at the expense of safety and QOS if the server hosting Shampoo is underpowered. See the related note in the Programme configuration page.
  • "default_public_stream_uri" is the default Shoutcast/Icecast URL listeners can connect to for listening to the Channel content. This URL is for example used by the internal radio player located in the Shampoo GUI.
• Stream-specific options:
  • Those options define how Liquidsoap should stream the Seat content.
  • "codec" is the format of the stream. Acceptable values are "MP3", "AAC", "AAC+", "VORBIS", "FLAC" and "WAV". The actual availability of those codecs depends on the build of your Liquidsoap program.
  • "bitrate" is the bitrate of the stream. Acceptable values are "320", "256", "192", "128", "96" and "64". This field is meaningless for FLAC and WAV codecs, and will not be read.
  • "icecast_host" is the hostname or IP of your Shoutcast or Icecast server. Multiple Seats can be streamed on the same server as long as they do not share the same port or mountpoint.
  • "icecast_user" and "icecast_password" correspond to the credentials required to connect to this server. The user must be set to "source" for Shoutcast.
  • "icecast_port" is the port to connect to.
  • "icecast_mountpoint" is the mountpoint to connect to. This parameter is only relevant for Icecast servers. leave it to "/" (slash) for Shoutcast servers.
  • "icecast_is_shoutcast" defines whether the server is Shoutcast-compliant only. Set the value to "false" if your server can speak Icecast, "true" otherwise.

If you modify settings in this file, Liquidsoap must be restarted for changes to take effect. Of course, you should only restart it if Liquidsoap's been properly configured first. Furthermore, you might not want to do it right now if this instance also handles other channels, you can for example wait for a scheduled maintenance or off-hours before restarting Liquidsoap.

Here's an example shampoo_channels.json. The Liquidsoap instance that can read this file will handle two Seats, numbered 1 and 2. It will accept Live hook connections with Shoutcast clients for both, with a different port. The first seat will only be broadcast to one stream, in MP3, whereas the second one will provide two MP3 and one Ogg Vorbis streams. The streams for Seat number 2 will be handled by the same Icecast server, but with a different mountpoint:

{"1": 
        {"common":
                {"master_key": "e87za5e5",
                "live_port": "8031",
                "live_mountpoint": "/",
                "live_is_shoutcast": "true",
                "lastfm_login": "",
                "lastfm_password": "",
                "queue_min_items": "2",
                "default_public_stream_uri": "http://localhost:8000/play.m3u"},
        "Single":
                {"codec": "MP3",
                "bitrate": "128",
                "icecast_host": "localhost",
                "icecast_user": "source",
                "icecast_password": "hackme!",
                "icecast_port": "8000",
                "icecast_mountpoint": "/",
                "icecast_is_shoutcast": "false"}            
        },
"2":
        {"common":
                {"master_key": "e87za5e5",
                "live_port": "8032",
                "live_mountpoint": "/",
                "live_is_shoutcast": "true",
                "lastfm_login": "",
                "lastfm_password": "",
                "queue_min_items": "2",
                "default_public_stream_uri": "http://icecast.ddcr.biz:9000/192mp3"},
        "MP3-64":
                {"codec": "MP3",
                "bitrate": "64",
                "icecast_host": "icecast.ddcr.biz",
                "icecast_user": "mp3sixtyfour",
                "icecast_password": "zer8vg1az5aa",
                "icecast_port": "9000",
                "icecast_mountpoint": "/64mp3",
                "icecast_is_shoutcast": "false"},
        "MP3-192":
                {"codec": "MP3",
                "bitrate": "192",
                "icecast_host": "icecast.ddcr.biz",
                "icecast_user": "mp3onehundredninetytwo",
                "icecast_password": "h8zz52z5zgtg",
                "icecast_port": "9000",
                "icecast_mountpoint": "/192mp3",
                "icecast_is_shoutcast": "false"},
        "OggVorbis-192":
                {"codec": "VORBIS",
                "bitrate": "192",
                "icecast_host": "icecast.ddcr.biz",
                "icecast_user": "vorbisonehundredninetytwo",
                "icecast_password": "65a2g8gvba88",
                "icecast_port": "9000",
                "icecast_mountpoint": "/192vorbis",
                "icecast_is_shoutcast": "false"}
}

An example file named shampoo_channels.json.example is available along with the scripts and can be used as a template.

Miscellaneous options in scripts

The file 1shampoo_conf.liq contains miscellaneous options you can easily tweak if you need to fine-tune the scripts behaviour.

• All parameters starting with the set prefix are internal Liquidsoap parameters. Please carefully read their description from the official Liquidsoap documentation before updating them. The full list of available settings can be found here.
• read_ahead_queue_time sets in milliseconds when Liquidsoap will start polling for a new track before the current track ends.
• min_playable_duration discards tracks for playing that are shorter that this duration, in milliseconds.
• dynamics_compression applies dynamics compression to the sound before broadcasting, 0 means disabled, 1 offers single band limiting and 2 multi-band limiting.
• poll_frequency defines how frequently Liquidsoap will query Shampoo for new available Seats and send heartbeat signals, in milliseconds. This is a global setting shared by all periodic background tasks.
• max_transient_queue_filling_errors specifies after how many consecutive tries during Queue building a critical error will be thrown.
• unallocate_on_temporary_disconnection allows transient network interruptions between the Icecast/Shoutcast servers and Liquidsoap to be considered critical errors.