Add MediaRouter support for Cast receiver communication

Issue #2900 resolved
Marshall Greenblatt created an issue

Background

Google Chrome supports communication with devices on the local network via the Cast and DIAL protocols. This functionality is exposed to JavaScript applications via the Cast SDK on desktop platforms, and is currently implemented as the closed-source Media Router component extension. The Cast SDK library sits on top of a web platform API called the Presentation API which defines the process through which a device is selected.

Casting from Google Chome is a multi-step process where:

  1. The JavaScript application requests access, or the user selects the “Cast…” option from the top menu.
  2. The user clicks play on the target device in the Cast selection dialog.
  3. The JavaScript application is given a session object, but details of the selected device (such as name, ID, etc) remain hidden for privacy reasons.

There are different ways in which Chromium can play via an external device:

  1. Tab or screen casting, where the audio/video content is streamed directly from the browser to the Cast device (supported via the Cast protocol only).
  2. Communication with a receiver application hosted on a Cast device, where an identifying token is sent to the receiver application and is then used to initiate playback directly via a backend service (supported via the Cast and DIAL protocols).

CEF Considerations

Using the existing Google Chrome cast implementation from CEF is problematic for the following reasons:

  • The Media Router component extension works in Chromium (enable “Media Router Component Extension” via chrome://flags), but it has dependencies on the Chrome extension system that are not currently supported by CEF (see issue #1947).
  • The user experience described above is not ideal for desktop applications that embed CEF and would like to develop a fully integrated and streamlined cast experience. For example, a client application may wish to integrate the list of devices into an application-specific device picker UI and trigger playback via that UI without additional intermediate steps.

While it would be nice to support all casting methods, there is substantial complexity involved with tab or screen casting. We will therefore focus at this time on only supporting communication with a receiver application.

Implementation Details

Chromium provides the following interfaces for device discovery, control and communication:

  • MediaRouter, retrieved via MediaRouterFactory::GetApiForBrowserContext(brower_context)
    -> CreateRoute(source_id, sink_id, web_contents, origin, callback) => callback receives async MediaRoute
    -> GetCurrentRoutes() => returns vector<MediaRoute>
    -> SendRouteMessage(route_id, message)
    -> AddPresentationConnectionStateChangedCallback(route_id, callback) => callback receives async PresentationConnectionStateChangeInfo (states include connecting, closed, etc)
    -> GetMediaController(route_id, MediaController, MediaStatusObserver) => ability to control/observe playback state
    -> TerminateRoute(route_id)
  • MediaSinksObserver(MediaRouter, source_id, origin), registered with MediaRouter via the constructor
    -> OnSinksUpdated(vector<MediaSink> sinks, vector<Origin> origins)
  • MediaRoutesObserver(MediaRouter, source_id), registered with MediaRouter via the constructor
    -> OnRoutesUpdated(vector<MediaRoute> routes, vector<route_id> joinable_route_ids)
  • RouteMessageObserver(MediaRouter, source_id), registered with MediaRouter via the constructor
    -> OnMessagesReceived(vector<RouteMessage> messages)

Identifiers are as follows:

  • source_id (MediaSource::Id) is an application-specific media source URN which will be already known to the client application (e.g. urn:x-cast:com.mycompany.cast) or which can be retrieved using methods like MediaSource::FromPresentationUrl.
  • sink_id (MediaSink::Id) uniquely identifies a MediaSink; retrieved via MediaSinksObserver::OnSinksUpdated.
  • route_id (MediaRoute::Id) uniquely identifies a MediaRoute; retrieved via MediaRoute::CreateRoute, MediaRoute::GetCurrentRoutes or MediaRoutesObserver::OnRoutesUpdated.

CEF will expose this functionality via a new C/C++ API that generally follows the Chromium organization and naming patterns (e.g. CefMediaRoute, CefMediaSink, CefMediaSource). Control and callback interfaces will be distributed among these objects as seems most intuitive.

Comments (18)

  1. Marshall Greenblatt reporter

    It looks like SendRouteMessage and RouteMessageObserver aren’t currently supported with Cast (see NOTIMPLEMENTED in CastMediaRouteProvider::SendRouteMessage and CastMediaRouteProvider::StartListeningForRouteMessages).

    The alternative is the RoutePresentationConnectionPtr passed to the MediaRouteResponseCallback after calling MediaRouter::CreateRoute. The connection_receiver replaces RouteMessageObserver and the connection_remote replaces SendRouteMessage .

    This is implemented in CastMediaRouteProvider using CastSessionClient, CastActivityRecord and CastActivityManager.

  2. Marshall Greenblatt reporter

    The debug console for Cast receiver apps can be accessed as follows:

    1. Register the device S/N in the Google Cast Developer Console. Wait for notification that registration is complete and then power cycle the Cast device.
    2. Launch your receiver app on the Cast device. For example, using the http://tests/media_router test in cefclient enter cast:[appId]?clientId=123456789 as the Source, select the Cast device as the Sink, and click the “Create Route” button. The [appId] value is the application ID that was registered with Google and uniquely identifies your receiver app.
    3. Open Google Chrome and enter chrome://inspect/#devices in the address bar. Wait for the device list to be populated. Click the “inspect” link next to the Cast device.

  3. Marshall Greenblatt reporter

    The following is an example of messages received and sent with a Chromecast Ultra device.

    Source URN: cast:[appId]?clientId=123456789

    Receive the following messages after connection:

    {
      "clientId": "123456789",
      "message": {
        "action": "cast",
        "receiver": {
          "capabilities": [
            "video_out",
            "audio_out"
          ],
          "displayStatus": null,
          "friendlyName": "Living Room TV",
          "isActiveInput": null,
          "label": "",
          "receiverType": "cast",
          "volume": null
        }
      },
      "timeoutMillis": 0,
      "type": "receiver_action"
    }
    
    {
      "clientId": "123456789",
      "message": {
        "appId": "[appId]",
        "appImages": [],
        "displayName": "MyApp",
        "namespaces": [
          {
            "name": "urn:x-cast:com.google.cast.cac"
          },
          {
            "name": "urn:x-cast:com.google.cast.debugoverlay"
          },
          {
            "name": "urn:x-cast:com.myapp.chromecast.secure.v1"
          },
          {
            "name": "urn:x-cast:com.google.cast.test"
          },
          {
            "name": "urn:x-cast:com.google.cast.broadcast"
          },
          {
            "name": "urn:x-cast:com.google.cast.media"
          }
        ],
        "receiver": {
          "capabilities": [
            "video_out",
            "audio_out"
          ],
          "displayStatus": null,
          "friendlyName": "Living Room TV",
          "isActiveInput": null,
          "label": "",
          "receiverType": "cast",
          "volume": {
            "controlType": "attenuation",
            "level": 0.8999999761581421,
            "muted": false,
            "stepInterval": 0.05000000074505806
          }
        },
        "senderApps": [],
        "sessionId": "75b99bde-abcd-42be-1234-5f97baafef91",
        "statusText": "MyApp",
        "transportId": "75b99bde-abcd-42be-1234-5f97baafef91"
      },
      "timeoutMillis": 0,
      "type": "new_session"
    }
    

    Send the following message:

    {
      "type": "app_message",
      "message": {
        "sessionId": "75b99bde-abcd-42be-1234-5f97baafef91",
        "namespaceName": "urn:x-cast:com.myapp.chromecast.secure.v1",
        "message": {
          "type": "getInfo",
          "payload": {
            "remoteName": "Living Room TV"
          }
        }
      },
      "sequenceNumber": 1,
      "timeoutMillis": 3000,
      "clientId": "123456789"
    }
    

    Get the following response:

    {
      "clientId": "123456789",
      "message": {
        "message": "{\"payload\":{\"remoteName\":\"Living Room TV\",\"modelDisplayName\":\"unknown\",\"brandDisplayName\":\"google\",\"deviceID\":\"[deviceId]\",\"deviceType\":\"CASTVIDEO\",\"deviceAPI_isGroup\":false,\"accountReq\":\"\",\"activeUser\":\"\",\"clientID\":\"[clientId]\",\"libraryVersion\":\"1.0.0\",\"publicKey\":\"empty\",\"myappError\":0,\"scope\":\"streaming\",\"status\":101,\"statusString\":\"OK\",\"tokenType\":\"accesstoken\",\"version\":\"3.2.2\"},\"type\":\"getInfoResponse\"}",
        "namespaceName": "urn:x-cast:com.myapp.chromecast.secure.v1",
        "sessionId": "75b99bde-abcd-42be-1234-5f97baafef91"
      },
      "timeoutMillis": 0,
      "type": "app_message"
    }
    

    Note that:

    • clientId is specified by the client app as part of the cast URI when initiating the route connection.
    • sessionId is returned by the receiver app and must be included in future messages from the client app.

  4. Marshall Greenblatt reporter

    Add support for media device discovery and messaging (fixes issue #2900)

    Chromium supports communication with media devices on the local network via the Cast and DIAL protocols. This takes two primary forms:

    1. Messaging, where strings representing state information are passed between the client and a dedicated receiver app on the media device. The receiver app communicates directly with an app-specific backend service to retrieve and possibly control media playback.
    2. Tab/desktop mirroring, where the media contents are streamed directly from the browser to a generic streaming app on the media device and playback is controlled by the browser.

    This change adds support for device discovery and messaging (but not mirroring) with functionality exposed via the new CefMediaRouter interface.

    To test: Navigate to http://tests/media_router in cefclient and follow the on-screen instructions.

    → <<cset 03fd5b15da9a>>

  5. Marshall Greenblatt reporter

    Add support for media device discovery and messaging (fixes issue #2900)

    Chromium supports communication with media devices on the local network via the Cast and DIAL protocols. This takes two primary forms:

    1. Messaging, where strings representing state information are passed between the client and a dedicated receiver app on the media device. The receiver app communicates directly with an app-specific backend service to retrieve and possibly control media playback.
    2. Tab/desktop mirroring, where the media contents are streamed directly from the browser to a generic streaming app on the media device and playback is controlled by the browser.

    This change adds support for device discovery and messaging (but not mirroring) with functionality exposed via the new CefMediaRouter interface.

    To test: Navigate to http://tests/media_router in cefclient and follow the on-screen instructions.

    → <<cset 409e14fe5aff>>

  6. Marshall Greenblatt reporter

    Add support for media device discovery and messaging (fixes issue #2900)

    Chromium supports communication with media devices on the local network via the Cast and DIAL protocols. This takes two primary forms:

    1. Messaging, where strings representing state information are passed between the client and a dedicated receiver app on the media device. The receiver app communicates directly with an app-specific backend service to retrieve and possibly control media playback.
    2. Tab/desktop mirroring, where the media contents are streamed directly from the browser to a generic streaming app on the media device and playback is controlled by the browser.

    This change adds support for device discovery and messaging (but not mirroring) with functionality exposed via the new CefMediaRouter interface.

    To test: Navigate to http://tests/media_router in cefclient and follow the on-screen instructions.

    → <<cset e6096029c751>>

  7. Marshall Greenblatt reporter

    Add CefMediaSink::GetIconType (see issue #2900)

    This attribute is useful for identifying different classes of cast devices without first requiring a connection (CAST, CAST_AUDIO, CAST_AUDIO_GROUP, etc).

    → <<cset ff0c36b465b3>>

  8. Marshall Greenblatt reporter

    Add CefMediaSink::GetIconType (see issue #2900)

    This attribute is useful for identifying different classes of cast devices without first requiring a connection (CAST, CAST_AUDIO, CAST_AUDIO_GROUP, etc).

    → <<cset 1f55c0d76292>>

  9. Marshall Greenblatt reporter

    Add CefMediaSink::GetIconType (see issue #2900)

    This attribute is useful for identifying different classes of cast devices without first requiring a connection (CAST, CAST_AUDIO, CAST_AUDIO_GROUP, etc).

    → <<cset 4e13d9991b74>>

  10. Marshall Greenblatt reporter

    Remove duplicate local_discovery file entries (see issue #2900)

    This fixes duplicate symbol errors when linking libcef on Linux. These files are now included by default with enable_service_discovery=true.

    → <<cset 640cd0f4115d>>

  11. Marshall Greenblatt reporter

    Remove duplicate local_discovery file entries (see issue #2900)

    This fixes duplicate symbol errors when linking libcef on Linux. These files are now included by default with enable_service_discovery=true.

    → <<cset 1fac1e551d13>>

  12. Log in to comment