SBS show list

Issue #26 resolved
Bit Bucket created an issue

+1 for having SBS back. Good to have webdl working well again - and it is working on OS X. I was just wondering if it is an aspect of the SBS OnDemand service that means not all shows appear. eg: http://www.sbs.com.au/ondemand/video/11900483725/all-watched-over-by-machines-of-loving-grace-love-and-power is viewable ondemand but not visible in any listing of docos etc.

I presume this is a facet of SBS system. (This is different to Issue24 - these shows don't appear as available via Webdl)

Comments (8)

  1. Robert Denham

    Yes, I noticed that too. The list of available programs comes from

    FULL_VIDEO_LIST = BASE + "/api/video_feed/f/Bgtm9B/sbs-section-programs/?form=json"

    but this doesn't have all the shows, movies in particular seem to be missing.

    Its not hard to make a script based on sbs.py to download the show you want. In your example, the video id is the numeric part (11900483725) and the title is all-watched-over-by-machines-of-loving-grace-love-and-power. That's pretty much all you need to get the url and then download it.

    I'm not smart enough to work out how to scrape the titles and id's from sbs.

  2. Bit Bucket reporter

    Hi Robert Thanks for that. No matter what you think, you're much smarter than me!

    I get the idea that I could make a one-off version of sbs.py to download a particular show using the id and title, rather than search through the list SBS provides... but I'm not a Python man (I can manage a bit of bash). I'll have a tinker and see what I can figure out. Much obliged.

  3. Bit Bucket reporter

    Hi Robert - Looking at sbs.py. I can imagine that %s in VIDEO_URL is the numeric id, but am lost about how the code could be turned into something that downloaded that specific video. Any pointers you can offer? Don't worry if not. I guess I'm going to have to learn python.

  4. Robert Denham

    Hi, heres an example, not really tidied up or anything but should be a start:

    !/usr/bin/env python
    
    from common import grab_html, grab_json, grab_xml, download_hls, Node
    
    import os
    import json
    import sys
    
    ## vidoe url should look like:
    #vidurl = "www.sbs.com.au/ondemand/video/467883075798/blood-brother"
    vidurl = sys.argv[1]
    (baseurl, title) = os.path.split(vidurl)
    (tmp, video_id) = os.path.split(baseurl)
    
    
    NS = {
        "smil": "http://www.w3.org/2005/SMIL21/Language",
    }
    BASE = "http://www.sbs.com.au"
    # VIDEO_URL="www.sbs.com.au/ondemand/video/467883075798/blood-brother"
    
    #video_id = "467883075798"
    VIDEO_URL = BASE + "/ondemand/video/single/%s" % video_id
    def get_player_params(doc):
        for script in doc.xpath("//script"):
           if not script.text:
                 continue
           for line in script.text.split("\n"):
               s = "var playerParams = {"
               if s in line:
                   p1 = line.find(s) + len(s) - 1
                   p2 = line.find("};", p1) + 1
                   if p1 >= 0 and p2 > 0:
                       return json.loads(line[p1:p2])
    
    
    
    doc = grab_html(VIDEO_URL ,0)
    player_params = get_player_params(doc)
    release_url = player_params["releaseUrls"]["html"]
    
    doc = grab_xml(release_url, 0)
    video = doc.xpath("//smil:video", namespaces=NS)[0]
    video_url = video.attrib["src"]
    filename = "title" + ".ts"
    download_hls(filename, video_url)
    ## should leave a file called title.mp4
    ## just move this file to whatever.
    os.rename('title.mp4', '%s.mp4' % title)
    
  5. Bit Bucket reporter

    Awesome. Thanks so much. I was getting a little further than my last post, as I worked out a bit more about how webdl was working generally, but this is great. A couple of errors came across with the copy/paste

    in the first line and def has become ef on line 24 -

    It makes so much more sense now. Thanks.

  6. Bit Bucket reporter

    That was meant to say # in the first line and def has become ef on line 24 but the # got stripped. Ooops. Thanks again

  7. Robert Denham

    yeah sorry that was a paste error on my part. Hope it helps. Could be made a little more robust of course.

  8. Log in to comment