online link containing "live" processes incorrectly in rss feed

As detailed below, if I add this link to online/livestreams it processes correctly even though the link is not currently active and will not display.

rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1

The log shows

{{{ 2011-11-23 00:25:46,799 DEBUG [RepositoryServerResource] Updating repository with values: OnlineRepository [fileType=VIDEO, repoType=LIVE_STREAM, repositoryUrl=rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1, thumbnailUrl=https://sites.google.com/site/jhbrsstest/cielotv.jpg]

2011-11-23 00:25:47,296 DEBUG [OnlineRepositoryDAOImpl] Creating a new Repository (url = rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1)

2011-11-23 00:25:47,717 DEBUG [OnlineContentCacheDecorator] Stored online content 'rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1' in the cache (online_feeds), returning it 2011-11-23 00:25:47,717 DEBUG [FeedUpdaterThread] Retrieving information about the video stream 'rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1' 2011-11-23 00:25:47,717 DEBUG [FFMPEGWrapper] Invoking FFMPEG to retrieve media information for file: rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1 2011-11-23 00:25:47,717 DEBUG [ProcessExecutor] Starting C:\Program Files (x86)\Serviio\bin\..\lib/ffmpeg.exe -i rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1

2011-11-23 00:25:49,017 WARN [FeedUpdaterThread] Failed to retrieve online item information for rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1. It might not play. java.io.IOException: org.serviio.library.local.metadata.extractor.InvalidMediaFormatException: Unknown video file type. at org.serviio.library.online.metadata.FeedUpdaterThread.retrieveAVMetadata(FeedUpdaterThread.java:258) at org.serviio.library.online.metadata.FeedUpdaterThread.retrieveTechnicalMetadata(FeedUpdaterThread.java:197) at org.serviio.library.online.metadata.FeedUpdaterThread.run(FeedUpdaterThread.java:98) Caused by: org.serviio.library.local.metadata.extractor.InvalidMediaFormatException: Unknown video file type. at org.serviio.library.metadata.FFmpegMetadataRetriever.validateCodecsFound(FFmpegMetadataRetriever.java:318) at org.serviio.library.metadata.FFmpegMetadataRetriever.retrieveMetadata(FFmpegMetadataRetriever.java:68) at org.serviio.library.online.metadata.FeedUpdaterThread.retrieveAVMetadata(FeedUpdaterThread.java:240) ... 2 more 2011-11-23 00:26:02,683 DEBUG [OnlineContentCacheDecorator] Found online content 'rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1' in the cache (online_feeds), returning it

2011-11-23 00:26:02,716 DEBUG [OnlineContentCacheDecorator] Found online content 'rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1' in the cache (online_feeds), returning it }}}

However If I add this as the 4th link in an rss feeed it generates the following error message for every link including the valid ones and prevents the other valid links in the rss from displaying. This should not happen. The valid links should process and display correctly. Additionally the invalid lik should generate messages similar to those if the link is added to serviio by itself. The live ; message appears erroneous.

{{{ 2011-11-23 08:55:00,534 DEBUG [OnlineLibraryManager] Feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss expired, will reload it 2011-11-23 08:55:00,534 DEBUG [OnlineLibraryManager] Feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss not in cache yet, loading it 2011-11-23 08:55:00,534 DEBUG [FeedParser] Parsing feed 'https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss' 2011-11-23 08:55:00,754 DEBUG [DiscoverySSDPMessageListener] Received a valid M-SEARCH message for search target upnp:rootdevice from address /192.168.1.10:57538 2011-11-23 08:55:00,754 DEBUG [DiscoverySearchResponder] Sending 1 M-SEARCH response message(s) to /192.168.1.10:57538 2011-11-23 08:55:01,029 DEBUG [FeedParser] Found matching url extractor (LiveFeeds) for feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss 2011-11-23 08:55:01,029 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Greek Mega TV 2011-11-23 08:55:01,380 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (LiveFeeds): The reference to entity "live" must end with the ';' delimiter. 2011-11-23 08:55:01,381 DEBUG [FeedParser] Cannot add feed entry of feed Live TV Stations because of invalid metadata. Message: Unknown feed entry URL. 2011-11-23 08:55:01,381 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Greek koul TV 2011-11-23 08:55:01,821 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (LiveFeeds): The reference to entity "live" must end with the ';' delimiter. 2011-11-23 08:55:01,821 DEBUG [FeedParser] Cannot add feed entry of feed Live TV Stations because of invalid metadata. Message: Unknown feed entry URL. 2011-11-23 08:55:01,821 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Italian Cielo TV 2011-11-23 08:55:02,354 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (LiveFeeds): The reference to entity "live" must end with the ';' delimiter. 2011-11-23 08:55:02,354 DEBUG [FeedParser] Cannot add feed entry of feed Live TV Stations because of invalid metadata. Message: Unknown feed entry URL. 2011-11-23 08:55:02,354 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Greek gw5 TV 2011-11-23 08:55:02,873 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (LiveFeeds): The reference to entity "live" must end with the ';' delimiter. 2011-11-23 08:55:02,873 DEBUG [FeedParser] Cannot add feed entry of feed Live TV Stations because of invalid metadata. Message: Unknown feed entry URL. 2011-11-23 08:55:02,873 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Crete TV 2011-11-23 08:55:03,369 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (LiveFeeds): The reference to entity "live" must end with the ';' delimiter. 2011-11-23 08:55:03,369 DEBUG [FeedParser] Cannot add feed entry of feed Live TV Stations because of invalid metadata. Message: Unknown feed entry URL. 2011-11-23 08:55:03,369 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Cyprus TV 2011-11-23 08:55:03,818 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (LiveFeeds): The reference to entity "live" must end with the ';' delimiter. 2011-11-23 08:55:03,818 DEBUG [FeedParser] Cannot add feed entry of feed Live TV Stations because of invalid metadata. Message: Unknown feed entry URL. 2011-11-23 08:55:03,818 DEBUG [OnlineContentCacheDecorator] Stored online content 'https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss' in the cache (online_feeds), returning it 2011-11-23 08:55:03,819 DEBUG [OnlineLibraryManager] Feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss will expire in the cache on Wed Nov 23 16:55:03 EST 2011 }}}

If I remove the link containing "live" from the rss the rss is processed correctly and the links display.

{{{ 2011-11-22 16:52:06,076 DEBUG [OnlineContentCacheDecorator] Found online content 'https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss' in the cache (online_feeds), returning it 2011-11-22 16:52:06,076 DEBUG [OnlineLibraryManager] Feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss expired, will reload it 2011-11-22 16:52:06,076 DEBUG [OnlineLibraryManager] Feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss not in cache yet, loading it 2011-11-22 16:52:06,076 DEBUG [FeedParser] Parsing feed 'https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss' 2011-11-22 16:52:06,501 DEBUG [FeedParser] Found matching url extractor (LiveFeeds) for feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss 2011-11-22 16:52:06,501 DEBUG [FeedItemUrlExtractor] LiveFeeds: Starting extraction of url for item: Greek koul TV 2011-11-22 16:52:07,034 DEBUG [FeedItemUrlExtractor] LiveFeeds: Finished extraction of url: ContentURLContainer [fileType=VIDEO, contentUrl=rtmp://koul.str.viiideo.gr/live/mp4:koul5.mp4, thumbnailUrl=https://sites.google.com/site/jhbrsstest/greektv.jpg, live=true, expiresImmediately=false, ] 2011-11-22 16:52:07,035 DEBUG [FeedParser] Added feed item 1: 'Greek koul TV' 2011-11-22 16:52:07,035 DEBUG [OnlineContentCacheDecorator] Stored online content 'https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss' in the cache (online_feeds), returning it 2011-11-22 16:52:07,035 DEBUG [OnlineLibraryManager] Feed https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss will expire in the cache on Wed Nov 23 00:52:07 EST 2011 2011-11-22 16:52:07,035 DEBUG [TechnicalMetadataCacheDecorator] Found technical metadata for online item 'rtmp://koul.str.viiideo.gr/live/mp4:koul5.mp4' in the cache (online_technical_metadata), returning it }}}

Here is a shortened rss containing the bad link that I've installed at https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStationsTest.rss so you can replicate the bug by adding that feed to serviio. The LiveFeeds.groovy below will also access this rss if run standalone.

Note that https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStations.rss does not contain the live link so you can also access it to see that the groovy works fine.

{{{ <?xml version="1.0" encoding="UTF-8" ?>

<channel> <title>Live TV Stations</title> <link>https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStationsTest.rss</link> <description>Live TV streams</description>

<item>
    <title>Greek koul TV</title>
    <link>http://link1</link>
    <media:stream url="rtmp://koul.str.viiideo.gr/live/mp4:koul5.mp4" />
    <media:thumbnail url="https://sites.google.com/site/jhbrsstest/greektv.jpg" />
</item>
<item>
    <title>Italian Cielo TV</title>
    <link>http://link1</link>
    <media:stream url="rtmp://cp86825.live.edgefcs.net/live/cielo_std@4017630&live=1&quiet=1" />
    <media:thumbnail url="https://sites.google.com/site/jhbrsstest/cielotv.jpg" />
</item>

</channel> </rss> }}}

and here is the LiveFeeds.groovy called by the rss. {{{ import org.serviio.library.metadata. import org.serviio.library.online.

/ * Generic LiveFeeds URL extractor plugin. * Invoked by anyurl/LiveFeeds *
* @author JHB50 * / class LiveFeeds extends FeedItemUrlExtractor {

final VALID_FEED_URL = '^http(s)*://.*?LiveFeeds.*?'
def saveUrl

String getExtractorName() {
    return 'LiveFeeds'
}

boolean extractorMatches(URL feedUrl) {
    //println "feedUrl = " + feedUrl
    saveUrl = "$feedUrl"        //save rss url in string format for getting the feed later
    //println "saveUrl = " + saveUrl
    return feedUrl ==~ VALID_FEED_URL
}

ContentURLContainer extractUrl(Map links, PreferredQuality requestedQuality) {
    def linkUrl = links.default  /* The <link> tag */
    def contentUrl    /* The video to be played to be returned */
    def thumbnailUrl  /* The thumbnail to be displayed in DLNA menu - to be returned */

    // display the <link> tag
    //println "linkUrl = " + linkUrl

    //goback and get the feed url
    def rssNode = new XmlParser().parse(saveUrl)


    // get a list of items
    List itemNode = rssNode.channel.item
    //println "itemNode = " + itemNode
    //println " "

    // look at each item to find the link url and keep the match

    def matchedItem

    // set matchedItem with item contents
    itemNode.each { it -> if(it.link.text() == linkUrl.toString()) matchedItem = it }

    //println " "
    //println "matchedItem = " + matchedItem
    //println " "

    //now extract the stream url
    def mediaLink = matchedItem =~ '(?s)stream\\[attributes=\\{url=([\\w:/\\?\\=\\.\\-]+)};'

    //println "mediaLink = " + mediaLink[0][1]
    //println " "

    contentUrl = mediaLink[0][1]

    //now extract the thumbnail url
    def thumbLink = matchedItem =~ '(?s)thumbnail\\[attributes=\\{url=([\\w:/\\?\\=\\.\\-]+)};'

    //println "thumbLink = " + thumbLink[0][1]
    //println " "

    thumbnailUrl = thumbLink[0][1]

    def live = true

    return new ContentURLContainer(contentUrl: contentUrl, thumbnailUrl: thumbnailUrl, live: live)
}

static void main(args) {
    // this is just to test
    LiveFeeds extractor = new LiveFeeds()

    assert extractor.extractorMatches( new URL("https://sites.google.com/site/jhbrsstest/LiveFeeds_TVStationsTest.rss"))

    Map videoLinks = ['default': new URL('http://link1')]
    ContentURLContainer result = extractor.extractUrl(videoLinks,PreferredQuality.MEDIUM)
    println "Result: $result"        
}

} }}}

Lastly here is the output you will get if you run the groovy standalone {{{ C:\Users\John>groovy "C:\Program Files (x86)\Serviio\plugins\LiveFeeds.groovy" [Fatal Error] LiveFeeds_TVStationsTest.rss?attachauth=ANoY7cpwcn8yjZTrLPDMZDWjzm yafflpnwU3Q0Zgbn1DzWDbUlnSkq-CAFuxML9VWtwAkbqYXRlTfpWUr-vwX87kYyG22rrbb1T29fXRaW fYxCltzsNR8WKyalJ1hOvc79OfXvJV_T3C7CbCJ-Omplh-NYaqznipq1jQgj0Zv8uLnC88rbOyVo3f0Q T_B21Y2pE0TXlvb7piAi5szLjuw3qh7Gtryly_tg%3D%3D&attredirects=0:19:81: The referen ce to entity "live" must end with the ';' delimiter. Caught: org.xml.sax.SAXParseException: The reference to entity "live" must end w ith the ';' delimiter. org.xml.sax.SAXParseException: The reference to entity "live" must end with the ';' delimiter. at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Ab stractSAXParser.java:1231) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.p arse(SAXParserImpl.java:522) at LiveFeeds.extractUrl(LiveFeeds.groovy:36) at LiveFeeds$extractUrl.call(Unknown Source) at LiveFeeds.main(LiveFeeds.groovy:86)

}}}

Comments (12)