Provide User Agent option for Thumbnail retrieval

[Former user]

See http://forum.serviio.org/viewtopic.php?f=22&t=10271

[Former user]

Seems ilive have changed their thumblinks so that's probably why I could not get them, and this request for UserAgent on thumbs is no longer required.

[Former user]

Looks like hd-trailers.net has done the same thing. New thumbnails are failing while older cached ones still display. Here is example. [code]

IMAGE AS RETREIVED USING BROWSER

GET /images/disney-s-planes-102420-poster-xlarge-resized.jpg HTTP/1.1

Accept: image/png, image/svg+xml, image/*;q=0.8, */*;q=0.5

Referer: http://www.hd-trailers.net/page/1/

Accept-Language: en-US

User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)

Accept-Encoding: gzip, deflate

Host: static.hd-trailers.net

If-Modified-Since: Thu, 04 Apr 2013 17:15:08 GMT

DNT: 1

Connection: Keep-Alive

Cookie: __cfduid=d0da170c9aa1ad471a7e2da32a20f92fb1368975773; __utma=31674087.181173897.1368975779.1368994399.1369801219.4; __utmz=31674087.1369801219.4.2.utmccn=(referral)|utmcsr=dogpile.com|utmcct=/search/web|utmcmd=referral; __qca=P0-449763262-1368975781402; __utmc=31674087


IMAGE AS RETREIVED BY SERVIIO

GET /images/disney-s-planes-102420-poster-xlarge-resized.jpg HTTP/1.1

User-Agent: Java/1.7.0_04

Host: static.hd-trailers.net

Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

Connection: keep-alive


ERROR WHEN RETREIVED BY SERVIIO

2013-05-29 08:27:19,539 DEBUG [OnlineLibraryManager] Thumbnail http://static.hd-trailers.net/images/disney-s-planes-102420-poster-xlarge-resized.jpg not in cache yet, loading it
2013-05-29 08:27:19,652 WARN  [FeedUpdaterThread] An error occured while retrieving thumbnail, will remove it from the item and will continue
org.serviio.library.online.CannotRetrieveThumbnailException: Failed to download thumbnail http://static.hd-trailers.net/images/disney-s-planes-102420-poster-xlarge-resized.jpg.
    at org.serviio.library.online.OnlineLibraryManager.findThumbnail(OnlineLibraryManager.java:266)
    at org.serviio.library.online.metadata.FeedUpdaterThread.run(FeedUpdaterThread.java:107)
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://static.hd-trailers.net/images/disney-s-planes-102420-poster-xlarge-resized.jpg
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at org.serviio.util.HttpClient.retrieveBinaryFileFromURL(HttpClient.java:72)
    at org.serviio.library.online.OnlineLibraryManager.findThumbnail(OnlineLibraryManager.java:258)
    ... 1 more

[/code]

[Former user]

Here is proof that Browser UserAgent is required for thumbs

FOR srcUrl = "http://static.hd-trailers.net/images/disney-s-planes-102420-poster-xlarge-resized.jpg" String html

---

USING html = new URL(srcUrl).getText()

GET /images/disney-s-planes-102420-poster-xlarge-resized.jpg HTTP/1.1

User-Agent: Java/1.6.0_10-beta

Host: static.hd-trailers.net

Accept: text/html, image/gif, image/jpeg, ; q=.2, /*; q=.2

Connection: keep-alive

HTTP/1.1 403 Forbidden

Server: cloudflare-nginx

Date: Wed, 29 May 2013 14:36:56 GMT

Content-Type: text/html

Transfer-Encoding: chunked

Connection: keep-alive

Cache-Control: max-age=10

Expires: Wed, 29 May 2013 14:37:06 GMT

CF-RAY: 763eb76fd7e0107

Set-Cookie: __cfduid=db90ca899f6da80b51ab8a6a1ee8c944a1369838216; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.hd-trailers.net

---

USING html = openURL(new URL(srcUrl), "Java/1.7.0_04")

GET /images/disney-s-planes-102420-poster-xlarge-resized.jpg HTTP/1.1

Date: Wed, 29 May 2013 14:39:07 GMT

Accept: /

Host: static.hd-trailers.net

User-Agent: Java/1.7.0_04

Content-Length: 0

HTTP/1.1 403 Forbidden

Server: cloudflare-nginx

Date: Wed, 29 May 2013 14:39:09 GMT

Content-Type: text/html

Transfer-Encoding: chunked

Connection: keep-alive

Cache-Control: max-age=10

Expires: Wed, 29 May 2013 14:39:19 GMT

CF-RAY: 763eea881940107

Set-Cookie: __cfduid=d61152bfa98adfe9088422ce99af382661369838347; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.hd-trailers.net

---

USING html = openURL(new URL(srcUrl), "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1")

GET /images/disney-s-planes-102420-poster-xlarge-resized.jpg HTTP/1.1

Date: Wed, 29 May 2013 14:42:47 GMT

Accept: /

Host: static.hd-trailers.net

User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1

Content-Length: 0

HTTP/1.1 200 OK

Server: cloudflare-nginx

Date: Wed, 29 May 2013 14:42:47 GMT

Content-Type: image/jpeg

Content-Length: 24407

Connection: keep-alive

Last-Modified: Thu, 04 Apr 2013 17:15:08 GMT

Cache-Control: public, max-age=2592000

Expires: Fri, 28 Jun 2013 14:42:47 GMT

CF-Cache-Status: HIT

Accept-Ranges: bytes

Set-Cookie: __cfduid=d6ee7e04c830e5639575c2db7e320e49c1369838567; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.hd-trailers.net

CF-RAY: 763f40464850107

[Petr Nejedly]

I will make it so that the UserAgent set in the plugin will be used for thumbnails too, ok?

[Former user]

" UserAgent set in the plugin "

UserAgent can only be set when reading Urls, so I do not know which read command you will use out of the many forms I use, some of which do not specify a ua and so just default to java.

Are you suggesting use of a dummy OpenURL to specify it? What if I am accessing various sites with different ua's.

[Former user]

Bump..I do not understand where you are getting the useragent.

[Petr Nejedly]

from ContentURLContainer coming from the plugin

[Former user]

You said "I will make it so that the UserAgent set in the plugin will be used for thumbnails too, ok?"

So I'll ask the same question again. My ContentURLContainer issues 3 URL reads. One sets UA = Java, one sets UA=Mozilla, and one sets no UA. Which UA will be used for the thumbnail?

[Petr Nejedly]

There is userAgent property on ContentURLContainer for each of your streams. The value of this property is then used to retrieve the media file itself (usually via FFmpeg), and now the same value will be used for retrieving the item's thumbnail.

[Former user]

Thanks, I did not know that. I thought you could only specify it on an openUrl.