Expand OpenUrl method

Issue #713 new
Former user created an issue

Please add redirect logic (detect and retry) to your existing OpenUrl method?

Can you also add a "referer" option, in addition to the existing "user-agent" and "cookie" options, or perhaps even better a "property" option where one can provide an option map of valid header pairs ( eg: "referer":"http://www.coolspot.com"," Accept-Language":"en-US","Range": "bytes=500-999" ) per http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html?

Comments (2)

  1. Petr Nejedly repo owner

    do you still need this? if yes, can you expand a bit on how the redirect logic should work?

  2. Former user Account Deleted

    I think it a useful addition for users since most of my plugin gets require the referrer.

    I actually now embed a set of my methods in each plugin using @Mixin, one of which is String OpenProtURL(String srcUrl,String refUrl, String Cookie) which uses the standard windows ua. and is used by String getSrcText(String SrcUrl, String RefUrl, String seq, String open) which uses the "open" value to do the get with your OpenUrl, or my OpenProtUrl or with GZIPInputStream.

    getSrcText also retrys the text gets with GZIP if the returned string does not contain text, and if the returned string contains regex '(?s)(?i)^<meta http-equiv="refresh" content="0;\surl=.?' will retry with that redirected url.

    I can then use String getSrcUrl(String srcText, String regex, String preUrl) to get the next url in the Webpage chain, and use preUrl as the referrer as required.

  3. Log in to comment