403 Client Error: Forbidden for url

[Tim]

No downloads appear to be working for ABC iview now.

INFO Downloading: Sammy J Series 4 Ep 25 The Ballad of Locky D.ts
[cli][info] Found matching plugin hls for URL hlsvariant://https://iview-vod-hls.akamaized.net/prod/le2/LE2006V025S00MA1D1/20210804192002/hls/720.m3u8?hdnea=st%3D1628311993~exp%3D1628
312893~acl%3D%2F%2A~hmac%3D3fa601f83f6945ff51a439e3613a845683c7d655c9a8aa026545bb5105a717fd
[cli][info] Available streams: 180p (worst), 288p, 360p, 450p, 576p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)
[cli][error] Try 1/1: Could not open stream <HLSStream('https://iview-vod-hls.akamaized.net/prod/le2/LE2006V025S00MA1D1/20210804192002/hls/3000000/hdntl=exp=1628398454~acl=%2F*~data=h
dntl~hmac=8015927559286d9d039a8504bd8b75cc2d7d57788928f94b8d3794d32005c733/stream.m3u8?aka_me_session_id=AAAAAAAAAAB2Yw9hAAAAABAIIZXEF2KDWYRBzWQX1vnfp700cE8DcfWKd5RKkUDvAgkpy%2F6iiruO
DeCuMGkIBb5VTk%2FyQBWI&aka_media_format_type=hls')> (Could not open stream: Unable to open URL: https://iview-vod-hls.akamaized.net/prod/le2/LE2006V025S00MA1D1/20210804192002/hls/3000
000/hdntl=exp=1628398454~acl=%2F*~data=hdntl~hmac=8015927559286d9d039a8504bd8b75cc2d7d57788928f94b8d3794d32005c733/stream.m3u8?aka_me_session_id=AAAAAAAAAAB2Yw9hAAAAABAIIZXEF2KDWYRBzW
QX1vnfp700cE8DcfWKd5RKkUDvAgkpy%2F6iiruODeCuMGkIBb5VTk%2FyQBWI&aka_media_format_type=hls (403 Client Error: Forbidden for url: https://iview-vod-hls.akamaized.net/prod/le2/LE2006V025S
00MA1D1/20210804192002/hls/3000000/hdntl=exp=1628398454~acl=%2F*~data=hdntl~hmac=8015927559286d9d039a8504bd8b75cc2d7d57788928f94b8d3794d32005c733/stream.m3u8?aka_me_session_id=AAAAAAA
AAAB2Yw9hAAAAABAIIZXEF2KDWYRBzWQX1vnfp700cE8DcfWKd5RKkUDvAgkpy%2F6iiruODeCuMGkIBb5VTk%2FyQBWI&aka_media_format_type=hls))
error: Could not open stream <HLSStream('https://iview-vod-hls.akamaized.net/prod/le2/LE2006V025S00MA1D1/20210804192002/hls/3000000/hdntl=exp=1628398454~acl=%2F*~data=hdntl~hmac=80159
27559286d9d039a8504bd8b75cc2d7d57788928f94b8d3794d32005c733/stream.m3u8?aka_me_session_id=AAAAAAAAAAB2Yw9hAAAAABAIIZXEF2KDWYRBzWQX1vnfp700cE8DcfWKd5RKkUDvAgkpy%2F6iiruODeCuMGkIBb5VTk%
2FyQBWI&aka_media_format_type=hls')>, tried 1 times, exiting
ERROR streamlink exited with error code: 1
Press return to continue...

‌

I tried changing the user-agent to be the same as my browser, no change. I can still watch ABC iview in the browser without signing in, so I don’t think it’s related to the proposed changes forcing users to have an account.

[Tim Cinel]

I’m getting this, too. Only affecting some episodes it seems - even within a show some episodes work and some result in the above error.

[Tim Cinel]

I have a workaround which involves swapping out streamlink for youtube-dl , works.

https://bitbucket.org/TimCinel/webdl/src/issue-118-youtube-dl-for-iview/

Still behaves with autograbber.py , my workflow didn’t require any changes. Do you want a pull request for this change?

[delx]

I can’t reproduce this sorry.

What ISP are you using? What version of streamlink are you using?

[Tim Cinel]

ISP: Superloop
streamlink version: 2.3.0

Repro steps:

$ git checkout origin master
... 33f4f8d

$ docker run --rm -ti -v "$(pwd):$(pwd)" -w "$(pwd)" python:3 /bin/bash
...
$ root@e7ca6301c2de:/Users/tim/src/webdl# pip install -r requirements.txt
...
Collecting streamlink
  Downloading streamlink-2.3.0-py3-none-any.whl (361 kB)
...
$ root@e7ca6301c2de:/Users/tim/src/webdl# ./grabber.py
  1) ABC iView
Choose> 1
...
$ Choose> 143
INFO Downloading: Sammy J Series 4 Ep 25 The Ballad of Locky D.ts
...
Exception in thread Thread-HLSStreamWorker:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/streamlink/plugin/api/http_session.py", line 160, in request
    res.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://iview-vod-hls.akamaized.net/prod/le2/LE2006V025S00MA1D1/20210804192002/hls/3000000/hdntl=exp=1630385624%E2%80%BEacl=%2F*%E2%80%BEdata=hdntl%E2%80%BEhmac=26eaa004811491bb23c8506e439999eb03a6549b519f6c2b184ab109a12689a1/stream.m3u8?aka_me_session_id=AAAAAAAAAADYtS1hAAAAAOo9sTOarEzrNZGSQFXkAEgBlwY0E8hOSxSCFH0VM0ria9iL1dO2T2VrvcMzT4062XKl1icZR6GY&aka_media_format_type=hls

‌

[delx]

This works for me with the same Streamlink version.

Is there any chance you can try with a different ISP? Perhaps your mobile phone provider?

[Tim Cinel]

Tried with Vodaphone AU (via Kogan Mobile). Still getting the 403.

Wonder if it’s to do with some other dependency (requests) or the runtime itself (Python).

OOC, which Python version are you using?

[Tim]

I’ve done a bit more debugging and the results are interesting…

I’ve always used the virtualenv, and it turns out a library in there is breaking things. Under Debian 10 and Ubuntu 20.04, if I setup a virtualenv then I end up with forbidden URLs. If I don’t use the virtualenv (and use system python libraries), it works.

Output when using virtualenv

[cli][info] Found matching plugin hls for URL hlsvariant://https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/720.m3u8?hdnea=st%3D1637585449~exp%3D1637586349~acl%3D%2F%2A~hmac%3D3664cd786a6be3e80fa8b491c998fa17d4cf4d8b1e9186c54ecda52468edddf5
[cli][info] Available streams: 180p (worst), 288p, 360p, 450p, 576p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)
[stream.hls][error] Unable to open URL: https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/4500000/hdntl=exp=1637671910%E2%80%BEacl=%2f*%E2%80%BEdata=hdntl%E2%80%BEhmac=de61be3283e3bcbbb7ac31e1553e493d015163a8f7c3a1238bb261e830040170/stream.m3u8?aka_me_session_id=AAAAAAAAAADm45xhAAAAAAEYWa6rXxMPxAuiRqEEIVgqWfTOC3lfYYWBSUUMOMbMf60cJSIHZB3NZvc80XdZBAjyD78FKZuR&aka_media_format_type=hls (403 Client Error: Forbidden for url: https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/4500000/hdntl=exp=1637671910%E2%80%BEacl=%2f*%E2%80%BEdata=hdntl%E2%80%BEhmac=de61be3283e3bcbbb7ac31e1553e493d015163a8f7c3a1238bb261e830040170/stream.m3u8?aka_me_session_id=AAAAAAAAAADm45xhAAAAAAEYWa6rXxMPxAuiRqEEIVgqWfTOC3lfYYWBSUUMOMbMf60cJSIHZB3NZvc80XdZBAjyD78FKZuR&aka_media_format_type=hls)
[cli][error] Try 1/1: Could not open stream <HLSStream('https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/4500000/hdntl=exp=1637671910%E2%80%BEacl=%2f*%E2%80%BEdata=hdntl%E2%80%BEhmac=de61be3283e3bcbbb7ac31e1553e493d015163a8f7c3a1238bb261e830040170/stream.m3u8?aka_me_session_id=AAAAAAAAAADm45xhAAAAAAEYWa6rXxMPxAuiRqEEIVgqWfTOC3lfYYWBSUUMOMbMf60cJSIHZB3NZvc80XdZBAjyD78FKZuR&aka_media_format_type=hls', 'https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/720.m3u8?hdnea=st%3D1637585449~exp%3D1637586349~acl%3D%2F%2A~hmac%3D3664cd786a6be3e80fa8b491c998fa17d4cf4d8b1e9186c54ecda52468edddf5')> (No data returned from stream)
error: Could not open stream <HLSStream('https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/4500000/hdntl=exp=1637671910%E2%80%BEacl=%2f*%E2%80%BEdata=hdntl%E2%80%BEhmac=de61be3283e3bcbbb7ac31e1553e493d015163a8f7c3a1238bb261e830040170/stream.m3u8?aka_me_session_id=AAAAAAAAAADm45xhAAAAAAEYWa6rXxMPxAuiRqEEIVgqWfTOC3lfYYWBSUUMOMbMf60cJSIHZB3NZvc80XdZBAjyD78FKZuR&aka_media_format_type=hls', 'https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/720.m3u8?hdnea=st%3D1637585449~exp%3D1637586349~acl%3D%2F%2A~hmac%3D3664cd786a6be3e80fa8b491c998fa17d4cf4d8b1e9186c54ecda52468edddf5')>, tried 1 times, exiting
[cli][info] Closing currently open stream...
ERROR streamlink exited with error code: 1

Output when not using virtualenv

[cli][info] Found matching plugin hls for URL hlsvariant://https://iview-vod-hls.akamaized.net/prod/ch2/CH2003Q001S00MA1D1/20211121092403/hls/720.m3u8?hdnea=st%3D1637586130~exp%3D1637587030~acl%3D%2F%2A~hmac%3Db97f05cc6e647622ffb2e261bccb34e0b77dde5bba7a173c0086f26432ebf7f8
[cli][info] Available streams: 180p (worst), 288p, 360p, 450p, 576p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)
[download][Bluey Bedroom.ts] Written 47.8 MB (13s @ 3.7 MB/s)                                                                                                                                                      
[cli][info] Stream ended
[cli][info] Closing currently open stream...

Streamlink is 1.3.1 on Ubuntu 20.04, and I tried both 1.0.0 and 2.0.0 from Debian Buster and Debian Buster Backports.

The virtualenv appears to be using streamlink 3.0.1

Python is 3.7.3 on Debian 10, and 3.8.10 on Ubuntu 20.04

‌

Downgrading streamlink to 2.0.0 in the virtualenv didn’t fix it, until I also forced the requests library down to 2.22.0. So something with newer streamlink and requests is breaking things.

The following requirements file git it working in virtualenv for me. Hopefully this gives you a starting point for working out where the real issue is.

streamlink==2.0.0
lxml
requests==2.22.0
requests-cache

‌

[Rado Faletič]

I had problems with your youtube-dl fix. Both your fix, and running youtube-dl stand-alone throws up:

ERROR: Unable to download JSON metadata: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)> (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

But adding the --no-check-certificate flag makes it work perfectly.

[Tim Cinel]

@Rado Faletič That problem is due to CA certificates on your system either being missing or out of date. Either that, or your connection to YouTube is being MITM’d .

‌

As you point out, --no-check-certificate is a workaround, but if you want to fix the root cause (which may affect other things) on that system, look through search results for “CERTIFICATE_VERIFY_FAILED python”

[Tim Cinel]

For completeness, pip freeze output after running above repro steps:

appdirs==1.4.4
attrs==21.2.0
cattrs==1.8.0
certifi==2021.10.8
charset-normalizer==2.0.7
idna==3.3
isodate==0.6.0
lxml==4.6.4
pycountry==20.7.3
pycryptodome==3.11.0
PySocks==1.7.1
requests==2.26.0
requests-cache==0.8.1
six==1.16.0
streamlink==3.0.1
url-normalize==1.4.3
urllib3==1.26.7
websocket-client==1.2.1

I’m not convinced it’s a result IP-based rate limiting. Can repro consistently from several source IPs.

[f ute]

Thanks Tim, I had this problem with a new install of ubuntu 20.04. Tried lots of things with no success. I changed my requirements to your posts and it works.

streamlink==2.0.0
lxml
requests==2.22.0
requests-cache

[Trade Surplus]

Thanks Tim Whiteit. This error started for me after I upgraded to Kubuntu 21.10. The changes you suggested for the requirements.txt file fixed it for me.