1. Gregory Petukhov
  2. grab
Issue #48 wontfix

Ошибка spider.setup_proxylist()

Alex Istinspring
created an issue

{{{

!python

    #proxy_list = map(lambda x: x.encode('utf-8'),
    #                 process_proxy_list(obj.value))
    proxy_list = process_proxy_list(obj.value)

    if proxy_list is not None:
        #spider.setup_proxylist(proxy_file='/var/proxylist.txt',
        #                       proxy_type='http', auto_change=True)
        spider.setup_proxylist(server_list=list(proxy_list),
                               proxy_type='http', auto_change=True)

}}}

вылетает такое

{{{

!python

line 15, in run_spider spider.run() File "/usr/local/lib/python2.6/dist-packages/grab/spider/base.py", line 209, in run for res_count, res in enumerate(self.get_next_response()): File "/usr/local/lib/python2.6/dist-packages/grab/spider/base.py", line 466, in get_next_response transport.process_task(task, grab, grab_config_backup) File "/usr/local/lib/python2.6/dist-packages/grab/spider/transport/multicurl.py", line 33, in process_task curl.grab.prepare_request() File "/usr/local/lib/python2.6/dist-packages/grab/base.py", line 328, in prepare_request self.transport.process_config(self) File "/usr/local/lib/python2.6/dist-packages/grab/transport/curl.py", line 277, in process_config self.curl.setopt(pycurl.PROXYUSERPWD, grab.config['proxy_userpwd']) TypeError: invalid arguments to setopt

}}}

там есть закомментированные строчки, вообщем если список перегнать в utf-8 с юникода, то все ок.