关于爬虫使用私密代理报错
urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme none
这两天用requests写爬虫，总是发现莫名其妙的报错，心里很不爽，然后先给一下源代码
import requests
import os
url = "http://www.baidu.com/"
headers = {
"User-Agent": "Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.9.168 Version/11.50"
}
# requests = requests.Request(url, data=None, headers=headers)
# 使用私密代理
proxyUser = os.environ.get("PROXYUSER")
proxyPasswd = os.environ.get("PROXYPASSWD")
print(proxyUser, "---------------", proxyPasswd)
proxyIp = "43.226.37.90"
proxyPort = "16817"
#需要添加完整的网址
proxy = {"http":"%s:%s@%s:%s"%(proxyUser, proxyPasswd,proxyIp, proxyPort)}
#账号:密码@ip:port
# handler = urllib.request.ProxyHandler({"http": "%s:%s@%s:%s"%(proxyUser, proxyPasswd, proxyIp, proxyPort)})
# opener = urllib.request.build_opener(handler)
response = requests.get(url,headers=headers,proxies=proxy)
print(response.text)
报错是这样的
Traceback (most recent call last):
File "G:/pycharm练习/.idea/threeday/6.使用私密代理.py", line 51, in <module>
response = requests.get(url,headers=headers,proxies=proxy)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\adapters.py", line 412, in send
conn = self.get_connection(request.url, proxies)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\adapters.py", line 309, in get_connection
proxy_manager = self.proxy_manager_for(proxy)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\adapters.py", line 199, in proxy_manager_for
**proxy_kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\urllib3\poolmanager.py", line 450, in proxy_from_url
return ProxyManager(proxy_url=url, **kw)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-packages\urllib3\poolmanager.py", line 401, in __init__
raise ProxySchemeUnknown(proxy.scheme)
urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme none
Process finished with exit code 1
经过高人指点，终于发现了问题，只需要添加完整网址即可，修改如下：
#修改前
proxy = {"http":"%s:%s@%s:%s"%(proxyUser, proxyPasswd,proxyIp, proxyPort)}
#修改后
proxy = {"http":"http://%s:%s@%s:%s"%(proxyUser, proxyPasswd,proxyIp, proxyPort)}
结果如下：
None --------------- None
client ip: 111.198.24.66
嘻嘻~边报错边成长，加油 | 
转播
