tornado curl_httpclient proxy

最新推荐文章于 2026-06-16 15:30:17 发布

原创最新推荐文章于 2026-06-16 15:30:17 发布 · 3.9k 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

标签

#tornado #curl_httpclient #http_proxy

tornado 专栏收录该内容

1 篇文章

订阅专栏

本文深入分析了Tornado框架下两种HTTP客户端的使用场景，重点对比了简单客户端和curl_httpclient在代理访问网络时的表现差异。通过代码示例，展示了如何在Tornado应用中配置并使用curl_httpclient实现代理访问，解决超时等问题，为开发者提供了实用的调试指南。

tornado httpclient

默认simple_httpclient，满足大部分普通需求.

curl_httpclient更多特性(需要libcurl pycurl)：

支持http proxy,以及使用指定网络接口
不死磕http规范，兼容更多站点
效率更高

curl_httpclient 使用proxy实例

以下代码执行的机器通过代理访问网络

>>> import tornado
>>> tornado.version
'4.2.1'
>>> from tornado.httpclient import HTTPClient
>>> from tornado.httpserver import HTTPRequest
>>> c = HTTPClient()
>>> req = HTTPRequest(url="http://t.cp31.ott.cibntv.net/XXXX", follow_redirects=False, proxy_host="10.103.11.11", proxy_port=81)
>>> c.fetch(req)
Traceback (most recent call last):
  File "", line 1, in 
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.2.1-py2.7-linux-x86_64.egg/tornado/httpclient.py", line 102, in fetch
    self._async_client.fetch, request, **kwargs))
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.2.1-py2.7-linux-x86_64.egg/tornado/ioloop.py", line 445, in run_sync
    return future_cell[0].result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.2.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 215, in result
    raise_exc_info(self._exc_info)
  File "", line 3, in raise_exc_info
tornado.httpclient.HTTPError: HTTP 599: Timeout

#simple_httpclient 不支持代理,599超时

>>> from tornado.curl_httpclient import CurlAsyncHTTPClient
>>> from tornado.ioloop import IOLoop
>>> c = CurlAsyncHTTPClient()
>>> def print_response(r):
...     print r.error or r.body
... 
>>> c.fetch(req, print_response)
>>> IOLoop.instance().start()
HTTP 301: Moved Permanently
ERROR:tornado.application:Future exception was never retrieved: HTTPError: HTTP 301: Moved Permanently

# curl_httpclient使用代理成功联网，取得该URL的301信息

所以有童鞋说curl可以访问某个URL,证明网络通畅,就是tornado代码无法访问该URL，大概原因就是如此了.调试步骤