Using an HTTP / HTTPS proxy

Note

  • Support for HTTP proxies is available in Libcloud v0.16.0 and higher.
  • Support for HTTPS proxies is available in Libcloud v2.5.1-dev and higher.
  • In versions prior to v2.5.1-dev, driver.connection.set_http_proxy()method is broken and you need to usedriver.connection.connection.set_http_proxy() instead.

Libcloud supports using an HTTP / HTTPS proxy for outgoing HTTP and HTTPSrequests.

Proxy support has been tested with the following Python versions:

  • Python 2.7 / PyPy
  • Python 3.4
  • Python 3.6
  • Python 3.7

You can specify which HTTP(s) proxy to use using one of the approaches describedbelow:

  • By setting http_proxy / https_proxy environment variable (thissetting is system / process wide)
  • By passing http_proxy argument to thelibcloud.common.base.LibcloudConnection class constructor (thissetting is local to the connection instance)
  • By calling libcloud.common.base.LibcloudConnection.set_http_proxy()method aka driver.connection.connection.set_http_proxy (this settingis local to the connection instance)

Known limitations

  • Only HTTP basic access authentication proxy authorization method is supported
  • If you are using HTTPS proxy you need to configure Libcloud to use CA certbundle path which is used by the proxy server. See an example below on how todo that.

Examples

This section includes some code examples which show how to use an HTTP(s) proxywith Libcloud.

1. Using http_proxy / htps_proxy environment variable

By setting http_proxy / https_proxy environment variable you canspecify which proxy to use for all of the outgoing requests for a duration /life-time of the process or a script.

Without authentication (http proxy):

  1. http_proxy=http://<proxy hostname>:<proxy port> python my_script.py

Without authentication (https proxy):

  1. http_proxy=https://<proxy hostname>:<proxy port> python my_script.py
  2. # or
  3. https_proxy=https://<proxy hostname>:<proxy port> python my_script.py

With basic auth authentication (http proxy):

  1. http_proxy=http://<username>:<password>@<proxy hostname>:<proxy port> python my_script.py

2. Passing proxy_url argument to the connection class constructor

. note:

  1. Some drivers don't correctly pass ``proxy_url`` argument to the connection
  2. class and don't support ``proxy_url`` constructor argument.
  3.  
  4. If you pass this argument to the driver constructor, but it doesn't appear
  5. to be working, it's likely the driver doesn't support this method.
  6.  
  7. In such scenarios, you are advised to use some other method of setting a
  8. proxy (e.g. by setting an environment variable or by using
  9. :meth:`libcloud.common.base.LibcloudConnection.set_http_proxy` method).

By passing proxy_url argument to thelibcloud.common.base.Connection class constructor, you can specifywhich proxy to use for a particular connection.

  1. from libcloud.compute.types import Provider
  2. from libcloud.compute.providers import get_driver
  3.  
  4. HTTP_PROXY_URL_NO_AUTH_1 = 'http://<proxy hostname 1>:<proxy port 2>'
  5. HTTPS_PROXY_URL_NO_AUTH_1 = 'https://<proxy hostname 1>:<proxy port 2>'
  6.  
  7. cls = get_driver(Provider.RACKSPACE)
  8.  
  9. # 1. Use http proxy
  10. driver = cls('username', 'api key', region='ord',
  11.              proxy_url=HTTP_PROXY_URL_NO_AUTH_1)
  12.  
  13. # 2. Use https proxy
  14. driver = cls('username', 'api key', region='ord',
  15.              proxy_url=HTTPS_PROXY_URL_NO_AUTH_1)

3. Calling set_http_proxy method

Calling set_http_proxy method allows you to specify which proxy to usefor all the outgoing requests which follow set_http_proxy method call.

This method also allows you to use a different proxy for each request as shownin the example below.

  1. from pprint import pprint
  2.  
  3. from libcloud.compute.types import Provider
  4. from libcloud.compute.providers import get_driver
  5.  
  6. HTTP_PROXY_URL_NO_AUTH_1 = 'http://<proxy hostname 1>:<proxy port 2>'
  7. HTTP_PROXY_URL_NO_AUTH_2 = 'http://<proxy hostname 1>:<proxy port 2>'
  8. HTTP_PROXY_URL_BASIC_AUTH = 'http://<user>:<pass>@<proxy hostname>:<port>'
  9.  
  10. cls = get_driver(Provider.RACKSPACE)
  11. driver = cls('username', 'api key', region='ord')
  12.  
  13. # Use proxy 1 for this request
  14. driver.connection.connection.set_http_proxy(proxy_url=HTTP_PROXY_URL_NO_AUTH_1)
  15. pprint(driver.list_nodes())
  16.  
  17. # Use proxy 2 for this request
  18. driver.connection.connection.set_http_proxy(proxy_url=HTTP_PROXY_URL_NO_AUTH_2)
  19. pprint(driver.list_nodes())

4. Using an HTTPS proxy

This example shows how to use an HTTPS proxy.

  1. import os.path
  2. from pprint import pprint
  3.  
  4. from libcloud.compute.types import Provider
  5. from libcloud.compute.providers import get_driver
  6.  
  7. import libcloud.security
  8.  
  9. HTTPS_PROXY_URL_NO_AUTH = 'https://<proxy hostname 1>:<proxy port 2>'
  10.  
  11. # 1. Use a custom CA bundle which is used by proxy server
  12. # This example uses CA cert bundle used by mitmproxy proxy server
  13. libcloud.security.CA_CERTS_PATH = os.path.expanduser(
  14.     '~/.mitmproxy/mitmproxy-ca-cert.pem')
  15.  
  16. # User an https proxy for subsequent requests
  17. cls = get_driver(Provider.RACKSPACE)
  18. driver = cls('username', 'api key')
  19.  
  20. driver.connection.connection.set_http_proxy(proxy_url=HTTPS_PROXY_URL_NO_AUTH)
  21. pprint(driver.list_nodes())

To use an HTTPS proxy, you also need to configure Libcloud to use CA cert bundlewhich is used by the HTTPS proxy server, to verify outgoing https request. If youdon’t do that, you will see errors similar to the one below:

  1. SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

Keep in mind that you will also receive a similar error message if you try touse HTTP proxy for HTTPS requests.