Proxies - HTTPX (original) (raw)

HTTPX supports setting up HTTP proxies via the proxy parameter to be passed on client initialization or top-level API functions like httpx.get(..., proxy=...).

Diagram of how a proxy works (source: Wikipedia). The left hand side "Internet" blob may be your HTTPX client requesting example.com through a proxy.

HTTP Proxies

To route all traffic (HTTP and HTTPS) to a proxy located at http://localhost:8030, pass the proxy URL to the client...

with httpx.Client(proxy="http://localhost:8030") as client: ...

For more advanced use cases, pass a mounts dict. For example, to route HTTP and HTTPS requests to 2 different proxies, respectively located at http://localhost:8030, and http://localhost:8031, pass a dict of proxy URLs:

`proxy_mounts = { "http://": httpx.HTTPTransport(proxy="http://localhost:8030"), "https://": httpx.HTTPTransport(proxy="http://localhost:8031"), }

with httpx.Client(mounts=proxy_mounts) as client: ... `

For detailed information about proxy routing, see the Routing section.

Gotcha

In most cases, the proxy URL for the https:// key should use the http:// scheme (that's not a typo!).

This is because HTTP proxying requires initiating a connection with the proxy server. While it's possible that your proxy supports doing it via HTTPS, most proxies only support doing it via HTTP.

For more information, see FORWARD vs TUNNEL.

Authentication

Proxy credentials can be passed as the userinfo section of the proxy URL. For example:

with httpx.Client(proxy="http://username:password@localhost:8030") as client: ...

Proxy mechanisms

Note

This section describes advanced proxy concepts and functionality.

FORWARD vs TUNNEL

In general, the flow for making an HTTP request through a proxy is as follows:

1. The client connects to the proxy (initial connection request).
2. The proxy transfers data to the server on your behalf.

How exactly step 2/ is performed depends on which of two proxying mechanisms is used:

• Forwarding: the proxy makes the request for you, and sends back the response it obtained from the server.
• Tunnelling: the proxy establishes a TCP connection to the server on your behalf, and the client reuses this connection to send the request and receive the response. This is known as an HTTP Tunnel. This mechanism is how you can access websites that use HTTPS from an HTTP proxy (the client "upgrades" the connection to HTTPS by performing the TLS handshake with the server over the TCP connection provided by the proxy).

Troubleshooting proxies

If you encounter issues when setting up proxies, please refer to our Troubleshooting guide.

SOCKS

In addition to HTTP proxies, httpcore also supports proxies using the SOCKS protocol. This is an optional feature that requires an additional third-party library be installed before use.

You can install SOCKS support using pip:

$ pip install httpx[socks]

You can now configure a client to make requests via a proxy using the SOCKS protocol:

httpx.Client(proxy='socks5://user:pass@host:port')