Scrapeless Wiki

What Is httpx? – Definition and How It Works

Informational P2 what is httpx

Learn what httpx offers Python developers: one API for sync and async, HTTP/2 support, strict timeouts, and a familiar interface.

httpx is a Python HTTP client offering synchronous and asynchronous interfaces from one library, with an API deliberately close to the long-established synchronous standard.

1. One API, Two Modes

import httpx

# synchronous
with httpx.Client(timeout=30.0) as client:
    response = client.get("https://api.example.com/items")
    response.raise_for_status()
    data = response.json()

# asynchronous
async with httpx.AsyncClient(timeout=30.0) as client:
    response = await client.get("https://api.example.com/items")

The methods and response objects match, so moving a codebase from synchronous to concurrent is mostly mechanical.

2. HTTP/2 Support

Enabling HTTP/2 lets many requests to one host share a single multiplexed connection:

client = httpx.AsyncClient(http2=True)

Beyond performance, it changes how the client appears to a destination β€” HTTP/2 framing carries implementation characteristics that HTTP/1.1 does not expose.

3. Timeouts Are Explicit by Default

httpx applies a default timeout rather than waiting forever, and allows connect, read, write, and pool timeouts to be set separately.

That default is a meaningful difference: the most common reliability bug in Python HTTP code is a call with no timeout, and this library does not let it happen silently.

4. Clients Own Pooling and State

As with any modern client, a Client holds the connection pool, cookies, and default headers. Creating one per request forfeits reuse and, for multi-step flows, loses the session state that makes them work.

Limits on total and per-host connections are set on the client, and per-host is the politeness control.

5. Useful Features

  • Explicit proxy configuration, including per-scheme routing.
  • Streaming responses, for downloads that should not be held in memory.
  • A transport layer that can be replaced, which makes testing without network access straightforward.
  • Multipart uploads and JSON bodies, handled with the same conventions as the familiar synchronous library.

6. Compared With the Alternatives

Library Sync Async HTTP/2 Notes
requests Yes No No Ubiquitous, mature, synchronous only
aiohttp No Yes No Mature async, also a server framework
httpx Yes Yes Yes One API for both, explicit timeouts

7. What It Does Not Do

It fetches; it does not render. Pages assembled by JavaScript return their served shell, and reaching that content needs a browser or the underlying endpoint.

Its TLS and header profile is also that of a Python client, which is observable to any destination that classifies clients.

8. Summary

httpx provides synchronous and asynchronous HTTP through one familiar API, with HTTP/2 support, default timeouts, and a replaceable transport that simplifies testing.

Reuse a client, set per-host connection limits, and remember it is a fetcher rather than a renderer β€” client-rendered content still requires a browser or a direct call to the endpoint behind it.