Scrapeless Wiki

requests vs. httpx – Definition and Differences

Comparison P2 requests vs httpx

Compare Python's two HTTP clients on async support, HTTP/2, timeout defaults, testing, and when switching is actually worthwhile.

Both are Python HTTP clients with deliberately similar interfaces. httpx was designed as a modern successor, adding asynchronous support and HTTP/2 while keeping the familiar API shape.

1. Key Differences

Dimension requests httpx
Synchronous Yes Yes
Asynchronous No Yes
HTTP/2 No Yes, when enabled
Default timeout None β€” waits forever A default is applied
Connection pooling Via a session Via a client
Testing support Third-party libraries Replaceable transport built in
Ecosystem Enormous Growing
Maturity Very high High

2. The Timeout Default Is the Most Consequential

requests waits indefinitely unless a timeout is passed. That single default is behind a large share of hung Python processes, because the omission is silent and only bites under specific conditions.

httpx applies a default and allows connect, read, write, and pool timeouts to be configured separately. Explicit timeouts remain good practice in both, and one fails safe while the other does not.

3. Async Support Changes What Is Possible

Fetching a thousand URLs with requests means threads or processes. With httpx's async client and a semaphore it is straightforward concurrency on one thread.

For I/O-bound collection that difference is the main reason to switch, and it is a large one.

4. HTTP/2

Enabling it lets many requests to one host share a multiplexed connection. Beyond throughput, it changes how the client appears at the protocol layer β€” relevant wherever destinations classify clients by protocol behaviour.

5. API Familiarity

# requests
r = requests.get(url, params={"q": "x"}, timeout=30)

# httpx
r = httpx.get(url, params={"q": "x"}, timeout=30)

The similarity is deliberate. Most synchronous code moves with an import change, though differences exist in redirect defaults and some parameter handling, so a test pass is warranted rather than assumed.

6. When to Stay With requests

  • The codebase is large, synchronous, and working.
  • Dependencies must be minimal and battle-tested.
  • Third-party libraries in use expect it.
  • The work is a handful of sequential calls.

There is no correctness argument for switching a working synchronous codebase.

7. When to Choose httpx

  • New projects, particularly asynchronous ones.
  • Concurrency is needed without threads.
  • HTTP/2 is wanted.
  • Testing benefits from a replaceable transport.
  • Both sync and async paths must share one client abstraction.

8. Summary

requests is the ubiquitous synchronous client; httpx offers the same shape plus async, HTTP/2, sane timeout defaults, and a transport layer that simplifies testing.

Choose httpx for new work and anything concurrent; keep requests where a synchronous codebase already works β€” and set explicit timeouts in either case.