What Is the Python requests Library? β Definition and Usage
Learn what requests does, how sessions, parameters, and error handling work, and when a synchronous HTTP client is the right choice.
requests is Python's most widely used HTTP client. It wraps the standard library's lower-level machinery in an interface where the common cases are one line and the details stay accessible.
1. What Is requests?
A synchronous HTTP client for Python.
- Key idea: make ordinary HTTP work readable β a call, a response object, a decoded body.
- Mechanism: a friendly layer over connection pooling, encoding, and redirects.
- Goal: remove boilerplate from API calls and page fetches.
Example
import requests
response = requests.get(
"https://api.example.com/v1/items",
params={"q": "scraping", "limit": 10},
headers={"Authorization": "Bearer " + token},
timeout=30,
)
response.raise_for_status()
data = response.json()
2. The Response Object
| Attribute | Gives |
|---|---|
status_code |
The numeric status |
text |
Body decoded to a string |
content |
Raw bytes, for binary payloads |
json() |
Parsed JSON, raising on invalid input |
headers |
Case-insensitive response headers |
url |
Final URL after redirects |
raise_for_status() converts a 4xx or 5xx into an exception, which is the simplest way to avoid silently processing an error page as data.
3. Sessions
A Session reuses the underlying connection and persists cookies across requests. Both matter.
with requests.Session() as session:
session.headers.update({"User-Agent": "example-client/1.0"})
session.get("https://example.com/login") # sets cookies
page = session.get("https://example.com/account") # sends them back
Anything spanning more than one request β logins, consent, multi-step flows β needs a session. Connection reuse also makes repeated calls to the same host noticeably faster.
4. Sending Data
params=builds the query string with correct encoding.data=sends form-encoded fields.json=serialises a body and sets the JSON content type.files=handles multipart uploads.
Letting the library encode is safer than string-building a URL, which is where injection and encoding bugs originate.
5. Timeouts Are Not Optional
Without timeout, a request can hang indefinitely, and a single stalled call can freeze a pipeline. Every call in production code should carry one. A tuple sets connect and read timeouts separately.
6. What requests Does Not Do
- It does not run JavaScript. Client-rendered pages return the served shell.
- It is synchronous. Concurrency needs threads, or an async client such as
httpxoraiohttp. - It does not mimic a browser. Its TLS and header profile belong to a Python client, which is observable.
7. Common Misconceptions
- "Status 200 means valid data." Some APIs return errors in a 200 body; check the payload too.
- "
textis always correct." Decoding uses headers and guessing; for known encodings, decodecontentexplicitly. - "Sessions are only for logins." Connection reuse alone justifies them for repeated calls.
- "requests is deprecated." It is stable and maintained; async work simply uses a different library.
8. Summary
requests makes synchronous HTTP in Python concise: a call, a response object, and helpers for parameters, JSON, and errors. Sessions add cookie persistence and connection reuse.
Always set a timeout, always check status explicitly, and remember it fetches rather than renders β anything that only exists after scripts run needs a browser instead.