38.HTTP & APIs with requests/httpx
In this lesson, we explore how Python interacts with web services using HTTP protocols. We focus on two popular libraries: `requests` and `httpx`. These libraries allow developers to send HTTP requests, handle responses, and integrate with APIs efficiently.
Theoretical Foundations
HTTP (HyperText Transfer Protocol) is the foundation of data communication on the web. APIs (Application Programming Interfaces) expose endpoints that allow clients to interact with services over HTTP. Python provides libraries to facilitate these interactions, abstracting the complexity of raw socket communication.
Using the requests Library
Example:
import requests
response = requests.get(“https://api.example.com/data”)
if response.status_code == 200:
print(response.json())
Using the httpx Library
The `httpx` library is a modern alternative to `requests`, supporting both synchronous and asynchronous HTTP calls.
Example (synchronous):
import httpx
response = httpx.get(“https://api.example.com/data”)
if response.status_code == 200:
print(response.json())
Example (asynchronous):
import httpx
import asyncio
async def fetch_data():
async with httpx.AsyncClient() as client:
response = await client.get(“https://api.example.com/data”)
if response.status_code == 200:
print(response.json())
asyncio.run(fetch_data())
Comparison: requests vs httpx
Feature | requests | httpx |
Synchronous Support | Yes | Yes |
Asynchronous Support | No | Yes |
HTTP/2 Support | No | Yes |
Connection Pooling | Basic | Advanced |
Timeouts and Retries | Yes | Yes |
Streaming Responses | Yes | Yes |
Use Cases
– Consuming REST APIs
– Sending form data or JSON payloads
– Downloading files
– Integrating with third-party services
– Building web scrapers
Best Practices
– Always handle exceptions (e.g., timeouts, connection errors)
– Use timeouts to avoid hanging requests
– Validate and sanitize API responses
– Use asynchronous calls for high-performance applications
– Respect API rate limits and authentication requirements
Common Pitfalls
– Ignoring error codes and assuming success
– Not using timeouts, leading to hanging requests
– Overlooking asynchronous capabilities of httpx
– Mismanaging authentication tokens or headers