Chapter 11: Working with APIs — Making API Requests (using requests)

⚙️ Making API Requests in Python with the requests Library

APIs communicate using HTTP requests and responses. Python’s requests library is one of the simplest and most powerful tools for working with HTTP-based APIs. It abstracts complex networking operations into clean, human-readable code.

---

🌍 1. The Request–Response Cycle

Whenever you interact with an API:

1. You send a request (with optional data, headers, etc.).
2. The server processes it and sends a response.
3. The response includes a status code, headers, and body (data).

Method	Description	Example
GET	Retrieve data	Fetch user info
POST	Send or create data	Submit form or upload file
PUT	Update data	Edit a record
PATCH	Partially update data	Update one field
DELETE	Remove data	Delete a resource

---

🧱 2. Sending a Basic GET Request

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

print("Status Code:", response.status_code)
print("Response Body:", response.text)

✅ Tip: Use .json() when you know the response is JSON:

data = response.json()
print("Title:", data["title"])

---

✉️ 3. Sending Query Parameters

You can pass parameters to refine your GET requests using the params argument.

url = "https://jsonplaceholder.typicode.com/comments"
params = {"postId": 1}

response = requests.get(url, params=params)
print("Request URL:", response.url)
print("Number of comments:", len(response.json()))

---

🧾 4. Sending Data with POST Requests

Sending Form Data

url = "https://httpbin.org/post"
data = {"name": "Rambod", "message": "Hello API!"}

response = requests.post(url, data=data)
print(response.json())

Sending JSON Payloads

url = "https://httpbin.org/post"
payload = {"title": "New Post", "body": "Learning APIs", "userId": 42}

response = requests.post(url, json=payload)
print(response.json())

✅ Always use json=payload to automatically encode the data and set the Content-Type header.

---

🧠 5. Custom Headers and Authentication

You can send headers or tokens to authenticate or customize your request.

url = "https://api.github.com/user"
headers = {"Authorization": "Bearer YOUR_GITHUB_TOKEN"}

response = requests.get(url, headers=headers)
print(response.status_code)
print(response.json())

---

🔒 6. Using Sessions for Persistent Connections

A Session object keeps cookies and headers across multiple requests — improving performance.

with requests.Session() as session:
    session.headers.update({"User-Agent": "RambodAPI/1.0"})
    r1 = session.get("https://httpbin.org/cookies/set/sessionid/123")
    r2 = session.get("https://httpbin.org/cookies")
    print(r2.json())

---

🕵️ 7. Handling Responses Properly

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

print("Status Code:", response.status_code)
print("Headers:", response.headers)
print("JSON Data:", response.json())

Common Response Attributes

Attribute	Description
.status_code	HTTP status (e.g., 200, 404)
.headers	Metadata like content type, date
.text	Raw string content
.json()	Parse response as JSON

---

🚨 8. Error Handling and Exceptions

Always check for errors before using the response.

import requests

url = "https://jsonplaceholder.typicode.com/posts/99999"

try:
    response = requests.get(url, timeout=5)
    response.raise_for_status()
except requests.exceptions.HTTPError as err:
    print("HTTP error:", err)
except requests.exceptions.RequestException as e:
    print("Network error:", e)
else:
    print("Success:", response.json())

💡 Use timeout to prevent hanging requests in case of network issues.

---

🔁 9. Uploading Files

You can upload files to APIs that accept file inputs.

url = "https://httpbin.org/post"
files = {"file": open("data.txt", "rb")}
response = requests.post(url, files=files)

print(response.json())

---

🧱 10. Downloading and Streaming Data

For large files, stream the response instead of loading it all into memory.

url = "https://www.example.com/largefile.zip"
response = requests.get(url, stream=True)

with open("downloaded.zip", "wb") as f:
    for chunk in response.iter_content(chunk_size=8192):
        f.write(chunk)

⚙️ Always use stream=True for large files to avoid memory overload.

---

🧩 11. Handling Pagination

Many APIs return data in pages. Use query parameters to iterate through them.

for page in range(1, 4):
    response = requests.get("https://jsonplaceholder.typicode.com/posts", params={"_page": page})
    print(f"Page {page}:", len(response.json()), "records")

---

🧠 12. Retrying Failed Requests (Advanced)

Use HTTPAdapter for automatic retries.

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
import requests

session = requests.Session()
retry = Retry(connect=3, backoff_factor=0.5)
adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)

response = session.get("https://httpbin.org/status/500")
print("Status:", response.status_code)

---

🧰 13. Debugging and Inspection

Enable request inspection using response.request.

print("Request Method:", response.request.method)
print("Request Headers:", response.request.headers)
print("Request Body:", response.request.body)

---

🧭 14. Best Practices for requests

✅ Always use timeout to avoid infinite waiting.
✅ Handle exceptions with try/except.
✅ Use Session for multiple calls.
✅ Don’t log API keys.
✅ Use pagination instead of downloading all data at once.
✅ Prefer json= instead of data= for structured APIs.
✅ Check .status_code before parsing .json().

---

🧠 Summary

Concept	Description	Example
GET	Retrieve data	requests.get(url)
POST	Send data	requests.post(url, json=payload)
Headers	Metadata for requests	headers={'Auth': 'token'}
Response	Contains status, headers, and content	response.json()
Error Handling	Detect and recover from issues	raise_for_status()
Session	Reuse settings across requests	requests.Session()

Mastering the requests library gives you the power to interact with nearly any API — efficiently, securely, and professionally.