Sessions: paying for the handshake once instead of every call

What a single call really does

requests.get(url) looks like one action. Underneath, it performs roughly this sequence:

1. Resolve the hostname to an IP address (DNS).
2. Open a TCP connection: one round trip to the server and back.
3. Negotiate TLS, the encryption under https: one or two more round trips, plus some arithmetic on both ends.
4. Send the request and wait for the response.
5. Close the connection.

Steps 1 to 3 happen before a single byte of your request moves. From a laptop in Delhi to a server in Virginia, a round trip is around 200 ms, so a bare call to a fast API can spend 400 to 600 ms setting up and 40 ms doing the work. Then step 5 throws the connection away, and the next call starts from nothing.

For one call, nobody cares. For a loop of five hundred, this is the difference between a script that finishes over coffee and one that finishes tomorrow.

The Session

import requests

session = requests.Session()
session.headers.update({
    "Authorization": f"Bearer {key}",
    "User-Agent": "my-tagger/0.1",
})

for item in items:
    r = session.post(url, json={"text": item}, timeout=30)
    r.raise_for_status()
    results.append(r.json())

Three things changed.

The connection stays open between calls. HTTP calls this keep-alive, and every modern server supports it. The second call to the same host skips DNS, TCP and TLS entirely.

The headers are set once on the session and sent with every request, so a forgotten header on one call is no longer possible.

The Session also holds cookies, which matters for a few APIs that set one on login and expect it back.

Measure it yourself:

import time, requests

url = "https://httpbin.org/get"

t = time.perf_counter()
for _ in range(20):
    requests.get(url, timeout=10)
print("bare:   ", round(time.perf_counter() - t, 2), "s")

t = time.perf_counter()
with requests.Session() as s:
    for _ in range(20):
        s.get(url, timeout=10)
print("session:", round(time.perf_counter() - t, 2), "s")

The gap depends on how far the server is. It is rarely less than two to one.

The pool underneath

A Session does not hold one connection; it holds a small pool of them per host, ten by default. If you later run requests on several threads, the pool hands each thread a connection and opens more when they run out, up to a limit. When more threads want a connection than the pool has, the extras wait, and requests logs a warning about the pool being full. Raising the limit is a one-line change:

from requests.adapters import HTTPAdapter

session.mount("https://", HTTPAdapter(pool_connections=20, pool_maxsize=20))

This is also where retries are configured, which the next lesson does.

Close it

A Session holds open sockets. On a short script the operating system reclaims them when the process exits, and nobody notices. In a long-running program that creates a new Session per request and never closes it, sockets accumulate until the process runs out of file descriptors and every network call starts failing with a message about "too many open files". The fix is to create one Session for the life of the program, or to use it as a context manager so it closes itself:

with requests.Session() as session:
    ...

Create it once at module level or pass it in; do not create one inside the function that makes the call.

What the provider SDKs do

If you use a provider's own library — openai, anthropic, httpx under both — this is already handled. The client object you construct once is the session, with the pool, keep-alive and default headers. Which is one reason to construct it once and reuse it, rather than building a new client inside every function. A later lesson covers what else those clients do for you.

Where the handshake still happens

Keep-alive is a request, not a guarantee. Servers close idle connections after a while, often 60 seconds, and some proxies and load balancers close them sooner. A Session that sits idle for five minutes will pay the handshake again on its next call, and requests handles that silently. Occasionally the server closes the connection at the exact moment you send on it, and you see a ConnectionError with "Connection reset by peer" or "RemoteDisconnected". That is not your bug. It is the reason retrying an idempotent request once is reasonable, which is the next lesson's subject.

Try this now

Take the loop from your first API call and move it onto a Session. Time both versions with time.perf_counter(). Then set a header on the session, make a call, and print r.request.headers to confirm the header went out without you passing it.