Python, from Zero, for AILesson 52 of 89

Module 6 · Talking to a service: HTTP from Python, done properly

Timeouts and retries: the code that decides whether you pay twice

In this lesson

  • 9 min read
  • 8 reading sections
  • Knowledge check
  • Lesson 4 of 11 in this chapter

This chapter’s goalWrite a client for a paid model API that reuses connections, times out, retries only what is safe to retry, walks pagination with a generator, consumes a streamed response as it arrives, runs many requests concurrently under a rate limit, validates the JSON the model returns before trusting it, and reports the cost of every call from the usage figures in the response.

This chapter’s media

The film explains chapter ideas. The audiobook narrates the complete teaching text.

FormatEnglishहिंदी
Chapter filmWatch2:28Not available
Full audiobookNot availableNot available
Which failures are safe to retryGETPOST that billsFailed to connectRetrynothing was sentRetrynothing was sentRead timed outRetryno side effectsDo not retrymay already be doneThe question is never how bad the error looks; it is whether the attempt provably did nothing. 429 and503 are safe for the same reason as a connect failure: both mean not now. A 500 is not, because youcannot tell whether it broke before or after the work. An Idempotency-Key, where the provider offersone, makes the bottom right cell safe.
The question is never how bad the error looks; it is whether the attempt provably did nothing. 429 and 503 are safe for the same reason as a connect failure: both mean not now. A 500 is not, because you cannot tell whether it broke before or after the work. An Idempotency-Key, where the provider offers one, makes the bottom right cell safe.

Figure

A little at a time

Explore an idea, try it, then check what stays.

Step 1 of 10Free · no account needed
Lesson steps

Two timeouts, not one

timeout=10 is shorthand for a pair:

Python
r = session.post(url, json=payload, timeout=(5, 120))

The first number is the connect timeout: how long to wait for the server to accept the connection. If this expires, nothing was sent, and you know it. The second is the read timeout: how long to wait, between bytes, for the server to send something. If this expires, the request was sent and the server may be working on it, or may have finished.

For a model API the two need different values. Connecting should take under a second; five is generous. Generating 1,000 tokens can legitimately take a minute, so a read timeout of 10 seconds guarantees failures on the longest, most expensive requests: precisely the ones you least want to abandon. Set it from the longest response you expect, then add half again.

Your place and first choices stay on this browser. Classroom credit is recorded separately.

Stuck on something?Ask Divya, your AI tutor

How to cite: Addaly (2026) ‘Timeouts and retries: the code that decides whether you pay twice’, in Python, from Zero, for AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/python-for-ai/timeouts-retries-and-backoff (Accessed: 21 November 2026).