Python, from Zero, for AILesson 56 of 89

Module 6 · Talking to a service: HTTP from Python, done properly

asyncio: twenty calls in the time of one

In this lesson

  • 10 min read
  • 8 reading sections
  • Knowledge check
  • Lesson 8 of 11 in this chapter

This chapter’s goalWrite a client for a paid model API that reuses connections, times out, retries only what is safe to retry, walks pagination with a generator, consumes a streamed response as it arrives, runs many requests concurrently under a rate limit, validates the JSON the model returns before trusting it, and reports the cost of every call from the usage figures in the response.

This chapter’s media

The film explains chapter ideas. The audiobook narrates the complete teaching text.

FormatEnglishहिंदी
Chapter filmWatch2:28Not available
Full audiobookNot availableNot available
Wall-clock time for one-second API calls0501005100Number of one-second requestsSeconds until all are done—— Plain loop, one at a time– – asyncio.gather with Semaphore(5)Sequential time is the sum of the waits. Concurrent time is the longest wait, plus a little. TheSemaphore line is what respecting a rate limit costs (five at a time rather than all at once) and itis still an order of magnitude faster than the loop. None of this makes one request quicker.
Sequential time is the sum of the waits. Concurrent time is the longest wait, plus a little. The Semaphore line is what respecting a rate limit costs (five at a time rather than all at once) and it is still an order of magnitude faster than the loop. None of this makes one request quicker.

Figure

A little at a time

Explore an idea, try it, then check what stays.

Step 1 of 10Free · no account needed
Lesson steps

Where the time goes

Call a model API in a loop and time it:

Python
for prompt in prompts:          # 20 prompts, about 1 second each
    results.append(ask(prompt)) # ≈ 20 seconds

Your program does almost nothing during those twenty seconds. It sends a request, then waits, then sends the next. The CPU is idle; the network is idle most of the time too. The waiting is the cost, and the waits could overlap: if all twenty requests were in flight at once, the whole batch would take about as long as the slowest one.

asyncio is Python’s way of overlapping waits on a single thread.

Your place and first choices stay on this browser. Classroom credit is recorded separately.

Stuck on something?Ask Divya, your AI tutor

How to cite: Addaly (2026) ‘asyncio: twenty calls in the time of one’, in Python, from Zero, for AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/python-for-ai/asyncio-and-concurrent-calls (Accessed: 21 November 2026).