{"course":{"slug":"python-for-ai","title":"Python, from Zero, for AI","url":"https://addaly.com/learn/python-for-ai","level":"beginner","track":"build"},"lesson":{"slug":"threads-processes-and-the-gil","title":"Threads, processes, and the lock that decides which one you need","url":"https://addaly.com/learn/python-for-ai/threads-processes-and-the-gil","position":57,"of":89,"minutes":9,"answer":"Python threads overlap waiting but not computing, because one interpreter lock lets only one thread run Python at a time; use a ThreadPoolExecutor for network calls in synchronous code.","keyIdea":"Python threads overlap waiting but not computing, because one interpreter lock lets only one thread run Python at a time; use a ThreadPoolExecutor for network calls in synchronous code, and processes when the bottleneck is your own arithmetic."},"body":{"format":"markdown","text":"## Two kinds of slow\n\nA program is slow in one of two ways. It is **waiting** (for a server, a disk, a database) or it is **computing**: running your own loops and arithmetic. The tools for the two are different, and using the wrong one gives you a program that is exactly as slow as before, with extra complexity.\n\nThe previous lesson handled waiting with `asyncio`. This lesson covers the two other tools and the fact about Python that decides between them.\n\n## Threads\n\nA thread is a second line of execution inside the same process, sharing the same memory. Python's standard library gives you a pool of them:\n\n```python\nfrom concurrent.futures import ThreadPoolExecutor\n\ndef ask(prompt):\n    r = session.post(url, headers=headers, json=payload_for(prompt), timeout=60)\n    r.raise_for_status()\n    return r.json()\n\nwith ThreadPoolExecutor(max_workers=5) as pool:\n    results = list(pool.map(ask, prompts))\n```\n\nThat is the whole change. `ask` is an ordinary synchronous function using `requests`; nothing became `async`. `pool.map` runs it on up to five threads at once and returns results in input order. Twenty one-second calls finish in about four seconds with five workers.\n\nThis is the right tool when the code is already synchronous and only one corner needs to overlap its waiting. It is also the right tool when you depend on a library that has no async version. `max_workers` is your concurrency cap, doing the job the Semaphore did in the async version.\n\n`as_completed` gives results as they finish rather than in order, useful for printing progress:\n\n```python\nfrom concurrent.futures import as_completed\n\nwith ThreadPoolExecutor(max_workers=5) as pool:\n    futures = {pool.submit(ask, p): p for p in prompts}\n    for fut in as_completed(futures):\n        prompt = futures[fut]\n        try:\n            save(prompt, fut.result())\n        except Exception as e:\n            log.warning(\"failed %s: %s\", prompt[:30], e)\n```\n\n`fut.result()` re-raises whatever exception the function raised on its thread, so errors are handled where you can see them, not lost on a background thread.\n\n## The lock\n\nHere is the fact. CPython (the Python you installed) has a **global interpreter lock**, the GIL. Only one thread can execute Python bytecode at any moment. Eight threads doing arithmetic on an eight-core machine take turns on one core and finish no faster than one thread would. The lock exists because CPython's memory management was designed around it, and removing it without slowing single-threaded code has taken decades of work; a free-threaded build exists as an option in recent versions and is not yet the default.\n\nSo why do threads help with network calls? Because a thread **releases the GIL while it waits**. Waiting for a socket, a file, a sleep, all of these hand the lock to another thread. Threads overlap waiting perfectly and overlap computing not at all.\n\nThe same is true of C code that releases the lock on purpose. NumPy releases it inside large array operations, which is one reason NumPy on eight cores can be parallel while a Python loop over the same numbers is not.\n\n## Processes\n\nFor computing, the tool is a process pool. Each process is a separate Python interpreter with its own lock, so eight of them use eight cores.\n\n```python\nfrom concurrent.futures import ProcessPoolExecutor\n\ndef score(chunk):\n    return [expensive_similarity(a, b) for a, b in chunk]\n\nif __name__ == \"__main__\":\n    chunks = split(pairs, 8)\n    with ProcessPoolExecutor(max_workers=8) as pool:\n        results = list(pool.map(score, chunks))\n```\n\nTwo costs come with it.\n\nNothing is shared. Every argument is serialised with Python's `pickle` module, sent to the worker process, and the result serialised back. Sending a 2 GB array to eight workers copies it eight times. Send small descriptions of work (a filename, a range of indices) and let each worker load what it needs. One safety note while the word is in front of you: `pickle` can execute code when it loads, so it is only ever safe between your own processes; never load a pickled file that came from someone else, and use JSON for anything that crosses a trust boundary.\n\nAnything you pass must be picklable. Functions defined at module level are; lambdas, nested functions and open connections are not. The `if __name__ == \"__main__\":` guard is mandatory on macOS and Windows, where a new process re-imports your script and would otherwise start another pool inside itself, forever.\n\n## Which one\n\nAsk what the program is doing while it is slow.\n\n- Waiting on the network, and the code is already async or can be: `asyncio`.\n- Waiting on the network, and the code is synchronous: `ThreadPoolExecutor`.\n- Computing in pure Python: `ProcessPoolExecutor`, or rewrite the hot loop in NumPy, which is usually faster than either.\n- Computing in NumPy already: it may be parallel already; measure before adding processes.\n\nMeasure. `time.perf_counter()` around the slow part, run once with one worker and once with several. If more workers do not help, you have the wrong tool, and the number tells you before you have restructured the program.\n\n## The shared-state trap\n\nThreads share memory, and that cuts both ways. Two threads appending to the same list is fine, because `list.append` is one bytecode operation and cannot be interrupted midway. Two threads doing `counter += 1` is not: read, add, write are three steps, and a thread can be switched out between them, so counts go missing. Guard shared mutation with a lock, or, more simply, have each thread return its result and combine them in the main thread afterwards, as `pool.map` already does.\n\nA Session from `requests` is safe to share across threads for ordinary use; the pool inside it is designed for that. A `sqlite3` connection is not; open one per thread.\n\n## Try this now\n\nWrite a pure-Python function that sums the squares of a million numbers. Time it once, then with `ThreadPoolExecutor(4)` splitting the range four ways, then with `ProcessPoolExecutor(4)`. Then do the same sum with `numpy.arange(1_000_000) ** 2` and `.sum()`. Write the four numbers down; they are the whole lesson."},"check":{"question":"A script computes cosine similarity between 100,000 pairs of vectors in pure Python and takes 40 seconds. The developer splits the work across a `ThreadPoolExecutor` with 8 workers on an 8-core machine and it takes 41 seconds. What happened?","options":[{"text":"The machine's cores were already busy with the operating system and background services, so no extra capacity was available to the threads, and the pool merely queued them.","correct":false,"tempting":"An 8-core machine running one script has spare cores; the limit is inside Python, not the hardware."},{"text":"The pairs were too small to divide efficiently, so the overhead of distributing them cancelled the gain.","correct":false,"tempting":"Overhead of that size would show as a small loss, not a complete absence of any speed-up across eight cores."},{"text":"The interpreter lock lets one thread run Python bytecode at a time, so eight threads of arithmetic take turns on one core; processes or NumPy would use the rest.","correct":true},{"text":"Threads share memory, and the vectors were copied into each thread on every access, adding more time than the parallelism saved.","correct":false,"tempting":"Shared memory is the opposite of copying; threads read the same objects without duplication."}]},"prev":{"slug":"asyncio-and-concurrent-calls","title":"asyncio: twenty calls in the time of one","url":"https://addaly.com/learn/python-for-ai/asyncio-and-concurrent-calls"},"next":{"slug":"validating-what-the-model-returns","title":"Validating what the model returns: from a string that looks like JSON to an object you can trust","url":"https://addaly.com/learn/python-for-ai/validating-what-the-model-returns"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Source and assessment review continues. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Threads, processes, and the lock that decides which one you need', in Python, from Zero, for AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/python-for-ai/threads-processes-and-the-gil (Accessed: 21 November 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Threads, processes, and the lock that decides which one you need","courseTitle":"Python, from Zero, for AI","module":{"index":6,"slug":"talking-to-a-service","title":"Talking to a service: HTTP from Python, done properly"},"answer":null,"keyIdea":"Python threads overlap waiting but not computing, because one interpreter lock lets only one thread run Python at a time; use a ThreadPoolExecutor for network calls in synchronous code, and processes when the bottleneck is your own arithmetic.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-kinds-of-slow","runs":[{"text":"Two kinds of slow"}]},{"t":"p","runs":[{"text":"A program is slow in one of two ways. It is "},{"text":"waiting","b":true},{"text":" (for a server, a disk, a database) or it is "},{"text":"computing","b":true},{"text":": running your own loops and arithmetic. The tools for the two are different, and using the wrong one gives you a program that is exactly as slow as before, with extra complexity."}]},{"t":"p","runs":[{"text":"The previous lesson handled waiting with "},{"text":"asyncio","code":true},{"text":". This lesson covers the two other tools and the fact about Python that decides between them."}]},{"t":"h2","id":"threads","runs":[{"text":"Threads"}]},{"t":"p","runs":[{"text":"A thread is a second line of execution inside the same process, sharing the same memory. Python’s standard library gives you a pool of them:"}]},{"t":"code","lang":"python","label":"Python","text":"from concurrent.futures import ThreadPoolExecutor\n\ndef ask(prompt):\n    r = session.post(url, headers=headers, json=payload_for(prompt), timeout=60)\n    r.raise_for_status()\n    return r.json()\n\nwith ThreadPoolExecutor(max_workers=5) as pool:\n    results = list(pool.map(ask, prompts))"},{"t":"p","runs":[{"text":"That is the whole change. "},{"text":"ask","code":true},{"text":" is an ordinary synchronous function using "},{"text":"requests","code":true},{"text":"; nothing became "},{"text":"async","code":true},{"text":". "},{"text":"pool.map","code":true},{"text":" runs it on up to five threads at once and returns results in input order. Twenty one-second calls finish in about four seconds with five workers."}]},{"t":"p","runs":[{"text":"This is the right tool when the code is already synchronous and only one corner needs to overlap its waiting. It is also the right tool when you depend on a library that has no async version. "},{"text":"max_workers","code":true},{"text":" is your concurrency cap, doing the job the Semaphore did in the async version."}]},{"t":"p","runs":[{"text":"as_completed","code":true},{"text":" gives results as they finish rather than in order, useful for printing progress:"}]},{"t":"code","lang":"python","label":"Python","text":"from concurrent.futures import as_completed\n\nwith ThreadPoolExecutor(max_workers=5) as pool:\n    futures = {pool.submit(ask, p): p for p in prompts}\n    for fut in as_completed(futures):\n        prompt = futures[fut]\n        try:\n            save(prompt, fut.result())\n        except Exception as e:\n            log.warning(\"failed %s: %s\", prompt[:30], e)"},{"t":"p","runs":[{"text":"fut.result()","code":true},{"text":" re-raises whatever exception the function raised on its thread, so errors are handled where you can see them, not lost on a background thread."}]},{"t":"h2","id":"the-lock","runs":[{"text":"The lock"}]},{"t":"p","runs":[{"text":"Here is the fact. CPython (the Python you installed) has a "},{"text":"global interpreter lock","b":true},{"text":", the GIL. Only one thread can execute Python bytecode at any moment. Eight threads doing arithmetic on an eight-core machine take turns on one core and finish no faster than one thread would. The lock exists because CPython’s memory management was designed around it, and removing it without slowing single-threaded code has taken decades of work; a free-threaded build exists as an option in recent versions and is not yet the default."}]},{"t":"p","runs":[{"text":"So why do threads help with network calls? Because a thread "},{"text":"releases the GIL while it waits","b":true},{"text":". Waiting for a socket, a file, a sleep, all of these hand the lock to another thread. Threads overlap waiting perfectly and overlap computing not at all."}]},{"t":"p","runs":[{"text":"The same is true of C code that releases the lock on purpose. NumPy releases it inside large array operations, which is one reason NumPy on eight cores can be parallel while a Python loop over the same numbers is not."}]},{"t":"h2","id":"processes","runs":[{"text":"Processes"}]},{"t":"p","runs":[{"text":"For computing, the tool is a process pool. Each process is a separate Python interpreter with its own lock, so eight of them use eight cores."}]},{"t":"code","lang":"python","label":"Python","text":"from concurrent.futures import ProcessPoolExecutor\n\ndef score(chunk):\n    return [expensive_similarity(a, b) for a, b in chunk]\n\nif __name__ == \"__main__\":\n    chunks = split(pairs, 8)\n    with ProcessPoolExecutor(max_workers=8) as pool:\n        results = list(pool.map(score, chunks))"},{"t":"p","runs":[{"text":"Two costs come with it."}]},{"t":"p","runs":[{"text":"Nothing is shared. Every argument is serialised with Python’s "},{"text":"pickle","code":true},{"text":" module, sent to the worker process, and the result serialised back. Sending a 2 GB array to eight workers copies it eight times. Send small descriptions of work (a filename, a range of indices) and let each worker load what it needs. One safety note while the word is in front of you: "},{"text":"pickle","code":true},{"text":" can execute code when it loads, so it is only ever safe between your own processes; never load a pickled file that came from someone else, and use JSON for anything that crosses a trust boundary."}]},{"t":"p","runs":[{"text":"Anything you pass must be picklable. Functions defined at module level are; lambdas, nested functions and open connections are not. The "},{"text":"if __name__ == \"__main__\":","code":true},{"text":" guard is mandatory on macOS and Windows, where a new process re-imports your script and would otherwise start another pool inside itself, forever."}]},{"t":"h2","id":"which-one","runs":[{"text":"Which one"}]},{"t":"p","runs":[{"text":"Ask what the program is doing while it is slow."}]},{"t":"ul","items":[[{"text":"Waiting on the network, and the code is already async or can be: "},{"text":"asyncio","code":true},{"text":"."}],[{"text":"Waiting on the network, and the code is synchronous: "},{"text":"ThreadPoolExecutor","code":true},{"text":"."}],[{"text":"Computing in pure Python: "},{"text":"ProcessPoolExecutor","code":true},{"text":", or rewrite the hot loop in NumPy, which is usually faster than either."}],[{"text":"Computing in NumPy already: it may be parallel already; measure before adding processes."}]]},{"t":"p","runs":[{"text":"Measure. "},{"text":"time.perf_counter()","code":true},{"text":" around the slow part, run once with one worker and once with several. If more workers do not help, you have the wrong tool, and the number tells you before you have restructured the program."}]},{"t":"h2","id":"the-shared-state-trap","runs":[{"text":"The shared-state trap"}]},{"t":"p","runs":[{"text":"Threads share memory, and that cuts both ways. Two threads appending to the same list is fine, because "},{"text":"list.append","code":true},{"text":" is one bytecode operation and cannot be interrupted midway. Two threads doing "},{"text":"counter += 1","code":true},{"text":" is not: read, add, write are three steps, and a thread can be switched out between them, so counts go missing. Guard shared mutation with a lock, or, more simply, have each thread return its result and combine them in the main thread afterwards, as "},{"text":"pool.map","code":true},{"text":" already does."}]},{"t":"p","runs":[{"text":"A Session from "},{"text":"requests","code":true},{"text":" is safe to share across threads for ordinary use; the pool inside it is designed for that. A "},{"text":"sqlite3","code":true},{"text":" connection is not; open one per thread."}]},{"t":"h2","id":"try-this-now","runs":[{"text":"Try this now"}],"role":"try"},{"t":"p","runs":[{"text":"Write a pure-Python function that sums the squares of a million numbers. Time it once, then with "},{"text":"ThreadPoolExecutor(4)","code":true},{"text":" splitting the range four ways, then with "},{"text":"ProcessPoolExecutor(4)","code":true},{"text":". Then do the same sum with "},{"text":"numpy.arange(1_000_000) ** 2","code":true},{"text":" and "},{"text":".sum()","code":true},{"text":". Write the four numbers down; they are the whole lesson."}]}],"check":{"question":[{"text":"A script computes cosine similarity between 100,000 pairs of vectors in pure Python and takes 40 seconds. The developer splits the work across a "},{"text":"ThreadPoolExecutor","code":true},{"text":" with 8 workers on an 8-core machine and it takes 41 seconds. What happened?"}],"options":[{"text":[{"text":"The machine’s cores were already busy with the operating system and background services, so no extra capacity was available to the threads, and the pool merely queued them."}],"correct":false,"tempting":[{"text":"An 8-core machine running one script has spare cores; the limit is inside Python, not the hardware."}]},{"text":[{"text":"The pairs were too small to divide efficiently, so the overhead of distributing them cancelled the gain."}],"correct":false,"tempting":[{"text":"Overhead of that size would show as a small loss, not a complete absence of any speed-up across eight cores."}]},{"text":[{"text":"The interpreter lock lets one thread run Python bytecode at a time, so eight threads of arithmetic take turns on one core; processes or NumPy would use the rest."}],"correct":true,"tempting":null},{"text":[{"text":"Threads share memory, and the vectors were copied into each thread on every access, adding more time than the parallelism saved."}],"correct":false,"tempting":[{"text":"Shared memory is the opposite of copying; threads read the same objects without duplication."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"asyncio-and-concurrent-calls","title":"asyncio: twenty calls in the time of one","minutes":10},"next":{"slug":"validating-what-the-model-returns","title":"Validating what the model returns: from a string that looks like JSON to an object you can trust","minutes":9}}}