{"course":{"slug":"python-for-ai","title":"Python, from Zero, for AI","url":"https://addaly.com/learn/python-for-ai","level":"beginner","track":"build"},"lesson":{"slug":"profiling-where-time-goes","title":"Profiling: finding where the time actually goes before you optimise anything","url":"https://addaly.com/learn/python-for-ai/profiling-where-time-goes","position":69,"of":89,"minutes":9,"answer":"Measure before you change: perf_counter around suspects, cProfile sorted by cumulative time for the whole program, tracemalloc for memory, and in an AI program the answer is almost always the network call.","keyIdea":"Measure before you change: perf_counter around suspects, cProfile sorted by cumulative time for the whole program, tracemalloc for memory, and in an AI program the answer is almost always the network call, not the loop you were about to rewrite."},"body":{"format":"markdown","text":"## Guessing is wrong more often than not\n\nEvery programmer has a story of optimising the wrong thing. The loop that looked slow was 0.3 per cent of the run; the innocent-looking line that formatted a date was called two million times. Intuition about performance is poor even in experienced people, because the cost of an operation is invisible in the source. The remedy is to measure, and Python ships the instruments.\n\n## The stopwatch\n\n```python\nimport time\n\nt = time.perf_counter()\nvectors = embed(chunks)\nprint(f\"embed: {time.perf_counter() - t:.2f}s\")\n```\n\n`perf_counter` is a high-resolution clock for measuring intervals. `time.time()` is wall-clock time and can jump when the system clock adjusts; do not use it for durations. The `timed` context manager from earlier in this module is this pattern packaged.\n\n> Figure: Where fourteen minutes went in a tagging script. Bars (% of total run time): Waiting for the model API 91; Parsing the JSON replies 4; Reading and writing the CSV 4; The loop everyone wanted to rewrite 1. Rewriting the loop perfectly saves about eight seconds. Caching repeated prompts and overlapping the calls saves twelve minutes. Nobody can tell these apart by reading the code, which is the entire argument for putting a stopwatch around three suspects before changing anything.\n\nPut stopwatches around the three or four things you suspect, run once, and read the numbers. This alone settles most performance questions in an AI program, and the answer is usually that the network call is 95 per cent of the time and everything you were about to rewrite is noise.\n\n## `timeit` for small things\n\nFor a single expression, the stopwatch is too coarse: one run is dominated by noise. `timeit` runs it many times and reports the best:\n\n```bash\npython -m timeit -s \"s = 'a' * 1000\" \"s.split()\"\n```\n\n```python\nimport timeit\ntimeit.timeit(\"''.join(parts)\", setup=\"parts = ['x'] * 1000\", number=10_000)\n```\n\nIn a notebook, `%timeit expr` does the same with a friendlier report. Use it to settle \"is a comprehension faster than a loop here\" questions honestly: usually a little, occasionally not, and rarely enough to matter.\n\n## `cProfile` for the whole program\n\n```bash\npython -m cProfile -o run.prof -m tagger.cli input.csv\npython -c \"import pstats; pstats.Stats('run.prof').sort_stats('cumulative').print_stats(20)\"\n```\n\nThe output is a table of every function called: how many times, how long inside it (`tottime`), and how long including everything it called (`cumtime`). Sort by `cumulative` and read from the top: the first lines are your program's entry points, and a few lines down is the first function that is not merely a wrapper: that is where the time lives. Sort by `tottime` to find the function that is itself expensive rather than expensive because of what it calls.\n\nTwo things to know about reading it. The profiler adds overhead, so absolute numbers are inflated; the proportions are what you trust. And `{built-in method ...}` or `{method 'read' of '_ssl._SSLSocket'}` near the top means the time is being spent waiting on the network, which no Python change will fix: only fewer calls, concurrent calls, or caching.\n\n`snakeviz run.prof` (pip-installable, free) draws the same data as a clickable sunburst, and is worth the install the first time a profile is more than a screen long.\n\n## The string trap, measured\n\nOne pure-Python cost that does matter and does show up:\n\n```python\ntext = \"\"\nfor part in parts:          # 100,000 parts\n    text += part            # copies the growing string each time\n```\n\nEach `+=` builds a new string containing everything so far, so the total work is proportional to the square of the length. `\"\".join(parts)` is one pass. At 100,000 parts the loop can take seconds and the join milliseconds, which is why module 6's streaming lesson collected fragments in a list. `timeit` both and you will remember it.\n\n## Import time\n\nA command that takes two seconds to print `--help` is usually paying for imports. `pandas` alone is around half a second; `torch` several.\n\n```bash\npython -X importtime -c \"import tagger.cli\" 2> imports.log\nsort -t'|' -k2 -n imports.log | tail -20\n```\n\nlists the slowest imports. The fix is to import heavy libraries inside the function that needs them, so `--help` and the fast paths do not pay for them.\n\n## Memory\n\n```python\nimport tracemalloc\n\ntracemalloc.start()\ndata = load_everything(path)\ncurrent, peak = tracemalloc.get_traced_memory()\nprint(f\"peak {peak / 1e6:.1f} MB\")\ntracemalloc.stop()\n```\n\n`peak` is the number that matters: it is what the machine had to have. For a closer look, `tracemalloc.take_snapshot().statistics(\"lineno\")[:10]` shows the ten lines that allocated the most. When a process is killed on a small server with no traceback, this is how you find out which structure grew.\n\n`sys.getsizeof(obj)` reports one object's own size and not what it references; a list of a million strings reports the list, not the strings. It is fine for \"how big is this array\" and misleading for containers.\n\n## The order of operations\n\n1. Measure with a stopwatch. Find the 80 per cent.\n2. If it is the network: fewer calls (cache, batch), overlapped calls (module 6), or a smaller model. Nothing else helps.\n3. If it is your Python: profile, find the top function, and look for a better algorithm or a NumPy replacement before micro-optimising syntax.\n4. Measure again. Keep the change only if the number moved.\n\nStep four is the one people skip. A change that made the code harder to read and the run two seconds faster out of fourteen minutes should be reverted, and only the number tells you that.\n\n## Try this now\n\nProfile your tagging script with `cProfile`, sort by cumulative, and write down what fraction of the total is inside the HTTP library. Then `timeit` string concatenation against `join` at 10,000 parts, and `-X importtime` your CLI to see what `--help` is paying for."},"check":{"question":"A script that tags 2,000 messages takes 14 minutes. The developer spends an afternoon replacing a list-building loop with a comprehension and a dict lookup with a set, and it now takes 13 minutes 58 seconds. A cProfile run sorted by cumulative time would most likely have shown what?","options":[{"text":"That the comprehension was already the fastest possible form and the set was slower than the dict for small collections, so the two changes cancelled each other out almost exactly.","correct":false,"tempting":"Sets and comprehensions are not slower; the two-second change shows those parts were never a meaningful share of the time."},{"text":"That the model API call took nearly all of the 14 minutes, so no change to the pure-Python parts could matter by more than seconds.","correct":true},{"text":"That garbage collection was consuming most of the run, and `gc.disable()` at the top would have fixed it.","correct":false,"tempting":"Garbage collection rarely dominates, and disabling it in a long-running job trades a small saving for growing memory."},{"text":"That `print` statements were the bottleneck, since terminal output is synchronous.","correct":false,"tempting":"Printing 2,000 lines costs well under a second; it cannot account for minutes."}]},"prev":{"slug":"packaging-with-pyproject","title":"pyproject.toml: making your project installable, and a command somebody can type","url":"https://addaly.com/learn/python-for-ai/packaging-with-pyproject"},"next":{"slug":"numpy-arrays","title":"NumPy arrays: why one line beats a loop by a hundred times","url":"https://addaly.com/learn/python-for-ai/numpy-arrays"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Source and assessment review continues. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Profiling: finding where the time actually goes before you optimise anything', in Python, from Zero, for AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/python-for-ai/profiling-where-time-goes (Accessed: 21 November 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Profiling: finding where the time actually goes before you optimise anything","courseTitle":"Python, from Zero, for AI","module":{"index":7,"slug":"objects-and-the-shape-of-programs","title":"Objects, generators and the shape a larger program takes"},"answer":null,"keyIdea":"Measure before you change: perf_counter around suspects, cProfile sorted by cumulative time for the whole program, tracemalloc for memory, and in an AI program the answer is almost always the network call, not the loop you were about to rewrite.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"caption":"Rewriting the loop perfectly saves about eight seconds. Caching repeated prompts and overlapping the calls saves twelve minutes. Nobody can tell these apart by reading the code, which is the entire argument for putting a stopwatch around three suspects before changing anything.","series":[{"label":"Waiting for the model API","value":91},{"label":"Parsing the JSON replies","value":4},{"label":"Reading and writing the CSV","value":4},{"highlight":true,"label":"The loop everyone wanted to rewrite","value":1}],"title":"Where fourteen minutes went in a tagging script","type":"bars","unit":"% of total run time"},"alt":"Figure: Where fourteen minutes went in a tagging script. Bars (% of total run time): Waiting for the model API 91; Parsing the JSON replies 4; Reading and writing the CSV 4; The loop everyone wanted to rewrite 1. Rewriting the loop perfectly saves about eight seconds. Caching repeated prompts and overlapping the calls saves twelve minutes. Nobody can tell these apart by reading the code, which is the entire argument for putting a stopwatch around three suspects before changing anything."}},"blocks":[{"t":"h2","id":"guessing-is-wrong-more-often-than-not","runs":[{"text":"Guessing is wrong more often than not"}]},{"t":"p","runs":[{"text":"Every programmer has a story of optimising the wrong thing. The loop that looked slow was 0.3 per cent of the run; the innocent-looking line that formatted a date was called two million times. Intuition about performance is poor even in experienced people, because the cost of an operation is invisible in the source. The remedy is to measure, and Python ships the instruments."}]},{"t":"h2","id":"the-stopwatch","runs":[{"text":"The stopwatch"}]},{"t":"code","lang":"python","label":"Python","text":"import time\n\nt = time.perf_counter()\nvectors = embed(chunks)\nprint(f\"embed: {time.perf_counter() - t:.2f}s\")"},{"t":"p","runs":[{"text":"perf_counter","code":true},{"text":" is a high-resolution clock for measuring intervals. "},{"text":"time.time()","code":true},{"text":" is wall-clock time and can jump when the system clock adjusts; do not use it for durations. The "},{"text":"timed","code":true},{"text":" context manager from earlier in this module is this pattern packaged."}]},{"t":"p","runs":[{"text":"Put stopwatches around the three or four things you suspect, run once, and read the numbers. This alone settles most performance questions in an AI program, and the answer is usually that the network call is 95 per cent of the time and everything you were about to rewrite is noise."}]},{"t":"h2","id":"timeit-for-small-things","runs":[{"text":"timeit","code":true},{"text":" for small things"}]},{"t":"p","runs":[{"text":"For a single expression, the stopwatch is too coarse: one run is dominated by noise. "},{"text":"timeit","code":true},{"text":" runs it many times and reports the best:"}]},{"t":"code","lang":"bash","label":"Bash","text":"python -m timeit -s \"s = 'a' * 1000\" \"s.split()\""},{"t":"code","lang":"python","label":"Python","text":"import timeit\ntimeit.timeit(\"''.join(parts)\", setup=\"parts = ['x'] * 1000\", number=10_000)"},{"t":"p","runs":[{"text":"In a notebook, "},{"text":"%timeit expr","code":true},{"text":" does the same with a friendlier report. Use it to settle “is a comprehension faster than a loop here” questions honestly: usually a little, occasionally not, and rarely enough to matter."}]},{"t":"h2","id":"cprofile-for-the-whole-program","runs":[{"text":"cProfile","code":true},{"text":" for the whole program"}]},{"t":"code","lang":"bash","label":"Bash","text":"python -m cProfile -o run.prof -m tagger.cli input.csv\npython -c \"import pstats; pstats.Stats('run.prof').sort_stats('cumulative').print_stats(20)\""},{"t":"p","runs":[{"text":"The output is a table of every function called: how many times, how long inside it ("},{"text":"tottime","code":true},{"text":"), and how long including everything it called ("},{"text":"cumtime","code":true},{"text":"). Sort by "},{"text":"cumulative","code":true},{"text":" and read from the top: the first lines are your program’s entry points, and a few lines down is the first function that is not merely a wrapper: that is where the time lives. Sort by "},{"text":"tottime","code":true},{"text":" to find the function that is itself expensive rather than expensive because of what it calls."}]},{"t":"p","runs":[{"text":"Two things to know about reading it. The profiler adds overhead, so absolute numbers are inflated; the proportions are what you trust. And "},{"text":"{built-in method ...}","code":true},{"text":" or "},{"text":"{method 'read' of '_ssl._SSLSocket'}","code":true},{"text":" near the top means the time is being spent waiting on the network, which no Python change will fix: only fewer calls, concurrent calls, or caching."}]},{"t":"p","runs":[{"text":"snakeviz run.prof","code":true},{"text":" (pip-installable, free) draws the same data as a clickable sunburst, and is worth the install the first time a profile is more than a screen long."}]},{"t":"h2","id":"the-string-trap-measured","runs":[{"text":"The string trap, measured"}]},{"t":"p","runs":[{"text":"One pure-Python cost that does matter and does show up:"}]},{"t":"code","lang":"python","label":"Python","text":"text = \"\"\nfor part in parts:          # 100,000 parts\n    text += part            # copies the growing string each time"},{"t":"p","runs":[{"text":"Each "},{"text":"+=","code":true},{"text":" builds a new string containing everything so far, so the total work is proportional to the square of the length. "},{"text":"\"\".join(parts)","code":true},{"text":" is one pass. At 100,000 parts the loop can take seconds and the join milliseconds, which is why module 6’s streaming lesson collected fragments in a list. "},{"text":"timeit","code":true},{"text":" both and you will remember it."}]},{"t":"h2","id":"import-time","runs":[{"text":"Import time"}]},{"t":"p","runs":[{"text":"A command that takes two seconds to print "},{"text":"--help","code":true},{"text":" is usually paying for imports. "},{"text":"pandas","code":true},{"text":" alone is around half a second; "},{"text":"torch","code":true},{"text":" several."}]},{"t":"code","lang":"bash","label":"Bash","text":"python -X importtime -c \"import tagger.cli\" 2> imports.log\nsort -t'|' -k2 -n imports.log | tail -20"},{"t":"p","runs":[{"text":"lists the slowest imports. The fix is to import heavy libraries inside the function that needs them, so "},{"text":"--help","code":true},{"text":" and the fast paths do not pay for them."}]},{"t":"h2","id":"memory","runs":[{"text":"Memory"}]},{"t":"code","lang":"python","label":"Python","text":"import tracemalloc\n\ntracemalloc.start()\ndata = load_everything(path)\ncurrent, peak = tracemalloc.get_traced_memory()\nprint(f\"peak {peak / 1e6:.1f} MB\")\ntracemalloc.stop()"},{"t":"p","runs":[{"text":"peak","code":true},{"text":" is the number that matters: it is what the machine had to have. For a closer look, "},{"text":"tracemalloc.take_snapshot().statistics(\"lineno\")[:10]","code":true},{"text":" shows the ten lines that allocated the most. When a process is killed on a small server with no traceback, this is how you find out which structure grew."}]},{"t":"p","runs":[{"text":"sys.getsizeof(obj)","code":true},{"text":" reports one object’s own size and not what it references; a list of a million strings reports the list, not the strings. It is fine for “how big is this array” and misleading for containers."}]},{"t":"h2","id":"the-order-of-operations","runs":[{"text":"The order of operations"}]},{"t":"ol","start":1,"items":[[{"text":"Measure with a stopwatch. Find the 80 per cent."}],[{"text":"If it is the network: fewer calls (cache, batch), overlapped calls (module 6), or a smaller model. Nothing else helps."}],[{"text":"If it is your Python: profile, find the top function, and look for a better algorithm or a NumPy replacement before micro-optimising syntax."}],[{"text":"Measure again. Keep the change only if the number moved."}]]},{"t":"p","runs":[{"text":"Step four is the one people skip. A change that made the code harder to read and the run two seconds faster out of fourteen minutes should be reverted, and only the number tells you that."}]},{"t":"h2","id":"try-this-now","runs":[{"text":"Try this now"}],"role":"try"},{"t":"p","runs":[{"text":"Profile your tagging script with "},{"text":"cProfile","code":true},{"text":", sort by cumulative, and write down what fraction of the total is inside the HTTP library. Then "},{"text":"timeit","code":true},{"text":" string concatenation against "},{"text":"join","code":true},{"text":" at 10,000 parts, and "},{"text":"-X importtime","code":true},{"text":" your CLI to see what "},{"text":"--help","code":true},{"text":" is paying for."}]}],"check":{"question":[{"text":"A script that tags 2,000 messages takes 14 minutes. The developer spends an afternoon replacing a list-building loop with a comprehension and a dict lookup with a set, and it now takes 13 minutes 58 seconds. A cProfile run sorted by cumulative time would most likely have shown what?"}],"options":[{"text":[{"text":"That the comprehension was already the fastest possible form and the set was slower than the dict for small collections, so the two changes cancelled each other out almost exactly."}],"correct":false,"tempting":[{"text":"Sets and comprehensions are not slower; the two-second change shows those parts were never a meaningful share of the time."}]},{"text":[{"text":"That the model API call took nearly all of the 14 minutes, so no change to the pure-Python parts could matter by more than seconds."}],"correct":true,"tempting":null},{"text":[{"text":"That garbage collection was consuming most of the run, and "},{"text":"gc.disable()","code":true},{"text":" at the top would have fixed it."}],"correct":false,"tempting":[{"text":"Garbage collection rarely dominates, and disabling it in a long-running job trades a small saving for growing memory."}]},{"text":[{"text":"That "},{"text":"print","code":true},{"text":" statements were the bottleneck, since terminal output is synchronous."}],"correct":false,"tempting":[{"text":"Printing 2,000 lines costs well under a second; it cannot account for minutes."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"packaging-with-pyproject","title":"pyproject.toml: making your project installable, and a command somebody can type","minutes":9},"next":{"slug":"numpy-arrays","title":"NumPy arrays: why one line beats a loop by a hundred times","minutes":9}}}