{"course":{"slug":"ai-video","title":"Video with AI","url":"https://addaly.com/learn/ai-video","level":"intermediate","track":"create"},"lesson":{"slug":"running-video-models-yourself","title":"What it takes to run one on your own card","url":"https://addaly.com/learn/ai-video/running-video-models-yourself","position":10,"of":66,"minutes":10,"answer":"A video model's VRAM requirement is weights plus activations plus the decode, and quantisation and offloading trade speed for fit.","keyIdea":"A video model's VRAM requirement is weights plus activations plus the decode, and quantisation and offloading trade speed for fit, which is why a 14-billion-parameter model can run on 8GB slowly or not at all on a phone."},"body":{"format":"markdown","text":"## The arithmetic before the download\n\nThree things have to fit in VRAM at once, and people usually count only the first.\n\n1. **Weights.** Parameters times bytes per parameter. A 14-billion-parameter model at bf16 is 2 bytes each: about 28GB. At fp8 it is roughly 14GB. At a 4-bit GGUF quantisation, around 8GB.\n2. **Activations.** The latent block plus every intermediate tensor the attention layers hold during a forward pass. This scales with resolution and duration, and at video sizes it is not small: often several gigabytes.\n3. **The VAE decode.** Expanding the latent back to pixels allocates the full-size frames. Decoding a 1080p clip in one go can spike higher than the whole generation.\n\nSo \"the model is 14GB\" and \"14GB of VRAM is enough\" are different claims, and the second one is usually false.\n\n> Figure: Three things that have to fit at once. Steps: 1. Weights (A 14-billion-parameter model is about 28GB at bf16, roughly 14GB at fp8 and around 8GB at a 4-bit quantisation. Quantise the transformer; leave the text encoder and the VAE alone if you can afford to.); 2. Activations (The latent block plus every intermediate tensor the attention layers hold during a forward pass. Several gigabytes at video sizes, and it grows with resolution and duration. Dropping 720p to 480p cuts it by more than half.); 3. The VAE decode (Expanding the latent back to pixels allocates full-size frames, and decoding a 1080p clip in one go can spike higher than the generation did. A tiled decode removes almost all of that spike.). People count the first and are surprised by the other two. Leave three or four gigabytes of headroom over the weights, or the run dies at the decode after you have paid for the whole denoise.\n\n## The four things that make it fit\n\n- **Quantisation.** fp8 roughly halves the weights against bf16, with a small quality cost. 4-bit GGUF halves again, with a larger one: motion tends to soften and fine detail degrades before anything looks obviously broken. Quantise the transformer; leave the text encoder and VAE alone if you can afford to.\n- **Offloading.** Keep most weights in system RAM and stream the needed blocks to the GPU. ComfyUI's block-swap and the `--lowvram` family of flags do this. It works and it is slow, because you are moving gigabytes across PCIe every step.\n- **Tiled VAE decode.** Decode the output in overlapping tiles instead of all at once. Removes the decode spike almost entirely, at the cost of faint seams if the tiles are too small.\n- **Smaller outputs.** 480p instead of 720p cuts activation memory by more than half. Generate drafts small. This is the cheapest lever and the one people reach for last.\n\n## What runs on what\n\nRough shape as of 2026, and it moves:\n\n- **24GB (e.g. a 3090 or 4090).** Comfortable. A 14B model at fp8, 720p, five seconds, in roughly one to three minutes.\n- **16GB.** Workable with fp8 and tiled decode. Expect a few minutes per clip.\n- **12GB.** Quantised models at 480p, with offloading. Several minutes per clip. Usable for learning and draft passes, painful for production.\n- **8GB.** 4-bit quantisations, low resolution, heavy offload. Minutes per clip, frequent out-of-memory failures. Possible, not pleasant.\n- **Apple Silicon.** Unified memory helps with fit; MPS support in video pipelines is behind CUDA and considerably slower. A 32GB or 64GB Mac will run things a 12GB PC cannot, and will take longer doing it.\n- **Integrated graphics, or a phone.** No. Not slowly: not at all. This is not pessimism, it is the memory arithmetic.\n\n## The free GPU paths that actually work\n\nIf you do not own a card, three routes are real:\n\n1. **Hugging Face Spaces.** Browser demos of most open video models, no install, no account needed for many. You queue behind everyone else and you cannot change much. On a phone this is the entire workflow, and it works.\n2. **Google Colab free tier.** A T4 with around 15GB, session limits, and disconnections. Enough for quantised models at low resolution. ComfyUI runs in Colab through community notebooks.\n3. **Kaggle notebooks.** A weekly GPU quota, typically around 30 hours, on T4s. More generous session length than free Colab and less prone to mid-render disconnection, which matters when a render takes four minutes.\n\nThe honest cost of all three is queue time and the risk of losing a session at eighty per cent. Use them to learn and to make your own work. Do not promise a client a deadline that depends on somebody else's free queue.\n\n## Why you would bother at all\n\nRunning locally is slower and more fiddly than a hosted endpoint. The reasons people do it anyway are specific:\n\n- **No per-second meter.** Experimentation becomes free, which changes how much you experiment.\n- **Real determinism.** Same graph, same seed, same output, which hosted APIs will not promise.\n- **No unexpected refusal.** Content filters on hosted services occasionally decline legitimate work with no explanation and no appeal.\n- **You can train.** A character LoRA needs the weights. That is covered later in the course, and it is the strongest identity tool available.\n- **The model does not change under you.** A checkpoint on your disk is the same checkpoint next year.\n\n## The setup, in one paragraph\n\nComfyUI is the practical front end: a node graph, an enormous community library, and workflows shared as JSON files you can drop in. Install it, download a model in the format your VRAM allows, load a known-good workflow from the model's own page rather than building one from scratch, and change one node at a time. Everybody's first week is spent on dependency versions and out-of-memory errors; that is normal and it ends.\n\nMore on the general mechanics in [running-models-yourself](/learn/running-models-yourself).\n\n---\n\nToday: look up your GPU's VRAM, then check the model card for the quantisation that fits in it with three or four gigabytes to spare for activations. If nothing fits, open a Hugging Face Space instead and lose nothing."},"check":{"question":"Someone with a 16GB card downloads a 14-billion-parameter video model quantised to fp8, reasoning that 14GB of weights leaves 2GB spare. The first generation fails with an out-of-memory error during the final stage. What did the estimate miss?","options":[{"text":"fp8 weights are unpacked to bf16 on load, so the model actually occupies 28GB during inference.","correct":false,"tempting":"Describes a dequantisation that some runtimes do per-layer but not for the whole model at once, which would make fp8 pointless."},{"text":"Video models reserve a fixed fraction of VRAM for the text encoder, which must stay resident throughout generation.","correct":false,"tempting":"Overstates the text encoder, which is comparatively small and can be unloaded after the prompt is encoded."},{"text":"The card cannot allocate its full 16GB because the display driver and the operating system reserve part of it, leaving roughly 13GB usable.","correct":false,"tempting":"Names a real overhead of a few hundred megabytes and inflates it into the whole explanation."},{"text":"Activations during attention and the full-size VAE decode need VRAM beyond the weights, and the decode is the largest single spike.","correct":true}]},"prev":{"slug":"choosing-a-model","title":"The ranking will be wrong by the time you read this","url":"https://addaly.com/learn/ai-video/choosing-a-model"},"next":{"slug":"start-from-a-still","title":"The still frame is where the decisions belong","url":"https://addaly.com/learn/ai-video/start-from-a-still"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What it takes to run one on your own card', in Video with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-video/running-video-models-yourself (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What it takes to run one on your own card","courseTitle":"Video with AI","module":{"index":1,"slug":"how-video-models-work","title":"What the machine is actually doing"},"answer":null,"keyIdea":"A video model’s VRAM requirement is weights plus activations plus the decode, and quantisation and offloading trade speed for fit, which is why a 14-billion-parameter model can run on 8GB slowly or not at all on a phone.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"Weights","detail":"A 14-billion-parameter model is about 28GB at bf16, roughly 14GB at fp8 and around 8GB at a 4-bit quantisation. Quantise the transformer; leave the text encoder and the VAE alone if you can afford to."},{"label":"Activations","detail":"The latent block plus every intermediate tensor the attention layers hold during a forward pass. Several gigabytes at video sizes, and it grows with resolution and duration. Dropping 720p to 480p cuts it by more than half."},{"label":"The VAE decode","detail":"Expanding the latent back to pixels allocates full-size frames, and decoding a 1080p clip in one go can spike higher than the generation did. A tiled decode removes almost all of that spike."}],"title":"Three things that have to fit at once","caption":"People count the first and are surprised by the other two. Leave three or four gigabytes of headroom over the weights, or the run dies at the decode after you have paid for the whole denoise."},"alt":"Figure: Three things that have to fit at once. Steps: 1. Weights (A 14-billion-parameter model is about 28GB at bf16, roughly 14GB at fp8 and around 8GB at a 4-bit quantisation. Quantise the transformer; leave the text encoder and the VAE alone if you can afford to.); 2. Activations (The latent block plus every intermediate tensor the attention layers hold during a forward pass. Several gigabytes at video sizes, and it grows with resolution and duration. Dropping 720p to 480p cuts it by more than half.); 3. The VAE decode (Expanding the latent back to pixels allocates full-size frames, and decoding a 1080p clip in one go can spike higher than the generation did. A tiled decode removes almost all of that spike.). People count the first and are surprised by the other two. Leave three or four gigabytes of headroom over the weights, or the run dies at the decode after you have paid for the whole denoise."}},"blocks":[{"t":"h2","id":"the-arithmetic-before-the-download","runs":[{"text":"The arithmetic before the download"}]},{"t":"p","runs":[{"text":"Three things have to fit in VRAM at once, and people usually count only the first."}]},{"t":"ol","start":1,"items":[[{"text":"Weights.","b":true},{"text":" Parameters times bytes per parameter. A 14-billion-parameter model at bf16 is 2 bytes each: about 28GB. At fp8 it is roughly 14GB. At a 4-bit GGUF quantisation, around 8GB."}],[{"text":"Activations.","b":true},{"text":" The latent block plus every intermediate tensor the attention layers hold during a forward pass. This scales with resolution and duration, and at video sizes it is not small: often several gigabytes."}],[{"text":"The VAE decode.","b":true},{"text":" Expanding the latent back to pixels allocates the full-size frames. Decoding a 1080p clip in one go can spike higher than the whole generation."}]]},{"t":"p","runs":[{"text":"So “the model is 14GB” and “14GB of VRAM is enough” are different claims, and the second one is usually false."}]},{"t":"h2","id":"the-four-things-that-make-it-fit","runs":[{"text":"The four things that make it fit"}]},{"t":"ul","items":[[{"text":"Quantisation.","b":true},{"text":" fp8 roughly halves the weights against bf16, with a small quality cost. 4-bit GGUF halves again, with a larger one: motion tends to soften and fine detail degrades before anything looks obviously broken. Quantise the transformer; leave the text encoder and VAE alone if you can afford to."}],[{"text":"Offloading.","b":true},{"text":" Keep most weights in system RAM and stream the needed blocks to the GPU. ComfyUI’s block-swap and the "},{"text":"--lowvram","code":true},{"text":" family of flags do this. It works and it is slow, because you are moving gigabytes across PCIe every step."}],[{"text":"Tiled VAE decode.","b":true},{"text":" Decode the output in overlapping tiles instead of all at once. Removes the decode spike almost entirely, at the cost of faint seams if the tiles are too small."}],[{"text":"Smaller outputs.","b":true},{"text":" 480p instead of 720p cuts activation memory by more than half. Generate drafts small. This is the cheapest lever and the one people reach for last."}]]},{"t":"h2","id":"what-runs-on-what","runs":[{"text":"What runs on what"}]},{"t":"p","runs":[{"text":"Rough shape as of 2026, and it moves:"}]},{"t":"ul","items":[[{"text":"24GB (e.g. a 3090 or 4090).","b":true},{"text":" Comfortable. A 14B model at fp8, 720p, five seconds, in roughly one to three minutes."}],[{"text":"16GB.","b":true},{"text":" Workable with fp8 and tiled decode. Expect a few minutes per clip."}],[{"text":"12GB.","b":true},{"text":" Quantised models at 480p, with offloading. Several minutes per clip. Usable for learning and draft passes, painful for production."}],[{"text":"8GB.","b":true},{"text":" 4-bit quantisations, low resolution, heavy offload. Minutes per clip, frequent out-of-memory failures. Possible, not pleasant."}],[{"text":"Apple Silicon.","b":true},{"text":" Unified memory helps with fit; MPS support in video pipelines is behind CUDA and considerably slower. A 32GB or 64GB Mac will run things a 12GB PC cannot, and will take longer doing it."}],[{"text":"Integrated graphics, or a phone.","b":true},{"text":" No. Not slowly: not at all. This is not pessimism, it is the memory arithmetic."}]]},{"t":"h2","id":"the-free-gpu-paths-that-actually-work","runs":[{"text":"The free GPU paths that actually work"}]},{"t":"p","runs":[{"text":"If you do not own a card, three routes are real:"}]},{"t":"ol","start":1,"items":[[{"text":"Hugging Face Spaces.","b":true},{"text":" Browser demos of most open video models, no install, no account needed for many. You queue behind everyone else and you cannot change much. On a phone this is the entire workflow, and it works."}],[{"text":"Google Colab free tier.","b":true},{"text":" A T4 with around 15GB, session limits, and disconnections. Enough for quantised models at low resolution. ComfyUI runs in Colab through community notebooks."}],[{"text":"Kaggle notebooks.","b":true},{"text":" A weekly GPU quota, typically around 30 hours, on T4s. More generous session length than free Colab and less prone to mid-render disconnection, which matters when a render takes four minutes."}]]},{"t":"p","runs":[{"text":"The honest cost of all three is queue time and the risk of losing a session at eighty per cent. Use them to learn and to make your own work. Do not promise a client a deadline that depends on somebody else’s free queue."}]},{"t":"h2","id":"why-you-would-bother-at-all","runs":[{"text":"Why you would bother at all"}]},{"t":"p","runs":[{"text":"Running locally is slower and more fiddly than a hosted endpoint. The reasons people do it anyway are specific:"}]},{"t":"ul","items":[[{"text":"No per-second meter.","b":true},{"text":" Experimentation becomes free, which changes how much you experiment."}],[{"text":"Real determinism.","b":true},{"text":" Same graph, same seed, same output, which hosted APIs will not promise."}],[{"text":"No unexpected refusal.","b":true},{"text":" Content filters on hosted services occasionally decline legitimate work with no explanation and no appeal."}],[{"text":"You can train.","b":true},{"text":" A character LoRA needs the weights. That is covered later in the course, and it is the strongest identity tool available."}],[{"text":"The model does not change under you.","b":true},{"text":" A checkpoint on your disk is the same checkpoint next year."}]]},{"t":"h2","id":"the-setup-in-one-paragraph","runs":[{"text":"The setup, in one paragraph"}]},{"t":"p","runs":[{"text":"ComfyUI is the practical front end: a node graph, an enormous community library, and workflows shared as JSON files you can drop in. Install it, download a model in the format your VRAM allows, load a known-good workflow from the model’s own page rather than building one from scratch, and change one node at a time. Everybody’s first week is spent on dependency versions and out-of-memory errors; that is normal and it ends."}]},{"t":"p","runs":[{"text":"More on the general mechanics in "},{"text":"running-models-yourself","href":"https://addaly.com/learn/running-models-yourself"},{"text":"."}]},{"t":"rule"},{"t":"p","runs":[{"text":"Today: look up your GPU’s VRAM, then check the model card for the quantisation that fits in it with three or four gigabytes to spare for activations. If nothing fits, open a Hugging Face Space instead and lose nothing."}]}],"check":{"question":[{"text":"Someone with a 16GB card downloads a 14-billion-parameter video model quantised to fp8, reasoning that 14GB of weights leaves 2GB spare. The first generation fails with an out-of-memory error during the final stage. What did the estimate miss?"}],"options":[{"text":[{"text":"fp8 weights are unpacked to bf16 on load, so the model actually occupies 28GB during inference."}],"correct":false,"tempting":[{"text":"Describes a dequantisation that some runtimes do per-layer but not for the whole model at once, which would make fp8 pointless."}]},{"text":[{"text":"Video models reserve a fixed fraction of VRAM for the text encoder, which must stay resident throughout generation."}],"correct":false,"tempting":[{"text":"Overstates the text encoder, which is comparatively small and can be unloaded after the prompt is encoded."}]},{"text":[{"text":"The card cannot allocate its full 16GB because the display driver and the operating system reserve part of it, leaving roughly 13GB usable."}],"correct":false,"tempting":[{"text":"Names a real overhead of a few hundred megabytes and inflates it into the whole explanation."}]},{"text":[{"text":"Activations during attention and the full-size VAE decode need VRAM beyond the weights, and the decode is the largest single spike."}],"correct":true,"tempting":null}],"order":[2,0,1,3],"why":null},"prev":{"slug":"choosing-a-model","title":"The ranking will be wrong by the time you read this","minutes":8},"next":{"slug":"start-from-a-still","title":"The still frame is where the decisions belong","minutes":9}}}