{"course":{"slug":"ai-video","title":"Video with AI","url":"https://addaly.com/learn/ai-video","level":"intermediate","track":"create"},"lesson":{"slug":"choosing-a-model","title":"The ranking will be wrong by the time you read this","url":"https://addaly.com/learn/ai-video/choosing-a-model","position":9,"of":66,"minutes":8,"answer":"Benchmark models on one hard shot from your own list, because public rankings measure which clip looks nicer, not whether it did what you asked.","keyIdea":"Benchmark models on one hard shot from your own list, because public rankings measure which clip looks nicer, not whether it did what you asked."},"body":{"format":"markdown","text":"## What is out there, in classes rather than in order\n\nThe specific league table changes every few months. The categories have been stable for a while, and knowing the categories is what transfers.\n\n**Hosted flagships.** Google's Veo line, notable for generating synchronised dialogue and ambience along with the picture. OpenAI's Sora line, app-first, with a consent-gated likeness feature built into the product. Runway, which pairs its generation models with director-style controls: camera moves, a motion brush, performance transfer. Kling from Kuaishou, strong on physical motion and start-and-end frames. Hailuo from MiniMax, cheap and lively. Luma's Dream Machine, with keyframes. Pika, effects-led.\n\n**Aggregators and director layers.** Higgsfield is the clearest consumer example: it resells access to several underlying models and adds camera-motion presets, character tools and a queue on top. fal.ai and Replicate do the same for developers, with an API instead of a UI. What you gain is one balance, one interface and the ability to switch models without opening new accounts. What you pay is a margin on top of the underlying price, and one more step of distance from the model's own controls.\n\n**Open weights you can run.** Wan from Alibaba, HunyuanVideo from Tencent, LTX-Video from Lightricks, CogVideoX, Mochi. Usually driven through ComfyUI. These are behind the hosted flagships on raw quality and they are catching up faster than most people expect.\n\nNames and versions here are as they stood in 2026. Check before you commit money.\n\n## Read a leaderboard for what it actually measures\n\nPublic arenas rank blind pairs: which of these two clips looks better. That is one axis. It is not the same as whether the model did what you asked, whether it holds a face, or how often it hands you something unusable.\n\nA model can win on aesthetics and lose your afternoon on prompt adherence. Those are separate properties and they are not correlated in any dependable way.\n\nDemo reels are worse. A demo reel is the top one per cent of thousands of generations, chosen by people who know the model's strengths and steered away from its weaknesses. It tells you what the ceiling looks like. It tells you nothing about your hit rate.\n\n## How to actually choose, in an hour\n\nTake one hard shot from your own list. Same still, same motion instruction. One generation on each of three candidates. Then answer four questions in writing:\n\n- Did it follow the camera instruction, or invent its own move?\n- Did the face and the wardrobe survive?\n- How bad is the last second?\n- What did it cost?\n\nThat hour beats every comparison video on the internet, because it is measured on your work rather than on somebody else's best case.\n\n## Open weights: what you gain and what you pay\n\nYou gain no per-second bill, seed determinism (same seed and settings, same output, which hosted APIs frequently will not promise) no content filter unexpectedly refusing a legitimate shot, and the ability to train a character model of your own. See [fine-tuning](/learn/fine-tuning) and [running-models-yourself](/learn/running-models-yourself).\n\nYou pay in time and VRAM. A comfortable local setup is a 16 to 24GB card. Quantised builds run on 8 to 12GB, slowly: minutes per clip rather than seconds. On a laptop with integrated graphics this is not happening locally at all, and no amount of patience changes that.\n\nThe free path that works: Hugging Face Spaces host browser demos of most open video models with no install, queueing behind everyone else; Google Colab's free tier and Kaggle notebooks give time-limited GPU sessions you can run ComfyUI or a diffusers script in. On a phone, a Space in a browser tab is the entire workflow, and it works.\n\n## Free tiers on hosted tools\n\nMost hosted platforms give a small allowance, often refreshing daily. Using several of them is legitimate and it is how a large number of people learn this. Two rules make that allowance go further: spend it on image-to-video rather than text-to-video, and spend it on real shots from a real list rather than on tests you will throw away regardless.\n\n## What survives the next release\n\nShot list. First frame. One move per clip. Generate many, discard most. Cut in an editor. Know your price per second. Every one of those transfers intact to a model that does not exist yet, which is why they are what this course is mostly about."},"check":{"question":"A studio switches to the model currently topping the public preference leaderboard and finds it ignores their camera instructions more often than the model they left, even though individual frames look better. What does this most likely reveal?","options":[{"text":"The leaderboard scores which clip looks better in a blind pair, which is a different property from prompt adherence; the two are not reliably correlated.","correct":true},{"text":"Their prompts were written in the previous model's dialect and simply need retranslating for the new one.","correct":false,"tempting":"Assumes a leaderboard measures everything a model is good at, so the fault must lie with the user."},{"text":"The new model defaults to a lower generation tier, and adherence improves on the paid high-quality setting.","correct":false,"tempting":"Reaches for a settings explanation where a difference in what was measured accounts for it."},{"text":"Leaderboard voting is gamed by the vendors, so the ranking does not reflect real quality at all.","correct":false,"tempting":"Substitutes bad faith for a straightforward difference in measurement scope."}]},"prev":{"slug":"frame-rate-and-interpolation","title":"Generated at sixteen, delivered at twenty-four","url":"https://addaly.com/learn/ai-video/frame-rate-and-interpolation"},"next":{"slug":"running-video-models-yourself","title":"What it takes to run one on your own card","url":"https://addaly.com/learn/ai-video/running-video-models-yourself"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The ranking will be wrong by the time you read this', in Video with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-video/choosing-a-model (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The ranking will be wrong by the time you read this","courseTitle":"Video with AI","module":{"index":1,"slug":"how-video-models-work","title":"What the machine is actually doing"},"answer":null,"keyIdea":"Benchmark models on one hard shot from your own list, because public rankings measure which clip looks nicer, not whether it did what you asked.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-is-out-there-in-classes-rather-than-in-order","runs":[{"text":"What is out there, in classes rather than in order"}]},{"t":"p","runs":[{"text":"The specific league table changes every few months. The categories have been stable for a while, and knowing the categories is what transfers."}]},{"t":"p","runs":[{"text":"Hosted flagships.","b":true},{"text":" Google’s Veo line, notable for generating synchronised dialogue and ambience along with the picture. OpenAI’s Sora line, app-first, with a consent-gated likeness feature built into the product. Runway, which pairs its generation models with director-style controls: camera moves, a motion brush, performance transfer. Kling from Kuaishou, strong on physical motion and start-and-end frames. Hailuo from MiniMax, cheap and lively. Luma’s Dream Machine, with keyframes. Pika, effects-led."}]},{"t":"p","runs":[{"text":"Aggregators and director layers.","b":true},{"text":" Higgsfield is the clearest consumer example: it resells access to several underlying models and adds camera-motion presets, character tools and a queue on top. fal.ai and Replicate do the same for developers, with an API instead of a UI. What you gain is one balance, one interface and the ability to switch models without opening new accounts. What you pay is a margin on top of the underlying price, and one more step of distance from the model’s own controls."}]},{"t":"p","runs":[{"text":"Open weights you can run.","b":true},{"text":" Wan from Alibaba, HunyuanVideo from Tencent, LTX-Video from Lightricks, CogVideoX, Mochi. Usually driven through ComfyUI. These are behind the hosted flagships on raw quality and they are catching up faster than most people expect."}]},{"t":"p","runs":[{"text":"Names and versions here are as they stood in 2026. Check before you commit money."}]},{"t":"h2","id":"read-a-leaderboard-for-what-it-actually-measures","runs":[{"text":"Read a leaderboard for what it actually measures"}]},{"t":"p","runs":[{"text":"Public arenas rank blind pairs: which of these two clips looks better. That is one axis. It is not the same as whether the model did what you asked, whether it holds a face, or how often it hands you something unusable."}]},{"t":"p","runs":[{"text":"A model can win on aesthetics and lose your afternoon on prompt adherence. Those are separate properties and they are not correlated in any dependable way."}]},{"t":"p","runs":[{"text":"Demo reels are worse. A demo reel is the top one per cent of thousands of generations, chosen by people who know the model’s strengths and steered away from its weaknesses. It tells you what the ceiling looks like. It tells you nothing about your hit rate."}]},{"t":"h2","id":"how-to-actually-choose-in-an-hour","runs":[{"text":"How to actually choose, in an hour"}]},{"t":"p","runs":[{"text":"Take one hard shot from your own list. Same still, same motion instruction. One generation on each of three candidates. Then answer four questions in writing:"}]},{"t":"ul","items":[[{"text":"Did it follow the camera instruction, or invent its own move?"}],[{"text":"Did the face and the wardrobe survive?"}],[{"text":"How bad is the last second?"}],[{"text":"What did it cost?"}]]},{"t":"p","runs":[{"text":"That hour beats every comparison video on the internet, because it is measured on your work rather than on somebody else’s best case."}]},{"t":"h2","id":"open-weights-what-you-gain-and-what-you-pay","runs":[{"text":"Open weights: what you gain and what you pay"}]},{"t":"p","runs":[{"text":"You gain no per-second bill, seed determinism (same seed and settings, same output, which hosted APIs frequently will not promise) no content filter unexpectedly refusing a legitimate shot, and the ability to train a character model of your own. See "},{"text":"fine-tuning","href":"https://addaly.com/learn/fine-tuning"},{"text":" and "},{"text":"running-models-yourself","href":"https://addaly.com/learn/running-models-yourself"},{"text":"."}]},{"t":"p","runs":[{"text":"You pay in time and VRAM. A comfortable local setup is a 16 to 24GB card. Quantised builds run on 8 to 12GB, slowly: minutes per clip rather than seconds. On a laptop with integrated graphics this is not happening locally at all, and no amount of patience changes that."}]},{"t":"p","runs":[{"text":"The free path that works: Hugging Face Spaces host browser demos of most open video models with no install, queueing behind everyone else; Google Colab’s free tier and Kaggle notebooks give time-limited GPU sessions you can run ComfyUI or a diffusers script in. On a phone, a Space in a browser tab is the entire workflow, and it works."}]},{"t":"h2","id":"free-tiers-on-hosted-tools","runs":[{"text":"Free tiers on hosted tools"}]},{"t":"p","runs":[{"text":"Most hosted platforms give a small allowance, often refreshing daily. Using several of them is legitimate and it is how a large number of people learn this. Two rules make that allowance go further: spend it on image-to-video rather than text-to-video, and spend it on real shots from a real list rather than on tests you will throw away regardless."}]},{"t":"h2","id":"what-survives-the-next-release","runs":[{"text":"What survives the next release"}]},{"t":"p","runs":[{"text":"Shot list. First frame. One move per clip. Generate many, discard most. Cut in an editor. Know your price per second. Every one of those transfers intact to a model that does not exist yet, which is why they are what this course is mostly about."}]}],"check":{"question":[{"text":"A studio switches to the model currently topping the public preference leaderboard and finds it ignores their camera instructions more often than the model they left, even though individual frames look better. What does this most likely reveal?"}],"options":[{"text":[{"text":"The leaderboard scores which clip looks better in a blind pair, which is a different property from prompt adherence; the two are not reliably correlated."}],"correct":true,"tempting":null},{"text":[{"text":"Their prompts were written in the previous model’s dialect and simply need retranslating for the new one."}],"correct":false,"tempting":[{"text":"Assumes a leaderboard measures everything a model is good at, so the fault must lie with the user."}]},{"text":[{"text":"The new model defaults to a lower generation tier, and adherence improves on the paid high-quality setting."}],"correct":false,"tempting":[{"text":"Reaches for a settings explanation where a difference in what was measured accounts for it."}]},{"text":[{"text":"Leaderboard voting is gamed by the vendors, so the ranking does not reflect real quality at all."}],"correct":false,"tempting":[{"text":"Substitutes bad faith for a straightforward difference in measurement scope."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"frame-rate-and-interpolation","title":"Generated at sixteen, delivered at twenty-four","minutes":9},"next":{"slug":"running-video-models-yourself","title":"What it takes to run one on your own card","minutes":10}}}