Video with AI

Rehan and Sneha had shot the venue on the ground twice. They had no drone permission for the site and no budget to hire one, so the plan was a generated reveal: start tight on the marigold arch, lift, and pull back to show the lawn, the lit mandap and the hills behind.

The brief said one continuous take. The venue’s owner was explicit about it. He had seen a competitor’s website with a single flowing shot and he believed cuts would make his lawn look smaller, which is not an unreasonable instinct for a man selling space.

The model gave them five-second clips. They tried the obvious thing first and pressed extend three times.

What came back was a twenty-second file that got worse as it went. The first five seconds were the shot they had paid attention to. By the third extension the marigolds had gone orange-red, the lawn had warmed by what Sneha measured as a noticeable shift on the parade scope, and the fine lattice on the mandap had smoothed into something painted. Nothing in the prompt had changed. Nothing in the settings had changed.

Rehan understood why once he thought about what extend does mechanically. The button takes the last frame of the previous generation, a frame that has already been decoded out of the latent and is therefore already lossy, and re-encodes it as the conditioning image for a fresh run. Three extensions is a photocopy of a photocopy of a photocopy. Contrast climbs, saturation creeps, high-frequency texture dies. There was no prompt that could have prevented it because the damage was not in the prompt.

That left a decision with a cost on each side.

They could deliver the twenty-second take and colour-match the four sections back together in the edit. That is a real technique and it works. But the join sits inside a continuous camera move, which is the worst place for a mismatch to live, because the eye has nothing else to attend to and any residual difference reads as a glitch rather than as a change of shot. Sneha estimated an evening of secondaries, and even then the lattice would still be soft in the last third.

Or they could cut. Four shots, four to five seconds each: the arch, the lift, the lawn, the hills. Each one generated fresh, each one a clean first frame, each one ending before the drift. That would cost them a conversation with a client who had already said no to cuts.

They chose to have the conversation, and they prepared for it rather than arguing. They built both versions. The extended one was ready first. Then they built the cut version and put a single grade and a light grain across all four shots, with a slow push on each so the movement carried through the joins, and laid the venue’s own ambience (recorded on a phone at the site, birds and a generator in the distance) continuously underneath so the audio did not break at the cuts.

Then they sent both, unlabelled as to which was which, and asked the owner which looked more expensive.

The honest part of this story is that they were not certain he would pick theirs. The cost of being wrong was a re-edit they would not be paid for, and a client who felt overruled.

Meena ran the business with her brother. The buyer’s portal wanted a short motion listing for each product line, and they had eleven lines. Her brother had spent a Saturday generating video from text and had eleven clips of babywear that was not theirs.

The garments looked convincing at a glance. They were the right family of colour, the right weight of fabric, the right softness of light. The neck labels were the problem. Their brand mark is a small woven label with three letters and a leaf, and what came back was a small woven label with three letters and a leaf that were not those three letters and not that leaf.

This is the failure that is worse than an obvious one. A garment with no label reads as a stock photograph. A garment with a label that is nearly theirs reads as a counterfeit, which is the single accusation an exporter cannot afford on a buyer’s portal.

Meena’s brother wanted to try again with a better prompt. He had a theory about describing the leaf more precisely. Meena stopped him, for a reason she could state: the model has no representation of their specific label. It was not in the training data and no amount of describing it retrieves information that is not there. More attempts would produce a different wrong label, not the right one.

So the decision was about the division of labour, and both options had a price.

Option one: generate the whole shot and accept the label. Fast, and they could deliver all eleven that evening. It also meant putting something on an international buyer’s portal that misrepresented their own product, which was not a risk about taste.

Option two: change the split. Let the model make the world (the wooden table, the window light, the slow drift of the camera) and put the real garment in themselves. That meant photographing eleven garments properly, cutting them out, and compositing.

They went with the second and did the arithmetic on it first, because three days is not a long time.

The photography took a morning. A sheet of white paper taped to the wall, the garments a metre in front of it so their own shadow did not fall on the background, the window on the left because that was the light direction they had already written at the top of the shot list. Flash off. Twenty frames each, one chosen. A phone.

Cutting out took longer than expected. The one-click background removal handled nine of the eleven and failed on two, both of them knitted pieces with loose fibres at the edge, which is exactly where automatic keying fails. Meena masked those two by hand in Photopea in a browser tab, because the laptop in the office cannot run Photoshop and never could.

Then the composite. Perspective first: the generated table had to be seen from the same height the garment was photographed from, so two of the eleven backgrounds were regenerated rather than distorted, because distortion is precisely what the eye catches. Light direction second, which was already right because they had decided it once. Then the contact shadow under each garment, softened to match the other shadows in the frame, and a curves adjustment to match the black point.

The last step was the one her brother thought was fussy. She added a small amount of grain to each composited garment and softened its cut-out edge by a pixel, because the generated plates are smooth and a real photograph is not, and a laser-sharp edge against a soft plate announces itself.

The lab had no budget line for credits. What it had was a free tier that refreshed each morning, a spreadsheet for booking slots, and a lecturer, Farid, who had watched the same thing happen for two terms.

The pattern was consistent. A student would get a slot, generate a clip, dislike it, rewrite the entire prompt, generate again, and repeat until the allowance was gone. Then they would post the final prompt on the class group as a discovery. The next student would copy it, get something quite different, and conclude that the tool was unreliable.

By the end of term the group chat held about forty prompts, no two of which agreed, and not one of which told anybody anything about the model. Farid described it accurately in a staff meeting as forty superstitions.

The cause was not laziness. Generation is stochastic, and two clips that differ in more than one respect cannot tell you which difference mattered. Every student had been comparing two random draws and reading meaning into the gap. Rewriting four words at once and keeping the better result produces a prompt you cannot explain and cannot repeat.

So at the start of the third term Farid proposed something the students did not like. For the first two weeks, nobody would make anything. The whole allowance would go on a fixed protocol: one still, one seed, one changed variable per generation, a written expectation before each result was viewed, and a shared notes file.

The cost of that was real and the students named it immediately. Two weeks of a fourteen-week term with nothing to show. No clips for their portfolios. Several of them had been waiting since first year to make something and were being told to run experiments instead.

The cost of the alternative was less visible, which is why Farid had to argue for it. Another term of forty superstitions, and every student rediscovering the same facts privately and wrongly.

They compromised on ten days. Six experiments were assigned, one per pair, each on the same supplied still with the seed fixed at a value written on the whiteboard.

The first pair tested motion strength, generating at four values and finding where visible artefacts began. The second tested whether the model would reframe from the supplied still at all when asked for a close-up. The third tested six phrasings of camera moves, one word changed each time, keeping the ones that produced a distinct and correct move. The fourth ran guidance at three values. The fifth tested whether the negative prompt did anything measurable by supplying an absurd one. The sixth found the step count at which improvement stopped being visible.

Each pair wrote one line of findings on the shared file and demonstrated it to the class with two clips side by side rather than describing it.

The experiment that changed the most behaviour was the second one. The model ignored requested shot sizes almost entirely and animated whatever framing the still already had. That single finding deleted a whole category of words from everybody’s prompts and moved shot size out of the text and into the first frame, where it is a fact rather than a request.

The fifth pair found that the negative prompt box did nothing at all on their model, which is what a distilled model at guidance one does. Nine students stopped typing into it.

The pitch was won on a moving storyboard. The production was where the trouble started.

Divya, who was directing, had generated the character once and liked her: a woman of about fifty in a dark green saree behind a counter. She built the rest of the shots by describing her again in each prompt.

By the third shot it was obvious that this was not the same person. Not wildly different (the same build, the same colouring, the same green) but the eye spacing had moved, the jaw was rounder, and a colleague looking at the three stills together said, without being prompted, that the middle one was the shopkeeper’s sister.

Divya already understood why. A reference or a description conditions the model toward a region of its output space. It reproduces a type reliably: build, colouring, wardrobe, the impression somebody would give in one sentence. It does not store the person, so the facial geometry re-samples within that region every time, by a few per cent. A few per cent is nothing on a chair and is the difference between two people on a face.

Then she did the arithmetic that decided the job. Her measured hit rate for a shot that had to hold this face at medium size was about two in three. Six shots that all had to match, at two in three, is a probability of about nine per cent that a first attempt at each lands. She was not going to try for six in a row (nobody does), but the number told her what the sequence would cost in takes, and it was roughly two and a half generations per face shot before anything else went wrong.

There were two routes and the agency spent a morning on the choice.

The first was to train a small character model. Twenty varied images of the shopkeeper, an hour or two of training on a free GPU session, and the base model would then know this specific identity as a concept that can be posed, lit and dressed. It is the strongest identity tool available. It cost them roughly half a day for somebody who had not done it before, a local setup, and a session that might disconnect at eighty per cent. It also raised a question they had not thought about: the character had been generated rather than based on a real person, so there was no consent problem, but Divya wrote that check into the process afterwards anyway, because the next brief might not be so simple.

The second was to need the face less. Build a proper character sheet (six images, one light direction, one wardrobe, front and back) derive every first frame from it, and then redesign the sequence so that the face carried less of the load. An over-the-shoulder for the opening. A wide where the face is forty pixels across. Two cutaways: hands counting notes, and the thing she is looking at. One clean medium shot where the face is genuinely visible and where they would spend the attempts.

The cost of the second route was a conversation with the client about a storyboard that had already been approved with six shots of a face in it.

What made the decision was the number of shots, not the technology. A trained model repays itself when a character appears in more than roughly ten shots, or across several projects, or where revisions are likely. This was six shots in one film.

The unit had forty minutes of archival footage of Dr Kulkarni, a paediatrician who had worked at the hospital for thirty-one years and retired in 2024. The footage was from an old awareness campaign. In the district he is close to a household name, and the unit’s lead, Anagha, knew that a message in his voice would be listened to in a way that a message in hers would not.

The new series needed thirty-one new sentences he had never said. The archival footage contained none of them.

The technical question was trivial. Cloning a recognisable voice from a short clean sample is a solved problem and it is cheap; forty minutes is a great deal more than anyone needs. A colleague had a working clone in an afternoon, and it was good enough that Anagha could not reliably tell it from the original on a laptop speaker.

The question that mattered was whether they were allowed to, and here the unit disagreed with itself for a week.

One argument was that Dr Kulkarni had signed a release for the original campaign. He had agreed to appear, on camera, speaking about immunisation, for the hospital. This was the same hospital, the same subject, and a message he had spent his career delivering.

Anagha’s position was that consent to appear is not consent to be synthesised. He agreed to say particular sentences. He did not agree to have new sentences generated in his voice, and the two are different acts, one of which did not exist as a practical possibility when he signed. Recording somebody for one purpose does not license synthesising them for another.

The cost of her position was not abstract. Dr Kulkarni is elderly and lives with his daughter in Pune. Asking meant a delay of at least two weeks on a series with a district health office deadline. It also meant he might say no, in which case the unit would lose the single strongest asset it had, and the alternative (Anagha reading the lines herself) would be a measurably less persuasive series.

The cost of the other position was that if he objected later, the hospital would be the institution that generated a retired doctor’s voice without asking, on medical advice, in a district where people trust him. No amount of production value survives that.

They asked. The letter said what would be generated, that the sentences would be new ones he had not spoken, which platforms it would appear on, for how long, and that the voice model would be deleted at the end of the campaign.

His reply distinguished two things the unit had been treating as one. He agreed to his voice. He refused any use of his face: he did not want to be shown saying things he had not said, and he was clear that a voice-over is a voice-over while a moving face is a claim about a person being somewhere. He also asked that the videos say plainly that the narration was synthesised.

That last request turned out to be the easiest to honour and the one the unit had been least prepared for.

The centre’s owner had two instructions. The film had to be in 4K, because a rival’s film said 4K on the video player, and it had to be finished by Sunday.

The student editing it, Prakash, had thirty-one generated clips in a folder. They came from two different tools. Nine were 720p at 24 frames per second, nineteen were 1080p at 30, and three were 1080p at 25, which nobody had asked for and which arrived because a free tier had defaulted that way. Several were 5.1 seconds long.

His first attempt was to do exactly what he had been told. He upscaled each clip to 4K individually, dropped them on a timeline, and discovered two things within an hour.

The laptop could not play it back. Not slowly: it stalled, and every trim took three attempts to judge.

And the cuts looked wrong in a way he could not immediately name. Each clip had been upscaled separately, so each carried a slightly different sharpening signature, and at every cut the texture of the picture changed. Some shots also juddered on slow moves, because the 30 and 25 frames-per-second clips were having frames dropped to fit a 24 frames-per-second timeline he had set without thinking about it.

He started again and made two decisions that both had a cost.

The first was to edit on proxies. A loop through FFmpeg turned all thirty-one clips into 640-pixel copies in about four minutes, and the laptop played those without complaint. The cost was that he could not judge colour or fine detail while cutting, and that he would have to relink to the originals before export and hope he had not broken the paths. He kept the originals in one folder and never moved them, which is the whole discipline.

The second was to tell the owner that the film would be 1080p.

That conversation was the difficult one, and Prakash prepared for it the way the venue argument in another case was prepared for: he built evidence rather than an argument. He exported thirty seconds at 4K from upscaled 720p sources, and thirty seconds at 1080p from the originals with one grade and one light grain pass across the whole sequence. He put both on the owner’s phone, which is where parents would watch it.

The 4K version was larger, softer and, on a phone, visibly worse: the upscaler had invented detail that did not survive the platform’s re-encode, and the per-clip sharpening differences were still visible at the cuts.

The rest was conforming. He set the project to 30 frames per second, because the film was for phones and a website rather than for anything that needed to read as film. He converted the 24 and 25 frames-per-second clips deliberately with frame blending rather than letting the timeline drop frames, checking each one: a static shot can lose frames with no cost, a slow pan cannot. He trimmed roughly a quarter-second from the head of every clip and a second from the tail, which removed the settle and the drift and took forty seconds off the running time before he had made a single editorial decision.

Then one shot-match pass to a hero clip, one grade on an adjustment layer, one grain layer over everything set so it was only visible on a paused frame, and an ambience bed recorded on his own phone in the centre’s corridor running unbroken underneath every cut.

The brief arrived as a list of twenty-eight shots and one sentence about the budget. Bhaskar had been freelancing for four years and had done three generative jobs. He had never written a quote that refused anything, and he was about to.

Two of the twenty-eight shots were not achievable and he could say why.

Shot nine was a hero close-up of a named tea brand’s packet, held and turned. The model has no reference for that packet. It would produce a member of the category with a label that is nearly right, which on a government tourism film promoting a specific producer is not a stylistic issue.

Shot twenty-two was a forty-second conversation between two characters across seven cuts, matching eyelines and wardrobe throughout. Seven shots that all had to match, at his measured hit rate of about one in two for a face at medium size, is arithmetic nobody would like: roughly two generations a shot before anything else went wrong, and a real chance that the sequence still would not cut together on light and wardrobe rather than on the faces.

He could have quoted for all twenty-eight and discovered both problems in week four with the deadline visible. Freelancers do this constantly, and the reason is not stupidity: saying no early feels like admitting you cannot do the work, and there is a live fear that the board will simply hire somebody who says yes.

He wrote the quote the other way round and put the two refusals at the top, each with an alternative and a price.

For shot nine: photograph the actual packet against a wall in window light, generate the environment and the camera move, and track the real packet in during the edit. Half a day more than a generated shot, and it survives being looked at.

For shot twenty-two: film it. Two people, one afternoon, a phone on a tripod and a borrowed lapel microphone. Cheaper than generating it, faster, and better.

Then he changed the shape of the rest of the quote. His previous jobs had been priced hourly, which hides the cost driver from both sides: the client asks for a twelfth version of shot nine without knowing that each version costs money, and the freelancer either absorbs it or has an awkward conversation about hours that does not describe what happened.

The new quote priced shots. Twenty-six shots, up to four attempts included per shot, further revisions billed per shot. Included: shot list, animatic, generation, edit, grade, sound, one delivery format. Excluded: music licensing, and any additional aspect ratio. Final generation begins after written approval of the animatic, and structural changes after that point are billed per shot.

He built the number from his own records rather than from a rate card. His cost per eight-second generation, in rupees, multiplied by his measured takes per shot, plus still preparation, plus the edit divided across the shot count, plus overhead.

The risk he was taking was specific. A committee comparing three quotes would see one that delivered twenty-six shots where two others delivered twenty-eight, and committees do not always read the reasons.