Rock, paper, scissors
On 6 December 2023 Google launched Gemini with a six-minute video, “Hands-on with Gemini”. In one scene a hand makes shapes on a table and the model says, unprompted, that the person is playing rock, paper, scissors. It looks like a live conversation with a machine that watches the world.
Quick check
How was that scene made?
Google’s developer blog, published the same day, shows the real prompt. The model was given three still photographs at once, with the text “What do you think I’m doing? Hint: It’s a game.” The video’s description says latency was reduced and outputs were shortened for brevity. After Parmy Olson reported the gap in Bloomberg Opinion, Google said the prompts and outputs were real and that the video showed what such tools could look like.
The demo still answered a different question from the one a viewer heard.
Possibility, not frequency
A demo shows that something can happen. It never shows how often it happens.
The usual edits are worth naming, because once named you see them in every launch.
Best of many. The output shown is the best of several tries.
The prompt is hidden. A casual request on screen may stand for a long, tuned instruction. The Gemini hint did work the viewer never saw.
Chosen inputs. Clean photographs, good light, a familiar layout.
Waiting cut out. A reply that takes forty seconds feels different in a meeting.
No failures. No demo shows the refusal, the timeout or the confident wrong answer.
One request that tells you most
Ask, politely, to see three failures. A team that has deployed its tool keeps a folder of them: the refusal, the timeout, the confident wrong answer, and what it cost to catch each one. The same conversation usually brings out the rest, such as the success rate on inputs nobody chose and how much human work sits inside the process today.
Running your own trial
How to build a small test set from your own work is in what a benchmark score leaves out. A demo adds one rule to it: fix the sample before you see any results, awkward cases included, and write down in advance what success means. Have the people who will use the tool run it, not its keenest supporter. Then time the whole job, checking included. A tool that drafts in four seconds and takes eleven minutes to check saves nothing if the job took you ten by hand.
Check your understanding
A vendor shows flawless extraction from ten invoices. Which single request tells you most about whether it will work for you?
Pick the one you would defend. Your answer stays private.
Your place and first choices stay on this browser. Classroom credit is recorded separately.
Stuck on something?Ask Rukmini, your AI tutor