{"course":{"slug":"building-apps-with-ai","title":"Building Apps with AI","url":"https://addaly.com/learn/building-apps-with-ai","level":"beginner","track":"build"},"lesson":{"slug":"the-first-call-from-your-own-server","title":"The first call from your own server","url":"https://addaly.com/learn/building-apps-with-ai/the-first-call-from-your-own-server","position":59,"of":87,"minutes":9,"answer":"A model call is an HTTP request that can take twenty seconds, so it needs a timeout, an error path for each status, and streaming before it feels like a feature rather than a hang.","keyIdea":"A model call is an HTTP request that can take twenty seconds, so it needs a timeout, an error path for each status, and streaming before it feels like a feature rather than a hang."},"body":{"format":"markdown","text":"## What a call actually is\n\nEvery major provider exposes roughly the same shape: an HTTP POST with a list of messages, returning a message. Most also accept the format OpenAI defined, which Groq, OpenRouter, Ollama and Gemini's compatibility endpoint all speak, so one piece of code reaches all of them by changing a URL and a key.\n\n```typescript\n// app/api/summarise/route.ts\nexport async function POST(request: Request) {\n  const session = await getSession(request);\n  if (!session) return new Response(\"Unauthorized\", { status: 401 });\n\n  const { text } = await request.json();\n  const controller = new AbortController();\n  const timer = setTimeout(() => controller.abort(), 30_000);\n\n  const res = await fetch(`${process.env.LLM_BASE_URL}/chat/completions`, {\n    method: \"POST\",\n    headers: { Authorization: `Bearer ${process.env.LLM_API_KEY}`, \"Content-Type\": \"application/json\" },\n    body: JSON.stringify({\n      model: process.env.LLM_MODEL,\n      messages: [\n        { role: \"system\", content: SUMMARISE_PROMPT },\n        { role: \"user\", content: text },\n      ],\n      max_tokens: 400,\n    }),\n    signal: controller.signal,\n  });\n  clearTimeout(timer);\n\n  if (!res.ok) return new Response(\"Model unavailable\", { status: 502 });\n  const data = await res.json();\n  return Response.json({ summary: data.choices[0].message.content ?? \"\" });\n}\n```\n\nFour things in that code are not in the generated version, and each is there for a reason.\n\n## The session check\n\nThe route is reachable by anyone who knows its address, which is anyone who opens the Network tab. Without the check, the world has a free model endpoint paid for by you. The check is the first line for the same reason it is the first line of every route in the data block.\n\n## The timeout\n\nA model call takes between one and thirty seconds, depending on the length of the answer. `fetch` has no default timeout; a stalled connection waits forever, and on a serverless host \"forever\" means until the function is killed, which costs you the function's running time and gives the user nothing. Thirty seconds is a reasonable ceiling for a non-streaming call; set it, and handle the abort as an error.\n\n## Reading the response\n\n`data.choices[0].message.content` is where the text lives in this format, and it can be `null` when the model produced nothing or stopped for a safety reason. Code that assumes a string crashes; the `?? \"\"` avoids that, and a later lesson covers what to show the user instead. `data.usage` reports the tokens consumed, which the cost lesson will want you to record.\n\n## The status codes that matter\n\n**401**: the key is wrong or missing. Check the variable name on the host.\n\n**429**: too many requests, or out of credit. On a free tier this is the per-minute limit and it is routine; the resilience lesson handles it.\n\n**400**: the request was malformed; often a `max_tokens` above the model's limit or a message with an empty content.\n\n**500 to 503**: the provider is having a bad minute. Not your bug.\n\n**200 with an empty answer**: the one that looks like success. Treat empty content as a failure, and say so in the log.\n\n## Why it feels broken without streaming\n\nA 400-token answer takes a model ten to twenty seconds to generate. With the code above, the user sees a spinner for all of it and then a wall of text. Most people give up at five seconds, and hosts kill long functions: ten seconds is a common default for the free tier.\n\nStreaming sends each token to the browser as it is generated. Add `stream: true` to the body, and the provider returns a sequence of small events instead of one document; your route passes them through to the browser, which appends them to the page:\n\n```typescript\nconst upstream = await fetch(url, { ...options, body: JSON.stringify({ ...body, stream: true }) });\nreturn new Response(upstream.body, { headers: { \"Content-Type\": \"text/event-stream\" } });\n```\n\nThe first word appears in under a second, the function is sending rather than waiting, and a twenty-second answer feels like reading rather than waiting. Libraries such as the Vercel AI SDK wrap the parsing on both ends in a few lines, for any provider, and are the sensible default once the plain version above has been understood.\n\n## Test it with a real length\n\nThe agent's test will send \"Hello\" and get \"Hello\" back. Send a 3,000-word document, then an empty string, then a document in Hindi, then one with a table. Watch the time, the tokens, and the status code for each. Those four inputs find most of what the first version got wrong, and none of them is the one the agent tried."},"check":{"question":"A summarise route works in testing. In production, users report that long documents show a spinner forever and then nothing. The host's logs show the function was killed at ten seconds. What is the fix?","options":[{"text":"Stream the response so bytes reach the browser as they are generated, and raise the route's timeout, because the host cut off a request that was still waiting for the whole answer.","correct":true},{"text":"Increase the model's temperature so it produces shorter answers and finishes faster.","correct":false,"tempting":"Temperature changes randomness, not length or speed; the function is being killed by the host regardless of the answer's content."},{"text":"Move the call back into the browser, where there is no function timeout to hit.","correct":false,"tempting":"Removes the timeout by putting the key in public, which is the problem the previous lesson exists to prevent."},{"text":"Add an automatic retry so that when the function is killed by the host the request is sent again from the browser, and keep retrying until an answer comes back for the whole document.","correct":false,"tempting":"A retry of a request that takes too long takes too long again; it doubles the cost and never completes."}]},"prev":{"slug":"the-key-never-goes-to-the-browser","title":"The key never goes to the browser","url":"https://addaly.com/learn/building-apps-with-ai/the-key-never-goes-to-the-browser"},"next":{"slug":"the-prompt-is-a-file-not-a-string","title":"The prompt is a file, not a string","url":"https://addaly.com/learn/building-apps-with-ai/the-prompt-is-a-file-not-a-string"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The first call from your own server', in Building Apps with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-apps-with-ai/the-first-call-from-your-own-server (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The first call from your own server","courseTitle":"Building Apps with AI","module":{"index":7,"slug":"a-model-inside-your-app","title":"Putting a model inside your own app"},"answer":null,"keyIdea":"A model call is an HTTP request that can take twenty seconds, so it needs a timeout, an error path for each status, and streaming before it feels like a feature rather than a hang.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-a-call-actually-is","runs":[{"text":"What a call actually is"}]},{"t":"p","runs":[{"text":"Every major provider exposes roughly the same shape: an HTTP POST with a list of messages, returning a message. Most also accept the format OpenAI defined, which Groq, OpenRouter, Ollama and Gemini’s compatibility endpoint all speak, so one piece of code reaches all of them by changing a URL and a key."}]},{"t":"code","lang":"typescript","label":"TypeScript","text":"// app/api/summarise/route.ts\nexport async function POST(request: Request) {\n  const session = await getSession(request);\n  if (!session) return new Response(\"Unauthorized\", { status: 401 });\n\n  const { text } = await request.json();\n  const controller = new AbortController();\n  const timer = setTimeout(() => controller.abort(), 30_000);\n\n  const res = await fetch(`${process.env.LLM_BASE_URL}/chat/completions`, {\n    method: \"POST\",\n    headers: { Authorization: `Bearer ${process.env.LLM_API_KEY}`, \"Content-Type\": \"application/json\" },\n    body: JSON.stringify({\n      model: process.env.LLM_MODEL,\n      messages: [\n        { role: \"system\", content: SUMMARISE_PROMPT },\n        { role: \"user\", content: text },\n      ],\n      max_tokens: 400,\n    }),\n    signal: controller.signal,\n  });\n  clearTimeout(timer);\n\n  if (!res.ok) return new Response(\"Model unavailable\", { status: 502 });\n  const data = await res.json();\n  return Response.json({ summary: data.choices[0].message.content ?? \"\" });\n}"},{"t":"p","runs":[{"text":"Four things in that code are not in the generated version, and each is there for a reason."}]},{"t":"h2","id":"the-session-check","runs":[{"text":"The session check"}]},{"t":"p","runs":[{"text":"The route is reachable by anyone who knows its address, which is anyone who opens the Network tab. Without the check, the world has a free model endpoint paid for by you. The check is the first line for the same reason it is the first line of every route in the data block."}]},{"t":"h2","id":"the-timeout","runs":[{"text":"The timeout"}]},{"t":"p","runs":[{"text":"A model call takes between one and thirty seconds, depending on the length of the answer. "},{"text":"fetch","code":true},{"text":" has no default timeout; a stalled connection waits forever, and on a serverless host “forever” means until the function is killed, which costs you the function’s running time and gives the user nothing. Thirty seconds is a reasonable ceiling for a non-streaming call; set it, and handle the abort as an error."}]},{"t":"h2","id":"reading-the-response","runs":[{"text":"Reading the response"}]},{"t":"p","runs":[{"text":"data.choices[0].message.content","code":true},{"text":" is where the text lives in this format, and it can be "},{"text":"null","code":true},{"text":" when the model produced nothing or stopped for a safety reason. Code that assumes a string crashes; the "},{"text":"?? \"\"","code":true},{"text":" avoids that, and a later lesson covers what to show the user instead. "},{"text":"data.usage","code":true},{"text":" reports the tokens consumed, which the cost lesson will want you to record."}]},{"t":"h2","id":"the-status-codes-that-matter","runs":[{"text":"The status codes that matter"}]},{"t":"p","runs":[{"text":"401","b":true},{"text":": the key is wrong or missing. Check the variable name on the host."}]},{"t":"p","runs":[{"text":"429","b":true},{"text":": too many requests, or out of credit. On a free tier this is the per-minute limit and it is routine; the resilience lesson handles it."}]},{"t":"p","runs":[{"text":"400","b":true},{"text":": the request was malformed; often a "},{"text":"max_tokens","code":true},{"text":" above the model’s limit or a message with an empty content."}]},{"t":"p","runs":[{"text":"500 to 503","b":true},{"text":": the provider is having a bad minute. Not your bug."}]},{"t":"p","runs":[{"text":"200 with an empty answer","b":true},{"text":": the one that looks like success. Treat empty content as a failure, and say so in the log."}]},{"t":"h2","id":"why-it-feels-broken-without-streaming","runs":[{"text":"Why it feels broken without streaming"}]},{"t":"p","runs":[{"text":"A 400-token answer takes a model ten to twenty seconds to generate. With the code above, the user sees a spinner for all of it and then a wall of text. Most people give up at five seconds, and hosts kill long functions: ten seconds is a common default for the free tier."}]},{"t":"p","runs":[{"text":"Streaming sends each token to the browser as it is generated. Add "},{"text":"stream: true","code":true},{"text":" to the body, and the provider returns a sequence of small events instead of one document; your route passes them through to the browser, which appends them to the page:"}]},{"t":"code","lang":"typescript","label":"TypeScript","text":"const upstream = await fetch(url, { ...options, body: JSON.stringify({ ...body, stream: true }) });\nreturn new Response(upstream.body, { headers: { \"Content-Type\": \"text/event-stream\" } });"},{"t":"p","runs":[{"text":"The first word appears in under a second, the function is sending rather than waiting, and a twenty-second answer feels like reading rather than waiting. Libraries such as the Vercel AI SDK wrap the parsing on both ends in a few lines, for any provider, and are the sensible default once the plain version above has been understood."}]},{"t":"h2","id":"test-it-with-a-real-length","runs":[{"text":"Test it with a real length"}]},{"t":"p","runs":[{"text":"The agent’s test will send “Hello” and get “Hello” back. Send a 3,000-word document, then an empty string, then a document in Hindi, then one with a table. Watch the time, the tokens, and the status code for each. Those four inputs find most of what the first version got wrong, and none of them is the one the agent tried."}]}],"check":{"question":[{"text":"A summarise route works in testing. In production, users report that long documents show a spinner forever and then nothing. The host’s logs show the function was killed at ten seconds. What is the fix?"}],"options":[{"text":[{"text":"Stream the response so bytes reach the browser as they are generated, and raise the route’s timeout, because the host cut off a request that was still waiting for the whole answer."}],"correct":true,"tempting":null},{"text":[{"text":"Increase the model’s temperature so it produces shorter answers and finishes faster."}],"correct":false,"tempting":[{"text":"Temperature changes randomness, not length or speed; the function is being killed by the host regardless of the answer’s content."}]},{"text":[{"text":"Move the call back into the browser, where there is no function timeout to hit."}],"correct":false,"tempting":[{"text":"Removes the timeout by putting the key in public, which is the problem the previous lesson exists to prevent."}]},{"text":[{"text":"Add an automatic retry so that when the function is killed by the host the request is sent again from the browser, and keep retrying until an answer comes back for the whole document."}],"correct":false,"tempting":[{"text":"A retry of a request that takes too long takes too long again; it doubles the cost and never completes."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"the-key-never-goes-to-the-browser","title":"The key never goes to the browser","minutes":8},"next":{"slug":"the-prompt-is-a-file-not-a-string","title":"The prompt is a file, not a string","minutes":7}}}