{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"logging-without-leaking","title":"Logging without building a liability","url":"https://addaly.com/learn/running-models-yourself/logging-without-leaking","position":61,"of":82,"minutes":9,"answer":"Self-hosting moves prompt-retention risk from a vendor's policy to yours, so log operations and accounting without content, make content logging opt-in and short-lived.","keyIdea":"Self-hosting moves prompt-retention risk from a vendor's policy to yours, so log operations and accounting without content, make content logging opt-in and short-lived, and verify what is actually stored by sending a canary string and grepping every path the request touched."},"body":{"format":"markdown","text":"## The reason people ran it locally in the first place\n\nMost teams that self-host say the same thing: the data cannot leave. Then they enable request logging to debug a problem, and within a month there is a file on disk containing every prompt anyone sent, including the medical history, the unreleased contract and the customer's address. That file is now the most sensitive object in the building, it is on a machine hardened for compute rather than for storage, and nobody has decided how long it lives.\n\nThe honest framing: **self-hosting moves the risk from a vendor's retention policy to your retention policy.** It does not remove it. The gain is real only if your policy is actually better, and a policy you never wrote is not better.\n\n## Decide what each log is for\n\nSeparate the questions, because they need different data and different lifetimes.\n\n**Operations**: is it up, is it fast, is it full? Needs timestamp, model, token counts, latency, status code, key identifier. Needs no prompt text whatsoever. Keep for weeks; it is what you look at during an incident.\n\n**Accounting**: who spent what? Needs key identifier, token counts, timestamp. Aggregate it daily and discard the per-request rows once aggregated.\n\n**Quality**: why was that answer wrong? This is the only one that needs content, and it is the one to make deliberately narrow. Log content only when a caller opts in with a flag, or only for requests a user explicitly reported, and never by default.\n\nMost debugging that people imagine needs prompts actually needs token counts and latencies. A truncation bug shows up as a prompt token count that stops rising at a suspicious number. A template bug shows up in the rendered prompt, which you can reproduce on demand from a test input rather than harvest from users.\n\n## The concrete settings\n\nvLLM logs prompts at debug level and a request summary at info. Run at info, and explicitly avoid `--disable-log-requests` being your only control: check what your version prints by sending one request with an obvious string in it and grepping the log for that string. Do that check after every upgrade, because the default has changed across releases.\n\nLiteLLM stores request and response bodies if you enable a database or a callback, and not otherwise. Turn on `turn_off_message_logging` when you want the spend tracking without the content. Ollama's server log records model, duration and token counts, not prompt text, but the interface in front of it almost certainly keeps a full conversation history in its own database, and that database is the thing people forget. Open WebUI, for instance, stores every conversation in SQLite or Postgres by design, because it is a chat application.\n\n```bash\n# find out what you are actually storing, rather than what you believe\ngrep -ril 'canary-string-9471' /var/log /var/lib /srv 2>/dev/null\n```\n\nSend a request containing that canary, then run the grep. It finds the copies you did not know about: the engine log, the gateway database, the interface's history, a stray `nohup.out`. This takes two minutes and it is the only reliable way to know.\n\n## Redaction is weaker than not collecting\n\nIf you must keep content, redact before it is written, not afterwards. A pattern-based scrubber catches card numbers, email addresses and national ID formats reliably enough, and catches free-text disclosure not at all: \"my manager Sarah in the Leeds office has been off sick since March\" has no pattern. Treat redaction as reducing volume, never as making a log safe to hand around.\n\nHashing a user identifier is useful (you can still group a person's requests without storing who they are) but a hash of a short, guessable value is reversible by brute force in seconds. Use a keyed hash and keep the key somewhere the log does not live.\n\n## Retention, and the law you cannot resolve here\n\nSet a lifetime and enforce it mechanically, because a rule that depends on somebody remembering will fail. `logrotate` with `rotate 7 daily compress` is two lines. For a database, a nightly `DELETE FROM logs WHERE created_at < now() - interval '30 days'` is enough, and a scheduled job that never runs is the usual failure, so check it deletes something.\n\nThe legal position differs by country and is unsettled in places. Under the GDPR the operator of the machine is the controller and prompt logs containing personal data attract subject-access and erasure rights; India's DPDP Act creates a broadly similar duty with its own timelines; other jurisdictions have sector rules, and some have almost nothing. Whether a model's stored conversation history counts as a record you must produce on request has not been comprehensively settled anywhere. This course cannot tell you your obligations and is not legal advice. What it can tell you is the engineering fact underneath: you can only honour a deletion request for data you can find, which is an argument for the canary grep above and for keeping the number of places content lands as small as possible.\n\n> A log you cannot enumerate is a log you cannot delete, and a log you cannot delete is a promise you cannot keep."},"check":{"question":"A team self-hosts specifically so that sensitive prompts never leave the building, and runs a chat interface in front of the engine. Which claim about their privacy position is correct?","options":[{"text":"Because the engine's own log records only model names and token counts, no prompt content is being retained.","correct":false,"tempting":"Stops at the engine; a chat interface keeps conversation history in its own database by design, which is usually the largest store of content in the deployment."},{"text":"Local inference means no personal data is processed, so retention duties do not arise.","correct":false,"tempting":"Confuses transmission with processing; storing prompts that contain personal data is processing wherever the machine sits."},{"text":"Pattern-based redaction applied to the stored history is sufficient to make the logs safe to share widely.","correct":false,"tempting":"Redaction catches structured identifiers and misses free-text disclosure entirely, so it reduces volume without making a log safe to circulate."},{"text":"The risk moved rather than disappeared: content now sits in stores they control and must be enumerated.","correct":true}]},"prev":{"slug":"running-it-as-a-service","title":"Running it as a service that survives a reboot","url":"https://addaly.com/learn/running-models-yourself/running-it-as-a-service"},"next":{"slug":"guardrails-on-your-own-endpoint","title":"You are now the moderation team","url":"https://addaly.com/learn/running-models-yourself/guardrails-on-your-own-endpoint"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Logging without building a liability', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/logging-without-leaking (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Logging without building a liability","courseTitle":"Running Models Yourself","module":{"index":7,"slug":"serving-it-to-others","title":"Serving it to other people"},"answer":null,"keyIdea":"Self-hosting moves prompt-retention risk from a vendor’s policy to yours, so log operations and accounting without content, make content logging opt-in and short-lived, and verify what is actually stored by sending a canary string and grepping every path the request touched.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-reason-people-ran-it-locally-in-the-first-place","runs":[{"text":"The reason people ran it locally in the first place"}]},{"t":"p","runs":[{"text":"Most teams that self-host say the same thing: the data cannot leave. Then they enable request logging to debug a problem, and within a month there is a file on disk containing every prompt anyone sent, including the medical history, the unreleased contract and the customer’s address. That file is now the most sensitive object in the building, it is on a machine hardened for compute rather than for storage, and nobody has decided how long it lives."}]},{"t":"p","runs":[{"text":"The honest framing: "},{"text":"self-hosting moves the risk from a vendor’s retention policy to your retention policy.","b":true},{"text":" It does not remove it. The gain is real only if your policy is actually better, and a policy you never wrote is not better."}]},{"t":"h2","id":"decide-what-each-log-is-for","runs":[{"text":"Decide what each log is for"}]},{"t":"p","runs":[{"text":"Separate the questions, because they need different data and different lifetimes."}]},{"t":"p","runs":[{"text":"Operations","b":true},{"text":": is it up, is it fast, is it full? Needs timestamp, model, token counts, latency, status code, key identifier. Needs no prompt text whatsoever. Keep for weeks; it is what you look at during an incident."}]},{"t":"p","runs":[{"text":"Accounting","b":true},{"text":": who spent what? Needs key identifier, token counts, timestamp. Aggregate it daily and discard the per-request rows once aggregated."}]},{"t":"p","runs":[{"text":"Quality","b":true},{"text":": why was that answer wrong? This is the only one that needs content, and it is the one to make deliberately narrow. Log content only when a caller opts in with a flag, or only for requests a user explicitly reported, and never by default."}]},{"t":"p","runs":[{"text":"Most debugging that people imagine needs prompts actually needs token counts and latencies. A truncation bug shows up as a prompt token count that stops rising at a suspicious number. A template bug shows up in the rendered prompt, which you can reproduce on demand from a test input rather than harvest from users."}]},{"t":"h2","id":"the-concrete-settings","runs":[{"text":"The concrete settings"}]},{"t":"p","runs":[{"text":"vLLM logs prompts at debug level and a request summary at info. Run at info, and explicitly avoid "},{"text":"--disable-log-requests","code":true},{"text":" being your only control: check what your version prints by sending one request with an obvious string in it and grepping the log for that string. Do that check after every upgrade, because the default has changed across releases."}]},{"t":"p","runs":[{"text":"LiteLLM stores request and response bodies if you enable a database or a callback, and not otherwise. Turn on "},{"text":"turn_off_message_logging","code":true},{"text":" when you want the spend tracking without the content. Ollama’s server log records model, duration and token counts, not prompt text, but the interface in front of it almost certainly keeps a full conversation history in its own database, and that database is the thing people forget. Open WebUI, for instance, stores every conversation in SQLite or Postgres by design, because it is a chat application."}]},{"t":"code","lang":"bash","label":"Bash","text":"# find out what you are actually storing, rather than what you believe\ngrep -ril 'canary-string-9471' /var/log /var/lib /srv 2>/dev/null"},{"t":"p","runs":[{"text":"Send a request containing that canary, then run the grep. It finds the copies you did not know about: the engine log, the gateway database, the interface’s history, a stray "},{"text":"nohup.out","code":true},{"text":". This takes two minutes and it is the only reliable way to know."}]},{"t":"h2","id":"redaction-is-weaker-than-not-collecting","runs":[{"text":"Redaction is weaker than not collecting"}]},{"t":"p","runs":[{"text":"If you must keep content, redact before it is written, not afterwards. A pattern-based scrubber catches card numbers, email addresses and national ID formats reliably enough, and catches free-text disclosure not at all: “my manager Sarah in the Leeds office has been off sick since March” has no pattern. Treat redaction as reducing volume, never as making a log safe to hand around."}]},{"t":"p","runs":[{"text":"Hashing a user identifier is useful (you can still group a person’s requests without storing who they are) but a hash of a short, guessable value is reversible by brute force in seconds. Use a keyed hash and keep the key somewhere the log does not live."}]},{"t":"h2","id":"retention-and-the-law-you-cannot-resolve-here","runs":[{"text":"Retention, and the law you cannot resolve here"}]},{"t":"p","runs":[{"text":"Set a lifetime and enforce it mechanically, because a rule that depends on somebody remembering will fail. "},{"text":"logrotate","code":true},{"text":" with "},{"text":"rotate 7 daily compress","code":true},{"text":" is two lines. For a database, a nightly "},{"text":"DELETE FROM logs WHERE created_at < now() - interval '30 days'","code":true},{"text":" is enough, and a scheduled job that never runs is the usual failure, so check it deletes something."}]},{"t":"p","runs":[{"text":"The legal position differs by country and is unsettled in places. Under the GDPR the operator of the machine is the controller and prompt logs containing personal data attract subject-access and erasure rights; India’s DPDP Act creates a broadly similar duty with its own timelines; other jurisdictions have sector rules, and some have almost nothing. Whether a model’s stored conversation history counts as a record you must produce on request has not been comprehensively settled anywhere. This course cannot tell you your obligations and is not legal advice. What it can tell you is the engineering fact underneath: you can only honour a deletion request for data you can find, which is an argument for the canary grep above and for keeping the number of places content lands as small as possible."}]},{"t":"quote","runs":[{"text":"A log you cannot enumerate is a log you cannot delete, and a log you cannot delete is a promise you cannot keep."}]}],"check":{"question":[{"text":"A team self-hosts specifically so that sensitive prompts never leave the building, and runs a chat interface in front of the engine. Which claim about their privacy position is correct?"}],"options":[{"text":[{"text":"Because the engine’s own log records only model names and token counts, no prompt content is being retained."}],"correct":false,"tempting":[{"text":"Stops at the engine; a chat interface keeps conversation history in its own database by design, which is usually the largest store of content in the deployment."}]},{"text":[{"text":"Local inference means no personal data is processed, so retention duties do not arise."}],"correct":false,"tempting":[{"text":"Confuses transmission with processing; storing prompts that contain personal data is processing wherever the machine sits."}]},{"text":[{"text":"Pattern-based redaction applied to the stored history is sufficient to make the logs safe to share widely."}],"correct":false,"tempting":[{"text":"Redaction catches structured identifiers and misses free-text disclosure entirely, so it reduces volume without making a log safe to circulate."}]},{"text":[{"text":"The risk moved rather than disappeared: content now sits in stores they control and must be enumerated."}],"correct":true,"tempting":null}],"order":[1,0,3,2],"why":null},"prev":{"slug":"running-it-as-a-service","title":"Running it as a service that survives a reboot","minutes":8},"next":{"slug":"guardrails-on-your-own-endpoint","title":"You are now the moderation team","minutes":9}}}