{"course":{"slug":"hugging-face","title":"Hugging Face, End to End","url":"https://addaly.com/learn/hugging-face","level":"intermediate","track":"create"},"lesson":{"slug":"storage-in-a-space","title":"The filesystem disappears, and what to do about it","url":"https://addaly.com/learn/hugging-face/storage-in-a-space","position":34,"of":75,"minutes":8,"answer":"A Space's local filesystem does not survive restarts, so state belongs in gr.State for a session, a paid /data volume for the model cache, or a private Hub dataset repository for anything collected.","keyIdea":"A Space's local filesystem does not survive restarts, so state belongs in `gr.State` for a session, a paid `/data` volume for the model cache, or a private Hub dataset repository for anything collected, and collecting user text at all is a commitment that has to be stated on the page."},"body":{"format":"markdown","text":"## Assume the disk is temporary\n\nA Space runs in a container. Rebuild it, restart it, let it sleep and wake, and anything written to the local filesystem may be gone. People discover this after a week of collected feedback vanishes on a routine restart.\n\nThere are four places to put state, and choosing correctly is most of this lesson.\n\n## 1. Nowhere: the default, and often right\n\nIf your Space transforms input into output and keeps nothing, say so in the readme and move on. \"Nothing you type is stored\" is a feature, it is free, and it removes a whole category of obligation. Most demos should be here.\n\n## 2. `gr.State`: per session, in memory\n\nConversation history, a wizard's current step, an uploaded file being worked on. Dies when the browser session ends, never shared between users. Covered in the Blocks lesson, and it is the right home for most of what people wrongly put on disk.\n\n> Figure: Four places to put state, and what survives a restart. Layers, top to bottom: Nowhere (Nothing is kept, and the readme says so. Right for most demos.); gr.State (One browser session, in memory. Never shared between visitors.); A paid /data volume (Survives restarts. Best spent on the model cache, not used as a database.); A private dataset repository (Free, versioned, durable. CommitScheduler batches the writes.). The container's own filesystem is not on the list, because it does not survive a restart and people discover that after a week of collected feedback disappears. Most demos belong on the first line, and saying so on the page is free.\n\n## 3. Persistent storage: a real disk, paid\n\nSpaces can be given a persistent volume mounted at `/data`, in tiers starting around 20 GB for a few dollars a month. It survives restarts and rebuilds.\n\nUse it for the model cache, which is the highest-value thing you can put there:\n\n```yaml\n---\n# in README.md front matter\nsdk: gradio\n---\n```\n\n```python\nimport os\nos.environ[\"HF_HOME\"] = \"/data/.huggingface\"   # before importing transformers\n```\n\nNow a cold start after sleep loads weights from the volume instead of downloading them again, which can turn a three-minute wake into fifteen seconds. On a Space that sleeps often, this is the best use of a few dollars a month available anywhere on the platform.\n\nIt is still not a database: one volume, one Space, no backups by default, and no concurrency control beyond what you write yourself.\n\n## 4. A Hub dataset repository: free, durable, versioned\n\nFor collecting anything (feedback, submitted examples, evaluation results) a dataset repository is better than a disk. It is free, it is versioned, it is backed up by being a git repository, and it can be private.\n\n```python\nfrom huggingface_hub import CommitScheduler\nimport json, uuid, pathlib\n\nfolder = pathlib.Path(\"feedback\"); folder.mkdir(exist_ok=True)\nfile = folder / f\"{uuid.uuid4()}.jsonl\"\n\nscheduler = CommitScheduler(\n    repo_id=\"your-name/space-feedback\",\n    repo_type=\"dataset\",\n    folder_path=folder,\n    every=5,                # minutes\n    private=True,\n)\n\ndef record(text, verdict):\n    with scheduler.lock:\n        with file.open(\"a\") as f:\n            f.write(json.dumps({\"text\": text, \"verdict\": verdict}) + \"\\n\")\n```\n\n`CommitScheduler` batches writes and pushes them on a timer, so you are not making a git commit per request. The lock matters: without it, concurrent requests interleave partial lines and you get corrupt JSON.\n\n## The part that is not technical\n\nIf you store what users type, you have taken on responsibility for it. Before writing that code:\n\n- **Say so on the page**, in plain words, above the fold. Not in a link.\n- **Make the repository private** unless you have a specific reason and consent for it to be public. A public dataset of things strangers typed into a demo is a privacy incident waiting for a search engine.\n- **Collect the minimum.** The text and a verdict, not IP addresses, not timestamps precise enough to re-identify, not the user agent.\n- **Say how long you keep it**, and then actually delete it.\n- **Do not collect at all** if the Space handles anything sensitive: health, finance, legal, anything about a named person.\n\nRules on personal data differ by country and are unsettled for AI training in several of them. This lesson is not legal advice and cannot be; the defensible engineering position is to collect as little as possible and to tell people exactly what you collect.\n\n## Secrets\n\nSpace settings have a Secrets section. Values set there arrive as environment variables and are not in the repository:\n\n```python\ntoken = os.environ[\"HF_TOKEN\"]\n```\n\nNever commit a token, never put one in a Gradio textbox default, and never print one into a log. Duplicating a Space does **not** copy its secrets: the duplicate prompts for its own, which is correct behaviour and occasionally confusing.\n\n## Do this now\n\nPick one Space you have made and decide, explicitly, which of the four storage answers it uses. Then write one sentence in the readme saying so. Most Spaces should say \"nothing is stored\", and most do not say anything at all."},"check":{"question":"A Space writes user feedback to `feedback.jsonl` in its working directory and loses everything after a rebuild. Which fix addresses the cause rather than the symptom?","options":[{"text":"Write the file to `/tmp`, which is outside the application directory and therefore untouched by rebuilds.","correct":false,"tempting":"Moves the file within the same ephemeral container filesystem."},{"text":"Commit the file to the Space repository after each write, so it is versioned with the code.","correct":false,"tempting":"Would persist the data but makes a git commit per request and publishes user text into the Space's own repository."},{"text":"Push the collected records to a private dataset repository on a scheduled commit.","correct":true},{"text":"Hold the records in a module-level list and write them out when the Space shuts down.","correct":false,"tempting":"Shares data across users and assumes a graceful shutdown that container restarts do not provide."}]},"prev":{"slug":"hardware-sleep-and-zerogpu","title":"Hardware, sleep, and the bill that runs while you are asleep","url":"https://addaly.com/learn/hugging-face/hardware-sleep-and-zerogpu"},"next":{"slug":"beyond-gradio","title":"Streamlit, Docker and a Space with no server at all","url":"https://addaly.com/learn/hugging-face/beyond-gradio"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The filesystem disappears, and what to do about it', in Hugging Face, End to End. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/hugging-face/storage-in-a-space (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The filesystem disappears, and what to do about it","courseTitle":"Hugging Face, End to End","module":{"index":4,"slug":"spaces-and-shipping-something","title":"Spaces: putting something in front of people"},"answer":null,"keyIdea":"A Space’s local filesystem does not survive restarts, so state belongs in `gr.State` for a session, a paid `/data` volume for the model cache, or a private Hub dataset repository for anything collected, and collecting user text at all is a commitment that has to be stated on the page.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"Four places to put state, and what survives a restart","layers":[{"label":"Nowhere","detail":"Nothing is kept, and the readme says so. Right for most demos."},{"label":"gr.State","detail":"One browser session, in memory. Never shared between visitors."},{"label":"A paid /data volume","detail":"Survives restarts. Best spent on the model cache, not used as a database."},{"label":"A private dataset repository","detail":"Free, versioned, durable. CommitScheduler batches the writes."}],"caption":"The container’s own filesystem is not on the list, because it does not survive a restart and people discover that after a week of collected feedback disappears. Most demos belong on the first line, and saying so on the page is free."},"alt":"Figure: Four places to put state, and what survives a restart. Layers, top to bottom: Nowhere (Nothing is kept, and the readme says so. Right for most demos.); gr.State (One browser session, in memory. Never shared between visitors.); A paid /data volume (Survives restarts. Best spent on the model cache, not used as a database.); A private dataset repository (Free, versioned, durable. CommitScheduler batches the writes.). The container’s own filesystem is not on the list, because it does not survive a restart and people discover that after a week of collected feedback disappears. Most demos belong on the first line, and saying so on the page is free."}},"blocks":[{"t":"h2","id":"assume-the-disk-is-temporary","runs":[{"text":"Assume the disk is temporary"}]},{"t":"p","runs":[{"text":"A Space runs in a container. Rebuild it, restart it, let it sleep and wake, and anything written to the local filesystem may be gone. People discover this after a week of collected feedback vanishes on a routine restart."}]},{"t":"p","runs":[{"text":"There are four places to put state, and choosing correctly is most of this lesson."}]},{"t":"h2","id":"1-nowhere-the-default-and-often-right","runs":[{"text":"1. Nowhere: the default, and often right"}]},{"t":"p","runs":[{"text":"If your Space transforms input into output and keeps nothing, say so in the readme and move on. “Nothing you type is stored” is a feature, it is free, and it removes a whole category of obligation. Most demos should be here."}]},{"t":"h2","id":"2-gr-state-per-session-in-memory","runs":[{"text":"2. "},{"text":"gr.State","code":true},{"text":": per session, in memory"}]},{"t":"p","runs":[{"text":"Conversation history, a wizard’s current step, an uploaded file being worked on. Dies when the browser session ends, never shared between users. Covered in the Blocks lesson, and it is the right home for most of what people wrongly put on disk."}]},{"t":"h2","id":"3-persistent-storage-a-real-disk-paid","runs":[{"text":"3. Persistent storage: a real disk, paid"}]},{"t":"p","runs":[{"text":"Spaces can be given a persistent volume mounted at "},{"text":"/data","code":true},{"text":", in tiers starting around 20 GB for a few dollars a month. It survives restarts and rebuilds."}]},{"t":"p","runs":[{"text":"Use it for the model cache, which is the highest-value thing you can put there:"}]},{"t":"code","lang":"yaml","label":"YAML","text":"---\n# in README.md front matter\nsdk: gradio\n---"},{"t":"code","lang":"python","label":"Python","text":"import os\nos.environ[\"HF_HOME\"] = \"/data/.huggingface\"   # before importing transformers"},{"t":"p","runs":[{"text":"Now a cold start after sleep loads weights from the volume instead of downloading them again, which can turn a three-minute wake into fifteen seconds. On a Space that sleeps often, this is the best use of a few dollars a month available anywhere on the platform."}]},{"t":"p","runs":[{"text":"It is still not a database: one volume, one Space, no backups by default, and no concurrency control beyond what you write yourself."}]},{"t":"h2","id":"4-a-hub-dataset-repository-free-durable-versioned","runs":[{"text":"4. A Hub dataset repository: free, durable, versioned"}]},{"t":"p","runs":[{"text":"For collecting anything (feedback, submitted examples, evaluation results) a dataset repository is better than a disk. It is free, it is versioned, it is backed up by being a git repository, and it can be private."}]},{"t":"code","lang":"python","label":"Python","text":"from huggingface_hub import CommitScheduler\nimport json, uuid, pathlib\n\nfolder = pathlib.Path(\"feedback\"); folder.mkdir(exist_ok=True)\nfile = folder / f\"{uuid.uuid4()}.jsonl\"\n\nscheduler = CommitScheduler(\n    repo_id=\"your-name/space-feedback\",\n    repo_type=\"dataset\",\n    folder_path=folder,\n    every=5,                # minutes\n    private=True,\n)\n\ndef record(text, verdict):\n    with scheduler.lock:\n        with file.open(\"a\") as f:\n            f.write(json.dumps({\"text\": text, \"verdict\": verdict}) + \"\\n\")"},{"t":"p","runs":[{"text":"CommitScheduler","code":true},{"text":" batches writes and pushes them on a timer, so you are not making a git commit per request. The lock matters: without it, concurrent requests interleave partial lines and you get corrupt JSON."}]},{"t":"h2","id":"the-part-that-is-not-technical","runs":[{"text":"The part that is not technical"}]},{"t":"p","runs":[{"text":"If you store what users type, you have taken on responsibility for it. Before writing that code:"}]},{"t":"ul","items":[[{"text":"Say so on the page","b":true},{"text":", in plain words, above the fold. Not in a link."}],[{"text":"Make the repository private","b":true},{"text":" unless you have a specific reason and consent for it to be public. A public dataset of things strangers typed into a demo is a privacy incident waiting for a search engine."}],[{"text":"Collect the minimum.","b":true},{"text":" The text and a verdict, not IP addresses, not timestamps precise enough to re-identify, not the user agent."}],[{"text":"Say how long you keep it","b":true},{"text":", and then actually delete it."}],[{"text":"Do not collect at all","b":true},{"text":" if the Space handles anything sensitive: health, finance, legal, anything about a named person."}]]},{"t":"p","runs":[{"text":"Rules on personal data differ by country and are unsettled for AI training in several of them. This lesson is not legal advice and cannot be; the defensible engineering position is to collect as little as possible and to tell people exactly what you collect."}]},{"t":"h2","id":"secrets","runs":[{"text":"Secrets"}]},{"t":"p","runs":[{"text":"Space settings have a Secrets section. Values set there arrive as environment variables and are not in the repository:"}]},{"t":"code","lang":"python","label":"Python","text":"token = os.environ[\"HF_TOKEN\"]"},{"t":"p","runs":[{"text":"Never commit a token, never put one in a Gradio textbox default, and never print one into a log. Duplicating a Space does "},{"text":"not","b":true},{"text":" copy its secrets: the duplicate prompts for its own, which is correct behaviour and occasionally confusing."}]},{"t":"h2","id":"do-this-now","runs":[{"text":"Do this now"}],"role":"try"},{"t":"p","runs":[{"text":"Pick one Space you have made and decide, explicitly, which of the four storage answers it uses. Then write one sentence in the readme saying so. Most Spaces should say “nothing is stored”, and most do not say anything at all."}]}],"check":{"question":[{"text":"A Space writes user feedback to "},{"text":"feedback.jsonl","code":true},{"text":" in its working directory and loses everything after a rebuild. Which fix addresses the cause rather than the symptom?"}],"options":[{"text":[{"text":"Write the file to "},{"text":"/tmp","code":true},{"text":", which is outside the application directory and therefore untouched by rebuilds."}],"correct":false,"tempting":[{"text":"Moves the file within the same ephemeral container filesystem."}]},{"text":[{"text":"Commit the file to the Space repository after each write, so it is versioned with the code."}],"correct":false,"tempting":[{"text":"Would persist the data but makes a git commit per request and publishes user text into the Space’s own repository."}]},{"text":[{"text":"Push the collected records to a private dataset repository on a scheduled commit."}],"correct":true,"tempting":null},{"text":[{"text":"Hold the records in a module-level list and write them out when the Space shuts down."}],"correct":false,"tempting":[{"text":"Shares data across users and assumes a graceful shutdown that container restarts do not provide."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"hardware-sleep-and-zerogpu","title":"Hardware, sleep, and the bill that runs while you are asleep","minutes":9},"next":{"slug":"beyond-gradio","title":"Streamlit, Docker and a Space with no server at all","minutes":8}}}