{"course":{"slug":"ai-careers","title":"Working in AI","url":"https://addaly.com/learn/ai-careers","level":"beginner","track":"work"},"lesson":{"slug":"the-data-you-can-legally-use","title":"Where portfolio data comes from, and what you may do with it","url":"https://addaly.com/learn/ai-careers/the-data-you-can-legally-use","position":30,"of":85,"minutes":9,"answer":"A dataset's licence, its provenance and whether it describes people decide what you may do with it, and the absence of a licence means you have no permission rather than a free hand.","keyIdea":"A dataset's licence, its provenance and whether it describes people decide what you may do with it, and the absence of a licence means you have no permission rather than a free hand."},"body":{"format":"markdown","text":"## The question nobody asks until it is a problem\n\nYou need data for a project. The instinct is to take some. Three questions decide whether that is fine, awkward, or career-ending: where did it come from, does it describe identifiable people, and what does its licence permit.\n\n## Where good data actually lives\n\n- **National statistics and open-government portals.** India's data.gov.in, the UK's data.gov.uk, the EU portal, the US data.gov, Nigeria's and Kenya's open data initiatives, Brazil's dados.gov.br. Census, health, transport, agriculture, budgets, procurement. Usually the most interesting data available anywhere, and almost nobody uses it, which is exactly why a project built on it stands out.\n- **International bodies.** World Bank, WHO, UN, FAO, OECD. Cleanly licensed and well documented.\n- **Research repositories.** UCI, Zenodo, OpenML, Hugging Face Datasets, Papers with Code.\n- **Kaggle.** Enormous and convenient, with a caveat below.\n- **Your own.** Data you generate, log or collect with consent. The strongest option for a portfolio, because nobody else has it.\n\n## Licences, in five minutes\n\nThe licence is a permission, not a formality:\n\n- **CC0 / public domain**: do anything.\n- **CC-BY**: do anything, credit the source.\n- **CC-BY-SA / ODbL**: as above, but derived datasets must carry the same licence. OpenStreetMap is ODbL, and this catches people.\n- **Non-commercial (CC-BY-NC)**: fine for a portfolio, not fine inside a product.\n- **\"Research use only\"**: common on academic sets, and it means what it says.\n- **No licence at all**: the default is that you have no permission. Absence of a licence is not permission.\n\nThe Kaggle caveat: many datasets there were uploaded by somebody who was not the rights holder, and the licence field was chosen by that uploader. A dataset marked CC0 that is plainly a scrape of a commercial site is not CC0. Trace it to the original source when the provenance looks thin.\n\n## Scraping: the honest state of it\n\nTechnically easy. Legally unsettled, and unsettled differently in different countries. The shape of the disagreement, which is the most anybody can honestly give you:\n\n- Access to **publicly available** pages has been treated relatively permissively in some United States computer-misuse rulings, while a site's **terms of service** are a separate contractual question that those rulings did not resolve.\n- The **European Union** has a database right that has no American equivalent, and separate text-and-data-mining provisions with an opt-out for commercial use. **Japan** and **Singapore** have broader mining exceptions.\n- **Copyright in AI training data** is being litigated in several countries right now, with decisions pointing in different directions. Anybody who tells you this is settled is describing their preference.\n\nNone of this is legal advice, and this course cannot give any. What is defensible practice regardless of jurisdiction: respect `robots.txt`, rate-limit yourself, prefer a documented API where one exists, never take personal data, never redistribute the raw scrape, and be able to explain in one paragraph why you believed you were entitled to the data. If a project cannot survive that paragraph, choose different data; there is plenty.\n\n## Personal data is regulated even when it is public\n\nIf it identifies a person, a data protection law probably applies: the GDPR in Europe, India's Digital Personal Data Protection Act, Nigeria's Data Protection Act, Brazil's LGPD, and others. \"It was on a public profile\" is not an exemption in any of them.\n\nFor a portfolio there is no upside. Aggregate, anonymise properly, or pick a different subject.\n\n## Never your employer's data. Not once.\n\nThis ends careers, and the mechanism is worth stating plainly. The data is not yours; it is covered by your employment contract, usually by a customer contract underneath that, and frequently by regulation on top. Publishing a chart built from it can be a breach of all three at once, and a reviewer who spots it will assume you would do the same to them.\n\nWhat to do instead: describe the problem and the method, and reproduce the *shape* of the data synthetically. `Faker`, `SDV` and twenty lines of NumPy generate a table with the same columns, cardinalities and rough distributions. Synthetic data is excellent for demonstrating a pipeline, a model or an interface. It is worthless for demonstrating a finding, and you should say which of the two you are showing.\n\n## The habit that costs one minute\n\nEvery project gets a `DATA.md` with the source URL, the licence, the download date and one line on any preprocessing. It takes a minute, it answers the reviewer's question before they ask it, and in a field where provenance is the live argument of the decade, it marks you as somebody who thinks about it."},"check":{"question":"An applicant's portfolio project analyses customer complaints. In the interview they explain the data came from their current employer's support system, with names removed. What does a reviewer most reasonably conclude?","options":[{"text":"The work is acceptable because removing names satisfies data protection requirements.","correct":false,"tempting":"Treats name removal as anonymisation, when free-text complaints routinely re-identify people and the contractual problem is untouched either way."},{"text":"The applicant is resourceful and has demonstrated access to realistic data, which is a point in their favour.","correct":false,"tempting":"Values the realism of the data over the judgement question the reviewer is actually being shown."},{"text":"The work should be accepted if the employer has no explicit policy against it.","correct":false,"tempting":"Assumes silence means permission, whereas employment and customer contracts normally assign the data to the employer by default."},{"text":"The applicant published data they did not own, likely breaching their employment contract and possibly a customer contract, and would do the same with this company's data.","correct":true}]},"prev":{"slug":"portfolio-from-zero","title":"A portfolio when you have never been paid for this","url":"https://addaly.com/learn/ai-careers/portfolio-from-zero"},"next":{"slug":"the-case-study-page","title":"Writing up a project as a decision record","url":"https://addaly.com/learn/ai-careers/the-case-study-page"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Where portfolio data comes from, and what you may do with it', in Working in AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-careers/the-data-you-can-legally-use (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Where portfolio data comes from, and what you may do with it","courseTitle":"Working in AI","module":{"index":3,"slug":"the-evidence","title":"The evidence: what gets tested, and how to prove you have it"},"answer":null,"keyIdea":"A dataset’s licence, its provenance and whether it describes people decide what you may do with it, and the absence of a licence means you have no permission rather than a free hand.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-question-nobody-asks-until-it-is-a-problem","runs":[{"text":"The question nobody asks until it is a problem"}]},{"t":"p","runs":[{"text":"You need data for a project. The instinct is to take some. Three questions decide whether that is fine, awkward, or career-ending: where did it come from, does it describe identifiable people, and what does its licence permit."}]},{"t":"h2","id":"where-good-data-actually-lives","runs":[{"text":"Where good data actually lives"}]},{"t":"ul","items":[[{"text":"National statistics and open-government portals.","b":true},{"text":" India’s data.gov.in, the UK’s data.gov.uk, the EU portal, the US data.gov, Nigeria’s and Kenya’s open data initiatives, Brazil’s dados.gov.br. Census, health, transport, agriculture, budgets, procurement. Usually the most interesting data available anywhere, and almost nobody uses it, which is exactly why a project built on it stands out."}],[{"text":"International bodies.","b":true},{"text":" World Bank, WHO, UN, FAO, OECD. Cleanly licensed and well documented."}],[{"text":"Research repositories.","b":true},{"text":" UCI, Zenodo, OpenML, Hugging Face Datasets, Papers with Code."}],[{"text":"Kaggle.","b":true},{"text":" Enormous and convenient, with a caveat below."}],[{"text":"Your own.","b":true},{"text":" Data you generate, log or collect with consent. The strongest option for a portfolio, because nobody else has it."}]]},{"t":"h2","id":"licences-in-five-minutes","runs":[{"text":"Licences, in five minutes"}]},{"t":"p","runs":[{"text":"The licence is a permission, not a formality:"}]},{"t":"ul","items":[[{"text":"CC0 / public domain","b":true},{"text":": do anything."}],[{"text":"CC-BY","b":true},{"text":": do anything, credit the source."}],[{"text":"CC-BY-SA / ODbL","b":true},{"text":": as above, but derived datasets must carry the same licence. OpenStreetMap is ODbL, and this catches people."}],[{"text":"Non-commercial (CC-BY-NC)","b":true},{"text":": fine for a portfolio, not fine inside a product."}],[{"text":"”Research use only”","b":true},{"text":": common on academic sets, and it means what it says."}],[{"text":"No licence at all","b":true},{"text":": the default is that you have no permission. Absence of a licence is not permission."}]]},{"t":"p","runs":[{"text":"The Kaggle caveat: many datasets there were uploaded by somebody who was not the rights holder, and the licence field was chosen by that uploader. A dataset marked CC0 that is plainly a scrape of a commercial site is not CC0. Trace it to the original source when the provenance looks thin."}]},{"t":"h2","id":"scraping-the-honest-state-of-it","runs":[{"text":"Scraping: the honest state of it"}]},{"t":"p","runs":[{"text":"Technically easy. Legally unsettled, and unsettled differently in different countries. The shape of the disagreement, which is the most anybody can honestly give you:"}]},{"t":"ul","items":[[{"text":"Access to "},{"text":"publicly available","b":true},{"text":" pages has been treated relatively permissively in some United States computer-misuse rulings, while a site’s "},{"text":"terms of service","b":true},{"text":" are a separate contractual question that those rulings did not resolve."}],[{"text":"The "},{"text":"European Union","b":true},{"text":" has a database right that has no American equivalent, and separate text-and-data-mining provisions with an opt-out for commercial use. "},{"text":"Japan","b":true},{"text":" and "},{"text":"Singapore","b":true},{"text":" have broader mining exceptions."}],[{"text":"Copyright in AI training data","b":true},{"text":" is being litigated in several countries right now, with decisions pointing in different directions. Anybody who tells you this is settled is describing their preference."}]]},{"t":"p","runs":[{"text":"None of this is legal advice, and this course cannot give any. What is defensible practice regardless of jurisdiction: respect "},{"text":"robots.txt","code":true},{"text":", rate-limit yourself, prefer a documented API where one exists, never take personal data, never redistribute the raw scrape, and be able to explain in one paragraph why you believed you were entitled to the data. If a project cannot survive that paragraph, choose different data; there is plenty."}]},{"t":"h2","id":"personal-data-is-regulated-even-when-it-is-public","runs":[{"text":"Personal data is regulated even when it is public"}]},{"t":"p","runs":[{"text":"If it identifies a person, a data protection law probably applies: the GDPR in Europe, India’s Digital Personal Data Protection Act, Nigeria’s Data Protection Act, Brazil’s LGPD, and others. “It was on a public profile” is not an exemption in any of them."}]},{"t":"p","runs":[{"text":"For a portfolio there is no upside. Aggregate, anonymise properly, or pick a different subject."}]},{"t":"h2","id":"never-your-employer-s-data-not-once","runs":[{"text":"Never your employer’s data. Not once."}]},{"t":"p","runs":[{"text":"This ends careers, and the mechanism is worth stating plainly. The data is not yours; it is covered by your employment contract, usually by a customer contract underneath that, and frequently by regulation on top. Publishing a chart built from it can be a breach of all three at once, and a reviewer who spots it will assume you would do the same to them."}]},{"t":"p","runs":[{"text":"What to do instead: describe the problem and the method, and reproduce the "},{"text":"shape","i":true},{"text":" of the data synthetically. "},{"text":"Faker","code":true},{"text":", "},{"text":"SDV","code":true},{"text":" and twenty lines of NumPy generate a table with the same columns, cardinalities and rough distributions. Synthetic data is excellent for demonstrating a pipeline, a model or an interface. It is worthless for demonstrating a finding, and you should say which of the two you are showing."}]},{"t":"h2","id":"the-habit-that-costs-one-minute","runs":[{"text":"The habit that costs one minute"}]},{"t":"p","runs":[{"text":"Every project gets a "},{"text":"DATA.md","code":true},{"text":" with the source URL, the licence, the download date and one line on any preprocessing. It takes a minute, it answers the reviewer’s question before they ask it, and in a field where provenance is the live argument of the decade, it marks you as somebody who thinks about it."}]}],"check":{"question":[{"text":"An applicant’s portfolio project analyses customer complaints. In the interview they explain the data came from their current employer’s support system, with names removed. What does a reviewer most reasonably conclude?"}],"options":[{"text":[{"text":"The work is acceptable because removing names satisfies data protection requirements."}],"correct":false,"tempting":[{"text":"Treats name removal as anonymisation, when free-text complaints routinely re-identify people and the contractual problem is untouched either way."}]},{"text":[{"text":"The applicant is resourceful and has demonstrated access to realistic data, which is a point in their favour."}],"correct":false,"tempting":[{"text":"Values the realism of the data over the judgement question the reviewer is actually being shown."}]},{"text":[{"text":"The work should be accepted if the employer has no explicit policy against it."}],"correct":false,"tempting":[{"text":"Assumes silence means permission, whereas employment and customer contracts normally assign the data to the employer by default."}]},{"text":[{"text":"The applicant published data they did not own, likely breaching their employment contract and possibly a customer contract, and would do the same with this company’s data."}],"correct":true,"tempting":null}],"order":[1,3,2,0],"why":null},"prev":{"slug":"portfolio-from-zero","title":"A portfolio when you have never been paid for this","minutes":7},"next":{"slug":"the-case-study-page","title":"Writing up a project as a decision record","minutes":8}}}