{"course":{"slug":"ai-for-students","title":"AI for Students","url":"https://addaly.com/learn/ai-for-students","level":"beginner","track":"work"},"lesson":{"slug":"statistics-for-coursework","title":"Statistics for coursework","url":"https://addaly.com/learn/ai-for-students/statistics-for-coursework","position":36,"of":73,"minutes":9,"answer":"Give the model the design, the measurement level, the sample sizes and the real question, then run the test in jamovi or R and plot the data before you trust any number.","keyIdea":"Give the model the design, the measurement level, the sample sizes and the real question, then run the test in jamovi or R and plot the data before you trust any number, because most analysis errors are visible in the plot."},"body":{"format":"markdown","text":"## Choosing the test is not the same as running it\n\nStatistics is where students hand over the most and understand the least, because the software produces a number and the number looks like an answer. A model will suggest a test in one line, and the suggestion will often be reasonable and occasionally wrong in a way that invalidates the whole analysis.\n\nThe division of labour that works: use the model to understand *why* a test suits your design, run the test in a deterministic tool, and interpret the output yourself.\n\n## The four things that determine the test\n\nModels give bad statistical advice mainly because students give them incomplete descriptions. Supply all four, every time:\n\n1. **The design.** Between-subjects, within-subjects, paired, repeated measures, how many groups, how many factors.\n2. **The measurement level of the outcome.** Continuous, ordinal, count, binary, time-to-event. This is the one most often omitted and it changes the answer completely: a five-point questionnaire item is ordinal, and a t-test on it is a choice that needs defending.\n3. **The sample sizes,** including how unequal they are.\n4. **The actual question.** Difference, association, prediction, agreement. These need different tests and students frequently describe one while wanting another.\n\nWith those four supplied, the recommendation is usually sound. With any of them missing, the model fills the gap with a default, and defaults are how you end up running a t-test on ordinal data with n of 7.\n\n> Figure: Choosing the test is not the same as running it. Steps: 1. State the design (Between or within subjects, how many groups, how many factors.); 2. State the measurement level (Continuous, ordinal, count, binary. The one most often left out, and it changes the answer.); 3. Ask what is defensible (Which tests fit, and what each one assumes. The model is good at this once it has the four facts.); 4. Run it in jamovi or R (Free, and it computes rather than predicting the number.); 5. Plot it before believing it (A group of three, a ceiling, an outlier, a bimodal split. Most errors are visible here.). Supply the design, the measurement level, the sample sizes and the actual question, and the recommendation is usually sound. Leave one out and the model fills the gap with a default, which is how a t-test ends up on a five-point questionnaire item with seven people in a group. jamovi and JASP are free and produce the output a marker expects.\n\n## Free software that a department will accept\n\n- **jamovi**: free, open source, a proper graphical interface, output that looks like the SPSS output your marker expects. This is the recommendation for most undergraduates.\n- **JASP**: free, similar, with unusually good Bayesian options.\n- **R** with RStudio: free, what most of the field is moving to, steeper but worth it if you will do more than one project.\n- **Python** with pandas, statsmodels and scipy: free, and the right choice if you already program.\n- **PSPP**: free, deliberately SPSS-compatible.\n- **LibreOffice Calc** or Google Sheets for descriptive statistics and simple tests.\n\nSPSS and Stata are what job ads name, and your university probably has a licence. jamovi and JASP teach the same concepts and produce publishable output, and the transfer takes an afternoon.\n\n## The assumptions are the part that gets skipped\n\nEvery test has conditions and the marks are usually in checking them.\n\n- **Normality**: of the residuals, not the raw data, which is a distinction models sometimes get wrong. Check with a plot, not only a test; a significance test for normality is oversensitive at large n and underpowered at small n, which is the opposite of useful.\n- **Homogeneity of variance**: and know what your test does when it fails, which is usually Welch's correction, often the better default anyway.\n- **Independence**: the one that quietly invalidates analyses. Repeated measurements on the same person, students within classes, sites within regions. If observations are clustered, an analysis that ignores it is wrong regardless of what the output says.\n\nAsk the model to *list the assumptions and how to check each one in jamovi*. That is a good, checkable, well-specified request.\n\n## Interpreting output without fooling yourself\n\nThe standard errors, all still common, all still marked:\n\n- **A p-value is not the probability the hypothesis is true**, and not the probability the result was chance. It is the probability of data at least this extreme if the null were true.\n- **Not significant does not mean no effect.** With a small sample it usually means you could not tell.\n- **Statistically significant does not mean important.** With n of 5,000 a trivial difference is significant.\n- **Always report the effect size and a confidence interval**, not only p. This is now expected in most disciplines and is frequently where the marks are.\n- **Multiple comparisons inflate false positives.** Twenty tests at 0.05 give you one significant result by chance on average. Say how many tests you ran.\n\nA model explains all of these well, because they are well represented in the training text. Use it for that.\n\n## The move that catches most errors\n\nBefore running anything, write one sentence: *I expect X to be larger than Y by roughly Z, because...* Then look at the descriptive statistics and the plot before the test.\n\nMost analysis errors are visible in the plot. A group of 3. A ceiling effect where everyone scored maximum. An outlier at ten times everything else. A bimodal distribution that means your single group is two. A test run on data you have not plotted is a number produced about a dataset nobody has looked at.\n\n## And the honest limitation\n\nStatistics is the area where a confident wrong recommendation is hardest for a beginner to detect, because the output arrives formatted and authoritative either way. If the analysis matters (a dissertation, anything that will be published) take your plan to a human who knows statistics before you collect data, not after. Most universities have a statistics support service, it is free, and it is enormously underused."},"check":{"question":"A student describes their study as \"comparing two groups\" and is advised to run an independent t-test. The outcome is a single five-point agreement item and the groups are 9 and 34. What was the root problem?","options":[{"text":"The model gave incorrect statistical advice, which is a known weakness on questions involving assumptions.","correct":false,"tempting":"Blames capability for an underspecification failure; given the full description the recommendation would likely have been sound."},{"text":"The description omitted measurement level and sample sizes, so a continuous-data default was applied.","correct":true},{"text":"A t-test is never valid on questionnaire data, so any recommendation involving one was wrong from the start.","correct":false,"tempting":"States a contested position as settled; analysing Likert items as continuous is defensible in some circumstances and is a choice to justify, not an automatic error."},{"text":"The sample sizes were too small for any inferential test at all, so the correct advice would have been to collect considerably more data first.","correct":false,"tempting":"Substitutes a blanket power objection for the specific mismatch; n of 9 and 34 supports several appropriate analyses."}]},"prev":{"slug":"learning-a-language","title":"Languages, where it is genuinely excellent","url":"https://addaly.com/learn/ai-for-students/learning-a-language"},"next":{"slug":"essays-across-subjects","title":"The essay subjects, and what a marker is reading for","url":"https://addaly.com/learn/ai-for-students/essays-across-subjects"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Statistics for coursework', in AI for Students. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-students/statistics-for-coursework (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Statistics for coursework","courseTitle":"AI for Students","module":{"index":4,"slug":"subject-by-subject","title":"Subject by subject"},"answer":null,"keyIdea":"Give the model the design, the measurement level, the sample sizes and the real question, then run the test in jamovi or R and plot the data before you trust any number, because most analysis errors are visible in the plot.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"State the design","detail":"Between or within subjects, how many groups, how many factors."},{"label":"State the measurement level","detail":"Continuous, ordinal, count, binary. The one most often left out, and it changes the answer."},{"label":"Ask what is defensible","detail":"Which tests fit, and what each one assumes. The model is good at this once it has the four facts."},{"label":"Run it in jamovi or R","detail":"Free, and it computes rather than predicting the number."},{"label":"Plot it before believing it","detail":"A group of three, a ceiling, an outlier, a bimodal split. Most errors are visible here."}],"title":"Choosing the test is not the same as running it","caption":"Supply the design, the measurement level, the sample sizes and the actual question, and the recommendation is usually sound. Leave one out and the model fills the gap with a default, which is how a t-test ends up on a five-point questionnaire item with seven people in a group. jamovi and JASP are free and produce the output a marker expects."},"alt":"Figure: Choosing the test is not the same as running it. Steps: 1. State the design (Between or within subjects, how many groups, how many factors.); 2. State the measurement level (Continuous, ordinal, count, binary. The one most often left out, and it changes the answer.); 3. Ask what is defensible (Which tests fit, and what each one assumes. The model is good at this once it has the four facts.); 4. Run it in jamovi or R (Free, and it computes rather than predicting the number.); 5. Plot it before believing it (A group of three, a ceiling, an outlier, a bimodal split. Most errors are visible here.). Supply the design, the measurement level, the sample sizes and the actual question, and the recommendation is usually sound. Leave one out and the model fills the gap with a default, which is how a t-test ends up on a five-point questionnaire item with seven people in a group. jamovi and JASP are free and produce the output a marker expects."}},"blocks":[{"t":"h2","id":"choosing-the-test-is-not-the-same-as-running-it","runs":[{"text":"Choosing the test is not the same as running it"}]},{"t":"p","runs":[{"text":"Statistics is where students hand over the most and understand the least, because the software produces a number and the number looks like an answer. A model will suggest a test in one line, and the suggestion will often be reasonable and occasionally wrong in a way that invalidates the whole analysis."}]},{"t":"p","runs":[{"text":"The division of labour that works: use the model to understand "},{"text":"why","i":true},{"text":" a test suits your design, run the test in a deterministic tool, and interpret the output yourself."}]},{"t":"h2","id":"the-four-things-that-determine-the-test","runs":[{"text":"The four things that determine the test"}]},{"t":"p","runs":[{"text":"Models give bad statistical advice mainly because students give them incomplete descriptions. Supply all four, every time:"}]},{"t":"ol","start":1,"items":[[{"text":"The design.","b":true},{"text":" Between-subjects, within-subjects, paired, repeated measures, how many groups, how many factors."}],[{"text":"The measurement level of the outcome.","b":true},{"text":" Continuous, ordinal, count, binary, time-to-event. This is the one most often omitted and it changes the answer completely: a five-point questionnaire item is ordinal, and a t-test on it is a choice that needs defending."}],[{"text":"The sample sizes,","b":true},{"text":" including how unequal they are."}],[{"text":"The actual question.","b":true},{"text":" Difference, association, prediction, agreement. These need different tests and students frequently describe one while wanting another."}]]},{"t":"p","runs":[{"text":"With those four supplied, the recommendation is usually sound. With any of them missing, the model fills the gap with a default, and defaults are how you end up running a t-test on ordinal data with n of 7."}]},{"t":"h2","id":"free-software-that-a-department-will-accept","runs":[{"text":"Free software that a department will accept"}]},{"t":"ul","items":[[{"text":"jamovi","b":true},{"text":": free, open source, a proper graphical interface, output that looks like the SPSS output your marker expects. This is the recommendation for most undergraduates."}],[{"text":"JASP","b":true},{"text":": free, similar, with unusually good Bayesian options."}],[{"text":"R","b":true},{"text":" with RStudio: free, what most of the field is moving to, steeper but worth it if you will do more than one project."}],[{"text":"Python","b":true},{"text":" with pandas, statsmodels and scipy: free, and the right choice if you already program."}],[{"text":"PSPP","b":true},{"text":": free, deliberately SPSS-compatible."}],[{"text":"LibreOffice Calc","b":true},{"text":" or Google Sheets for descriptive statistics and simple tests."}]]},{"t":"p","runs":[{"text":"SPSS and Stata are what job ads name, and your university probably has a licence. jamovi and JASP teach the same concepts and produce publishable output, and the transfer takes an afternoon."}]},{"t":"h2","id":"the-assumptions-are-the-part-that-gets-skipped","runs":[{"text":"The assumptions are the part that gets skipped"}]},{"t":"p","runs":[{"text":"Every test has conditions and the marks are usually in checking them."}]},{"t":"ul","items":[[{"text":"Normality","b":true},{"text":": of the residuals, not the raw data, which is a distinction models sometimes get wrong. Check with a plot, not only a test; a significance test for normality is oversensitive at large n and underpowered at small n, which is the opposite of useful."}],[{"text":"Homogeneity of variance","b":true},{"text":": and know what your test does when it fails, which is usually Welch’s correction, often the better default anyway."}],[{"text":"Independence","b":true},{"text":": the one that quietly invalidates analyses. Repeated measurements on the same person, students within classes, sites within regions. If observations are clustered, an analysis that ignores it is wrong regardless of what the output says."}]]},{"t":"p","runs":[{"text":"Ask the model to "},{"text":"list the assumptions and how to check each one in jamovi","i":true},{"text":". That is a good, checkable, well-specified request."}]},{"t":"h2","id":"interpreting-output-without-fooling-yourself","runs":[{"text":"Interpreting output without fooling yourself"}]},{"t":"p","runs":[{"text":"The standard errors, all still common, all still marked:"}]},{"t":"ul","items":[[{"text":"A p-value is not the probability the hypothesis is true","b":true},{"text":", and not the probability the result was chance. It is the probability of data at least this extreme if the null were true."}],[{"text":"Not significant does not mean no effect.","b":true},{"text":" With a small sample it usually means you could not tell."}],[{"text":"Statistically significant does not mean important.","b":true},{"text":" With n of 5,000 a trivial difference is significant."}],[{"text":"Always report the effect size and a confidence interval","b":true},{"text":", not only p. This is now expected in most disciplines and is frequently where the marks are."}],[{"text":"Multiple comparisons inflate false positives.","b":true},{"text":" Twenty tests at 0.05 give you one significant result by chance on average. Say how many tests you ran."}]]},{"t":"p","runs":[{"text":"A model explains all of these well, because they are well represented in the training text. Use it for that."}]},{"t":"h2","id":"the-move-that-catches-most-errors","runs":[{"text":"The move that catches most errors"}]},{"t":"p","runs":[{"text":"Before running anything, write one sentence: "},{"text":"I expect X to be larger than Y by roughly Z, because...","i":true},{"text":" Then look at the descriptive statistics and the plot before the test."}]},{"t":"p","runs":[{"text":"Most analysis errors are visible in the plot. A group of 3. A ceiling effect where everyone scored maximum. An outlier at ten times everything else. A bimodal distribution that means your single group is two. A test run on data you have not plotted is a number produced about a dataset nobody has looked at."}]},{"t":"h2","id":"and-the-honest-limitation","runs":[{"text":"And the honest limitation"}]},{"t":"p","runs":[{"text":"Statistics is the area where a confident wrong recommendation is hardest for a beginner to detect, because the output arrives formatted and authoritative either way. If the analysis matters (a dissertation, anything that will be published) take your plan to a human who knows statistics before you collect data, not after. Most universities have a statistics support service, it is free, and it is enormously underused."}]}],"check":{"question":[{"text":"A student describes their study as “comparing two groups” and is advised to run an independent t-test. The outcome is a single five-point agreement item and the groups are 9 and 34. What was the root problem?"}],"options":[{"text":[{"text":"The model gave incorrect statistical advice, which is a known weakness on questions involving assumptions."}],"correct":false,"tempting":[{"text":"Blames capability for an underspecification failure; given the full description the recommendation would likely have been sound."}]},{"text":[{"text":"The description omitted measurement level and sample sizes, so a continuous-data default was applied."}],"correct":true,"tempting":null},{"text":[{"text":"A t-test is never valid on questionnaire data, so any recommendation involving one was wrong from the start."}],"correct":false,"tempting":[{"text":"States a contested position as settled; analysing Likert items as continuous is defensible in some circumstances and is a choice to justify, not an automatic error."}]},{"text":[{"text":"The sample sizes were too small for any inferential test at all, so the correct advice would have been to collect considerably more data first."}],"correct":false,"tempting":[{"text":"Substitutes a blanket power objection for the specific mismatch; n of 9 and 34 supports several appropriate analyses."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"learning-a-language","title":"Languages, where it is genuinely excellent","minutes":9},"next":{"slug":"essays-across-subjects","title":"The essay subjects, and what a marker is reading for","minutes":8}}}