The purpose and criteria for choosing inferential statistical tests

Inferential statistical tests are essential tools in psychological research, used to analyse data and determine whether differences or relationships between sets of data are significant, meaning they are unlikely to have occurred by chance. These tests help make predictions about larger populations based on sample data.

Key considerations for selecting a statistical test

• Research design - The type of design used in the study influences the choice of test. Options include:
  • Independent groups design (IGD), where different participants are used in each condition.
  • Repeated measures design (RMD), including matched pairs design (MPD), where the same or matched participants are tested in all conditions.
  • Correlational design, which examines relationships between variables.
• Outcome being tested - Determining whether the research seeks to identify a difference between data sets or a relationship (correlation) between them.
• Level of measurement - The type of data collected, which can be nominal, ordinal, or interval/ratio, affects the suitability of the test.

Guide to choosing the appropriate test

Probability, significance levels, and types of errors

Statistical analysis in psychology relies on the concept of probability to assess whether results are significant or due to chance. Significance levels provide a threshold for making this determination, while errors in judgement can occur during interpretation.

Understanding probability and significance

• Probability - Refers to the likelihood that results are due to chance factors rather than a real effect.
• Significance level - Psychology typically uses a significance level of p < 0.05, indicating a 95% confidence that results are not due to chance. This means there is a 5% chance of incorrectly identifying results as significant.
• Stringent levels - In critical areas like drug testing, a stricter level of p < 0.01 is used, ensuring a 99% confidence level, reducing the chance of error to 1%.

Types of errors in statistical testing

• Type I error - Occurs when results are accepted as significant (beyond chance) when they are actually due to chance, often because the significance level was too lenient.
• Type II error - Happens when results are dismissed as insignificant (due to chance) when they are actually significant, often because the significance level was too strict.

Levels of measurement in data collection

Data in psychological research is categorised into different levels of measurement, each providing varying degrees of detail and influencing the choice of statistical test.

Categories of data measurement

• Nominal data - Represents frequencies or categories, such as counting how many participants preferred one option over another. It provides basic information but lacks detail on magnitude or intensity.
• Ordinal data - Involves ranking data, like placing students in order of test scores. It shows relative position but does not indicate the exact difference between ranks.
• Interval/Ratio data - Offers the most detailed information with standardised units, such as measuring reaction times in seconds. Interval data has an arbitrary zero (e.g., temperature in Celsius), while ratio data has an absolute zero (e.g., height in centimetres), meaning zero represents a complete absence of the variable.

Interpretation of statistical significance

Determining whether results are statistically significant involves comparing observed values from a test to critical values found in reference tables. This process helps decide if findings support the research hypothesis.

Process of significance interpretation

• Observed value - The result calculated from the statistical test based on the collected data.
• Critical value - A benchmark value from critical value tables, which depends on factors like the type of hypothesis (one-tailed or two-tailed), the number of participants or pairs, and the chosen significance level.

For tests like Mann-Whitney, Wilcoxon signed-matched ranks, and the sign test, the observed value must be equal to or less than the critical value to be significant.

For Chi-squared, independent t-test, repeated t-test, Spearman's rho, and Pearson product moment, the observed value must be equal to or greater than the critical value to be significant.

Types of statistical tests and their specific applications

Various statistical tests are available for different research scenarios, each suited to specific designs, data levels, and research goals.

Tests for differences between data sets

• Sign test - Applied when predicting a difference between two sets of data, using at least nominal level data, and employing a repeated measures or matched pairs design.
• Chi-squared test - Used to predict a difference between two sets of data with at least nominal level data in an independent groups design. It can also test for associations (relationships) between variables.
• Mann-Whitney test - Suitable for predicting a difference between two data sets with at least ordinal level data in an independent groups design.
• Wilcoxon signed-matched ranks test - Used when predicting a difference between two data sets with at least ordinal level data in a repeated measures or matched pairs design.
• Independent t-test - Applied to predict a difference between two data sets with normally distributed interval/ratio data in an independent groups design.
• Repeated t-test - Used for predicting a difference between two data sets with normally distributed interval/ratio data in a repeated measures or matched pairs design.

Tests for relationships between data sets

• Spearman's rho - Employed when predicting a relationship (correlation) between two data sets with at least ordinal level data, consisting of paired scores from the same individual or event.
• Pearson product moment - Used for predicting a relationship between two data sets with normally distributed interval/ratio data, consisting of paired scores from the same individual or event.