Purpose and application of the chi-square test

The chi-square test is a statistical method used to determine whether an observed distribution of data differs significantly from what would be expected by chance. It helps researchers assess if there is a relationship between two categorical variables, making it particularly useful in psychology for analysing data from independent measures designs.

This test is often referred to as a 'goodness of fit' statistic because it evaluates how well the observed data match an expected pattern, assuming the variables are independent. If the test reveals a substantial difference between observed and expected values, it suggests that the variables may not be independent, indicating a possible association.

Data requirements and when to use the chi-square test

The chi-square test is specifically designed for nominal data, which means information that is categorised or counted without any inherent order.

Key requirements for using the chi-square test

• Nominal (categorical) data - This involves grouping observations into distinct categories, such as 'yes' or 'no' responses, rather than continuous measurements like height or test scores.
• Frequency counts - The data must consist of raw counts or frequencies, not percentages, averages, or other transformed values.
• Independent measures - The test applies to designs where participants are in separate groups, and each observation is independent.

This test is not suitable for parametric data, which assumes a normal distribution, or for continuous variables. For instance, raw exam scores from 0 to 100 cannot be used directly, but categorising them into 'pass' and 'fail' would make them appropriate.

By meeting these criteria, the test can reliably indicate if differences in category frequencies are due to chance or a meaningful effect.

Null hypothesis in the chi-square test

Before conducting the test, researchers formulate a null hypothesis, which is the assumption that there is no significant association between the variables being studied. This leads to the expectation that observed frequencies should closely match those predicted by chance.

If the calculated chi-square value shows a large discrepancy, the null hypothesis can be rejected, suggesting that the variables interact in some way. Conversely, a small difference supports retaining the null hypothesis, implying the distributions are essentially similar.

Formula and steps for calculating the chi-square value

The chi-square test involves comparing observed frequencies (actual counts from the data) with expected frequencies (values predicted under the null hypothesis).

Formula for chi-square

$X^2=\sum \frac{{(O-E)}^2}{E}$

Where:

• O = Observed frequency (actual count in each category)
• E = Expected frequency (calculated value assuming no association)

To perform the calculation, follow these steps in sequence.

Steps to calculate chi-square:

1. Organise the data into a contingency table showing observed frequencies for each category combination.
2. Calculate the expected frequency for each cell using: $E=\frac{\text{row total}\times \text{column total}}{\text{grand total}}$
3. For each cell, subtract the expected frequency from the observed frequency (O - E).
4. Square the result ((O - E)²).
5. Divide by the expected frequency (((O - E)²) / E).
6. Sum these values across all cells to get the chi-square statistic (X²).

These steps quantify the overall difference between what was observed and what was expected.

Determining degrees of freedom and significance

Once the chi-square value is calculated, it must be compared to a critical value to determine if the results are statistically significant.

Formula for degrees of freedom

$\text{Degrees of freedom (df)}=(R-1)\times (C-1)$

Where:

• R = Number of rows in the contingency table
• C = Number of columns in the contingency table

For a standard 2x2 table, df = 1.

Assessing significance:

• Compare the calculated X² to critical values from a chi-square distribution table, based on the chosen significance level (commonly p ≤ 0.05 or p ≤ 0.01) and df.
• If X² is greater than or equal to the critical value, reject the null hypothesis, indicating a significant association.
• If X² is less than the critical value, retain the null hypothesis, suggesting no significant difference.

Worked example - Applying the chi-square test to behavioural data

Researchers investigated whether male and female students complied with a request to switch off their laptops during a seminar. The observed frequencies are shown below.

Calculate the chi-square value, degrees of freedom, and determine significance at p ≤ 0.05 and p ≤ 0.01.

Step 1: Calculate expected frequencies

For each cell: E = (row total × column total) / grand total

• Cell A (Female, Switched off): (100 × 110) / 165 ≈ 66.67
• Cell B (Male, Switched off): (100 × 55) / 165 ≈ 33.33
• Cell C (Female, Did not): (65 × 110) / 165 ≈ 43.33
• Cell D (Male, Did not): (65 × 55) / 165 ≈ 21.67

Step 2: Compute (O - E)² / E for each cell

• Cell A: (65 - 66.67)² / 66.67 = (-1.67)² / 66.67 ≈ 2.7889 / 66.67 ≈ 0.042
• Cell B: (35 - 33.33)² / 33.33 = (1.67)² / 33.33 ≈ 2.7889 / 33.33 ≈ 0.084
• Cell C: (45 - 43.33)² / 43.33 = (1.67)² / 43.33 ≈ 2.7889 / 43.33 ≈ 0.064
• Cell D: (20 - 21.67)² / 21.67 = (-1.67)² / 21.67 ≈ 2.7889 / 21.67 ≈ 0.129

Step 3: Sum to find chi-square value

X² = 0.042 + 0.084 + 0.064 + 0.129 ≈ 0.319

Step 4: Determine degrees of freedom and significance

df = (2 - 1) × (2 - 1) = 1

Critical values: 3.84 (p ≤ 0.05), 6.63 (p ≤ 0.01)

Since 0.319 < 3.84, the result is not significant at p ≤ 0.05 or p ≤ 0.01. Retain the null hypothesis; there is no significant association between sex and compliance.