22.7 - Evaluating a Method - Errors, Anomalies & Reliability
What an evaluation question looks like
- After the results are in, you may be asked to judge how well the method worked.
- There's no single fixed question - a paper picks from a small set of evaluation tasks, each worth a mark or two.
- Each task follows a fixed rule that the mark scheme applies every series, so the marks are there once you know the rule.
A typical question
During the experiment the temperature of the water bath fell.
(i) Identify a source of error in this investigation. [1]
(ii) Suggest an improvement that would reduce this error. [1]
- The two parts are marked together: the error only scores if the improvement you give actually fixes that error.
The full set of evaluation tasks
- Error + improvement - Identify a source of error, and pair it with an improvement that reduces it (2 marks)
- Anomalies - Spot an anomalous result and say what to do with it (1 mark)
- Repeats and means - Explain why the investigation is repeated, or why a mean is taken (1 mark)
- Sample size - Explain why a large or representative sample is used (1 mark)
Sources of error and improvements
- A source of error is something in the method that makes a reading less accurate.
- Every error must be paired with an improvement that actually reduces it - an error on its own scores nothing (just as a safety hazard scores nothing without a precaution).
- The rule is error must match improvement: name the error, then say what you'd change to fix that particular error.
Common errors and their matching improvements
| Source of error | Why it affects the result | Improvement that fixes it |
|---|---|---|
| Adding a liquid drop by drop from a dropping pipette | Drop size varies, so the volume added isn't consistent | Measure the volume with a syringe or a measuring cylinder |
| Judging the end-point (a colour change) by eye | The exact moment of the change is a matter of opinion, so different people time it differently | Compare against a colour standard, a prepared colour chart, or a colorimeter |
| The water bath cooling during the run | The temperature should be held constant, so a drift changes the rate | Use a thermostatically controlled water bath |
| Shaking or stirring the tubes by hand | The amount of mixing differs from one tube to the next | Stir each tube at a fixed speed with a mechanical stirrer |
- Each improvement must target its own error: a syringe fixes an inconsistent volume, a thermostatic bath fixes a drifting temperature.
- Reaching for the wrong pair - such as "use a syringe" to fix a temperature that fell - does not score.
Anomalies and reliability
- An anomaly is a result that doesn't fit the pattern of the others - an odd one out, well away from the trend the rest of the readings make.
- When you find one, identify it and leave it out of the mean, so one stray reading doesn't drag the average off.
- You can only tell a reading is anomalous once you have others to compare it against - which is exactly why you repeat: you repeat to identify anomalous results.
Repeating and taking a mean makes a result reliable
- The set phrase to use: repeat the whole investigation at least twice more and take a mean of the readings at each value.
- Two reasons behind it:
- A single reading might be an anomaly you'd never spot on its own.
- Repeats that give much the same value show the result is not a one-off.
- That is what "more reliable" means here.
A representative sample is a separate idea
- This comes up whenever the experiment measures a number of living things, rather than repeating one measurement.
- Measuring just one tells you little, because that one might be unusual - e.g. the average length of a leaf, or how much water a raisin takes up.
- Instead, measure about ten and take a mean so the figure represents the group, choosing them without bias.
- A study that measured a single raisin, or a single animal of each type, is faulted: you should take measurements from more than one of each type to get a representative sample.
Common exam questions
Question 1
A student times a colour change by watching for it by eye. Give one source of error and an improvement that would reduce it.
The end-point is judged by eye, so the exact moment is subjective; compare the colour against a prepared colour standard or use a colorimeter so the end-point is decided the same way each time.
Question 2
Why does "the water bath is a source of error" on its own not score full marks?
Because an error only scores when it is paired with a matching improvement; you also need to say what you would change, such as using a thermostatically controlled water bath to hold the temperature steady.
Question 3
One of five readings is far higher than the other four. What is it called, and what should you do with it when finding the mean?
It is an anomaly, and you leave it out of the mean, calculating the average from the four consistent readings.
Question 4
Why should the whole investigation be repeated and a mean taken?
Because a single reading could be an anomaly, and repeats that give much the same value make the result more reliable.
Question 5
A student measured the mass gained by one raisin left in water. Why is measuring several raisins better?
Because one raisin might be unusual and not represent the rest; measuring several and taking a mean gives a representative result and reduces the effect of any single odd one.