Is there a standard set percentage/range when determining the difference between true value and theoretical value inorder to consider a method valid?
J,
Unfortunately I haven’t come across a simple answer to this one. I look at it this way…
Firstly, let me quickly make sure we are using the same terms:
Bias = Difference of your result from the expected value.
Trueness = Recovery = The lack of bias. How close the measured result is to the expected value. A relative (non-numerical) term, usually expressed as a percentage.
Accuracy = Probably the most mis-used term in method validation. Despite my misgivings, it tends to be associated with the same meaning as trueness.
e.g, if
expected = 15 mg/L, s = 0.5 mg/L
measured = x-bar = 14.8 mg/kg, s = 0.4 mg/L, n = 10
Then
Bias (mg/L) = measured – expected = 14.8-15 = -0.2 mg/L
Trueness (%) = Recovery (%) = measured/expected * 100 = 14.8/15*100 = 98.7 %
Bias (%) = (measured – expected) / expected * 100 = -1.3 %
Or Bias (%) = Bias (mg/L) / expected * 100 = -1.3 %
Back to the topic.
If the method in question is based on a standardised, published method with stated performance limits, then these criteria would presumably include the minimum trueness/recovery level you need to meet.
If, however:
-
there is no specified recovery criterion in the standard method, or
-
you are using the standard method for an application that the standard method was not designed for (e.g., same technique on a different sample type), or
-
you do not have company/industry/regulatory required specifications you need to meet, or
-
you are dealing with a method developed by yourself (in-house method),
then unfortunately (or perhaps fortunately?) you need to determine an appropriate recovery criterion that is “fit for purpose”.
I once used set percentages in such situations, but clearly this approach is completely subjective. What is a statistically valid tolerance? These days, when no set limits are specified, I complete one of the following:
a) When using a standard reference material:
When using a standard reference material with a published value and standard deviation to determine trueness/recovery, ensure your result +/- your standard deviation, overlap.
-
Is Bias (mg/L) < 1 standard deviation, s, of the expected result?
In the example above: Is -0.2 mg/L < 0.5 mg/L = Yes -
Complete a student’s t-test on your measured results
Is the measured t-test < t-critical (two tailed, P=0.05, df=n-1)
In the example above: Is 1.58 < 2.26 = Yes -
Are all measured results within expected mean +/- 1 expected standard deviation (or is the expected x-bar + 1 s > maximum measured result & expected x-bar - 1 s < minimum measured result)
Note: You could substitute 1 s for 2 s in the cases above if that is “fit for purpose”.
b) For comparisons via the use of another technique (needs 5 different samples, with three reps each, minimum:
-
Grubbs test (or any other outlier test) to check for outliers for individual sample replicates;
-
Complete an F-test on the standard deviations of each technique to ensure it is less than the critical F statistic.
-
Plot one technique against the other. It should be 1:1, and hence gradient (slope) should be 1 (+/- conf. interval) and y-intercept should be 0 (+/- conf. interval).
-
Check residuals of the plotted results (ensure residuals are homoscedatic)
If someone knows of a better, easier, and more statistically valid approach, I’d love to hear it. Nevertheless, since using the approach above I’ve always passed any scrutiny.