Is there a standard set percentage/range when determining the difference between true value and theoretical value inorder to consider a method valid?
Unfortunately I haven’t come across a simple answer to this one. I look at it this way…
Firstly, let me quickly make sure we are using the same terms:
Bias = Difference of your result from the expected value.
Trueness = Recovery = The lack of bias. How close the measured result is to the expected value. A relative (non-numerical) term, usually expressed as a percentage.
Accuracy = Probably the most mis-used term in method validation. Despite my misgivings, it tends to be associated with the same meaning as trueness.
expected = 15 mg/L, s = 0.5 mg/L
measured = x-bar = 14.8 mg/kg, s = 0.4 mg/L, n = 10
Bias (mg/L) = measured – expected = 14.8-15 = -0.2 mg/L
Trueness (%) = Recovery (%) = measured/expected * 100 = 14.8/15*100 = 98.7 %
Bias (%) = (measured – expected) / expected * 100 = -1.3 %
Or Bias (%) = Bias (mg/L) / expected * 100 = -1.3 %
Back to the topic.
If the method in question is based on a standardised, published method with stated performance limits, then these criteria would presumably include the minimum trueness/recovery level you need to meet.
there is no specified recovery criterion in the standard method, or
you are using the standard method for an application that the standard method was not designed for (e.g., same technique on a different sample type), or
you do not have company/industry/regulatory required specifications you need to meet, or
you are dealing with a method developed by yourself (in-house method),
then unfortunately (or perhaps fortunately?) you need to determine an appropriate recovery criterion that is “fit for purpose”.
I once used set percentages in such situations, but clearly this approach is completely subjective. What is a statistically valid tolerance? These days, when no set limits are specified, I complete one of the following:
a) When using a standard reference material:
When using a standard reference material with a published value and standard deviation to determine trueness/recovery, ensure your result +/- your standard deviation, overlap.
Is Bias (mg/L) < 1 standard deviation, s, of the expected result?
In the example above: Is -0.2 mg/L < 0.5 mg/L = Yes
Complete a student’s t-test on your measured results
Is the measured t-test < t-critical (two tailed, P=0.05, df=n-1)
In the example above: Is 1.58 < 2.26 = Yes
Are all measured results within expected mean +/- 1 expected standard deviation (or is the expected x-bar + 1 s > maximum measured result & expected x-bar - 1 s < minimum measured result)
Note: You could substitute 1 s for 2 s in the cases above if that is “fit for purpose”.
b) For comparisons via the use of another technique (needs 5 different samples, with three reps each, minimum:
Grubbs test (or any other outlier test) to check for outliers for individual sample replicates;
Complete an F-test on the standard deviations of each technique to ensure it is less than the critical F statistic.
Plot one technique against the other. It should be 1:1, and hence gradient (slope) should be 1 (+/- conf. interval) and y-intercept should be 0 (+/- conf. interval).
Check residuals of the plotted results (ensure residuals are homoscedatic)
If someone knows of a better, easier, and more statistically valid approach, I’d love to hear it. Nevertheless, since using the approach above I’ve always passed any scrutiny.