I work in bioanalytical chemistry. I have used an analytical instrument (LC-MS) to measure the response (peak area) of several compounds spiked in to equivalent samples at known concentrations. Each sample, at each concentration, was analysed three times over three separate days. Spike concentrations, for this example, are: 0 (blank), 1, 2, 5, 10, 50, 125, 250, 500, 1000 ng/L. I now wish to construct sensible calibration curves using linear least squares regression (hence, assuming homoscedacity of data, but still acknowledging the potential for outliers) of the response (peak area) versus spike concentration (i.e. the spike concentration).
From visual inspection of data files, it appears that for some compounds there are samples that were spiked at > 0 ng/L that are indistinguishable from blank samples (i.e. those at 0 ng/L). I’d like to test my suspicions statistically and, if significant at a reasonable threshold, would exclude these samples prior to performing the least squares regression.
Given that, for many compounds, the data are often non-‘normal’ around the lower-end of the calibration curve in such dataset, I thought to use permutation testing to generate a sampling distribution. Here, I’d define a null hypothesis that the median peak area of blank samples (0 ng/L) is equal to that of non-blank samples at a defined concentration (e.g. 1 ng/L), i.e. a statistical test of location. Drawing from the world of parametric tests, I’d hoped to use: [median(peak areas of non-blank samples at single concentration) - median(peak areas of blank)] / median absolute deviation(peak areas of non-blank samples at single concentration).
Here, the sampling distribution would be generated by repeatedly randomising class labels (i.e. concentration labels for 0 ng/L and the calibrant level being tested) and calculating the test statistic. The p-value would be derived as the proportion of the permuted test statistics that were equal-to or greater-than the actual measured statistic, i.e. as calculated before randomising labels. If the p-value turns out to be greater than some pre-determined threshold (probably going to be 0.05 or 0.01), I accept the null hypothesis. Given that I would repeat this test for each concentration level, I’d also plan to apply a Benjamini-Hochberg correction to the final list of p-values and reject then exclude concentration levels with adjusted p-values above the cutoff (I’d aim for either 0.05 or 0.01 – yet to decide).
So, feedback on the following would be great:
- Does the proposed test statistic seem reasonable as a non-parametric replacement to the t-statistic?
- Is it appropriate to exclude samples from the regression, for which the p-value for the test statistic exceeds the defined threshold?