Can one exclude calibration points using a combination of permutation testing and a non-parametric test statistic?

Question

I work in bioanalytical chemistry. I have used an analytical instrument (LC-MS) to measure the response (peak area) of several compounds spiked in to equivalent samples at known concentrations. Each sample, at each concentration, was analysed three times over three separate days. Spike concentrations, for this example, are: 0 (blank), 1, 2, 5, 10, 50, 125, 250, 500, 1000 ng/L. I now wish to construct sensible calibration curves using linear least squares regression (hence, assuming homoscedacity of data, but still acknowledging the potential for outliers) of the response (peak area) versus spike concentration (i.e. the spike concentration).

From visual inspection of data files, it appears that for some compounds there are samples that were spiked at > 0 ng/L that are indistinguishable from blank samples (i.e. those at 0 ng/L). I’d like to test my suspicions statistically and, if significant at a reasonable threshold, would exclude these samples prior to performing the least squares regression.

Given that, for many compounds, the data are often non-‘normal’ around the lower-end of the calibration curve in such dataset, I thought to use permutation testing to generate a sampling distribution. Here, I’d define a null hypothesis that the median peak area of blank samples (0 ng/L) is equal to that of non-blank samples at a defined concentration (e.g. 1 ng/L), i.e. a statistical test of location. Drawing from the world of parametric tests, I’d hoped to use: [median(peak areas of non-blank samples at single concentration) - median(peak areas of blank)] / median absolute deviation(peak areas of non-blank samples at single concentration).

Here, the sampling distribution would be generated by repeatedly randomising class labels (i.e. concentration labels for 0 ng/L and the calibrant level being tested) and calculating the test statistic. The p-value would be derived as the proportion of the permuted test statistics that were equal-to or greater-than the actual measured statistic, i.e. as calculated before randomising labels. If the p-value turns out to be greater than some pre-determined threshold (probably going to be 0.05 or 0.01), I accept the null hypothesis. Given that I would repeat this test for each concentration level, I’d also plan to apply a Benjamini-Hochberg correction to the final list of p-values and reject then exclude concentration levels with adjusted p-values above the cutoff (I’d aim for either 0.05 or 0.01 – yet to decide).

So, feedback on the following would be great:

1. Does the proposed test statistic seem reasonable as a non-parametric replacement to the t-statistic?
2. Is it appropriate to exclude samples from the regression, for which the p-value for the test statistic exceeds the defined threshold?

Answer 1

This is more of a comment, but a typical comment doesn't allow a figure...

You have a very wide range (1-1000) for the independent variable. Using a simple linear least squares analysis isn't appropriate.

I grabbed the image below from webpage https://www.mathworks.com/matlabcentral/answers/392636-confidence-band-around-linear-least-squares-line

The point I want to make is shown by the shaded area. Assuming homoscedasticity, you wouldn't weight the errors. Thus the confidence interval for the line is narrowest at the center of the fitted range and gets much wider at either end of the range. So your low samples ( 1, 2, 5, and 10) will essentially be irrelevant.

So you need to assume hetroscedasticity and weight the sample errors somehow.

---

1. Does the proposed test statistic seem reasonable as a non-parametric replacement to the t-statistic?

Just because in general non-parametric tests are less powerful than parametric tests, I would try hard to avoid using a non-parametric test on parametric data.

2. Is it appropriate to exclude samples from the regression, for which the p-value for the test statistic exceeds the defined threshold?

I'll assume that you mean to exclude some of the standards not samples. It is possible to exclude standards, but don't use three standards and then drop the one that you "know" isn't on the line. :-)

---

Hope this helps a little. I just don't know enough about S/N for LC/MS or the detector system that you're using to tell you what you should do.

Your statement "Given that, for many compounds, the data are often non-‘normal’ around the lower-end of the calibration curve in such dataset,..." is disconcerting.

Answer 2

Before running a nonparametric analysis, try viewing the problem from a parametric vantage of Generalized Least-Squares. To quote a source:

In that regard, GLM repeated measure is a statistical technique that takes a dependent, or criterion variable, measured as correlated, non-independent data. Commonly used when measuring the effect of a treatment at different time points. The independent variables may be categorical or continuous. GLM repeated measure can be used to test the main effects within and between the subjects, interaction effects between factors, covariate effects and effects of interactions between covariates and between subject factors. GLM repeated measures in SPSS is done by selecting “general linear model” from the “analyze” menu. From general linear model, select “repeated measures” and then preform “GLM repeated measures.

So, one of the important aspects of your data is that you have repeated observations, likely with correlated error structure.

Also, you measured your explanatory variable concentration amounts. Depending on how easy this was accomplished, there could be some associated measurement error also in your explanatory variable. This may lead to bias regression coefficients (see this source).