Quantify the effectiveness of baseline correction in Raman spectra

Question

I've produced several thousand Raman spectra using hyperspectral imaging techniques. These suffer from significant fluorescent interference which produces a curved, sloped baseline. Using Matlab, I have attempted several baseline subtraction techniques but I can't say quantitatively which is performing best - beyond a visual inspection I'm not sure how to compare them.

Is there any quantitative way of assessing the performance of a baseline subtraction? Or is visual inspection really the best we can do?

Thanks for taking the time to consider this.

Edit: I've added an example spectrum along with one with a corrected baseline.

Thank you for your suggestions as regards methods for baseline correction, however, I have already used several different methods successfully. My intention was to discover if there was a way of measuring the performance of various techniques in order to asses which is the most effective.

Thanks again for your consideration.

Edit 2: I just wish to clarify that I am not looking for methods for baseline correction. I have used several which have given me varying results - these include polynomial fitting, piece wise fitting, ModPoly, subtraction of iteratively smoothed savitzky-golay baselines, and WPLS. What I'm looking for is a way to quantify their performance, is there any metric that can be used to do this?

Answer 1

Could you also mention the correction method? Your auto-fluorescence background is really huge and your signal to noise ratio is low. One of the most cited methods in automated background correction is indeed based on polynomial fitting, albeit little but more sophisticated than just least square fitting. This article is worth reading.

Automated Method for Subtraction of Fluorescence from Biological Raman Spectra https://journals.sagepub.com/doi/abs/10.1366/000370203322554518

Another simple option is that you can choose a spectral range where to apply a polynomial baseline. Say from 0 to 1200 you apply a polynomial of degree 5, but 1200 onward you choose polynomial of degree 6. However, you have thousands of spectra and it is not clear whether your background shape is similar in each case or not.

Addendum: I am afraid there is no solid way to compare the baseline correction approaches. The reason is that you do not know the "true" model of the autofluorescence. This is also stated by Ed in comments. He is a leading spectroscopist. In such cases, you should simulate a baseline like these authors and do a comparison. Make a synthetic baseline by polynomials with say n=5, make it more complicated by mixing certain regions with polynomial n=6, 3, 2, and then try your approaches. Don't add any peaks to the baseline, and just check which technique is showing least residuals. Is there a trend in the residuals? The trend is also important. The residuals should be scattered. What technique did you use in your case?

Alternatively, you can try a known fluorescent compound, Rhodamine G, and collect a fluorescence spectrum. In this case you will know the true shape of the spectrum. Use that as a true model and try your baseline corrections.

One general advice given to us by a professor, who was an author of Raman spectroscopy monograph, was that a better and better fit does not indicate reality. The model may be completely wrong but may have a correlation coefficient close to unity. Thus fitting a polynomial or any experimental baseline correction is just a utilitarian approach, which is changing the aesthetics and cosmetic appearance of a Raman spectrum. There is nothing fundamental going on.

Answer 2

Bit late to the party but here we go:

I fully agree with @MFarooq that we don't have the reference information of how the corrected spectrum should look like to directly compare baseline correction approaches (that is, outside very special and often rather artificial scenarios).

We can say, though that the proof is in the pudding if we don't do automated baseline correction for the sake of baseline correction but rather as preprocessing step to remove a known distortion of the spectra (the background) before a predictive model. In that case, we can compare the predictive ability of the models we get based on the various baseline correction approaches.

This basically means that we treat the choice of baseline correction as a hyperparameter of the overall model. Unfortunately, the sample sizes (numbers) I've had for these biospectroscic studies were not sufficient to allow a meaningful comparison of the predictive quality which is the basis for such a hyperparameter optimization. (Many people happily do rather massive numbers of model comparisons though, being blissfully unaware that their sample size allows barely to distinguish "practically perfect, you're going to get the best classifier award and a spin-off founding stipend" from "I barely dare to show this in the group meeting" models).

Based on that, I'd recommend to

• use your spectroscopic expert knowledge in judging the baseline correction and
• be very conservative in terms of the complexity of the baseline.

Of course, if you are in the happy situation to sufficient sample numbers to allow such comparisons: go ahead.

(I think I see a bit of light at the end of the tunnel here, but am not yet fully there.)

---

That being said,

• I think it is important to realize that for many data analysis models, baseline correction isn't all that important because they can incorporate their own baseline correction: if you consider bilinear models (i.e. $\mathbf Y = \mathbf B \mathbf X), they can contain their own polynomial baseline correction. As a very easy example, consider a coefficient pattern -1/2, +1, -1/2. This pattern measures intensity at the middle position over a linear baseline through the first and last point. This corresponds to @MFarooq's observation that baseline correction is often rather cosmetic.

• In addition, in order to be good (helpful), the baseline correction must be sufficiently certain so that we have a net information gain. IMHO this is far from guaranteed: I've done very simple simulations of a "Raman band" on top of a perfectly flat background with the corresponding Poisson noise. Fitting and subtracting a linear baseline outside that band in such a situation that was a) ridiculously easy by eye in terms of saying what is signal and what is baseline, and b) using far more data points to fit the baseline than we typically do, I still observed that subtracting this fitted baseline decreased the SNR of the integral intensity of my "Raman band" easily by a factor 2 compared to subtracting the true baseline!

• My very personal opinion about the automated polynomial baselines for biological* tissues is that consider much of what I've seen in that respect overfitted: the resulting baselines are too wiggly compared to what structure I'd (not) expect in the background below the Raman spectrum, and far too wiggly compared to what I think we can afford considering the level of shot noise that 200 counts signal on 4000 counts background mean.
  Still, I do use polynomial baselines if I don't have anything better (EMSC, see below), and often find that local polynomials allow baselines with overall fewer (equivalent) degrees of freedom: they can "concentrate" curvature in one region which is sometimes useful.

  * I'm talking about biological tissues because that's where my Raman experience is. However, I do not expect other tissues (thinking cotton) to be totally different with respect to the general aspects I consider here.

• I did compare several preprocessing approaches for classification of mid-IR spectra of biological tissues by spectral region selection followed by LDA in my Diplom thesis: baseline correction didn't matter.

• Keep in mind that not all background that disturbs Raman measurements is fluorescence. While that doesn't matter for the algorithmic correction, it may offer optical ways of getting rid of some part of the background optically. That would have an additional benefit on the signal to noise ratio: the background is light, so it does lead to shot noise, and that shot noise does increase (absolutely) with the total light.
  So even if you could perfectly subtract the correct background spectrum, you'd still be left with the additional noise caused by that additional light. Thus a mathematical correction of the measured background can never be as good as (optically) avoiding that background in the first place.

  Two possibilities than in my experience can make a whole lot of a difference here are measuring in immersion (also gives more collection efficiency with higher NA) and higher confocality: both reduce stray light, higher confocality can also help against fluorescence. The former by having smaller changes in refractive index and the latter by excluding light coming from the wrong directions.

• Iff the background is fluorescence, it may be possible to bleach it by the excitation laser. If you collect spectra while doing this, you may be able to get a decent estimate of the fluorescent background (at least of the component that does bleach).

• This finally leads me to one baseline correction approach which I think stands out of the many entirely data-driven baseline fitting algorithms (sorry, I cannot keep this opinion back even though I perfectly understand that you are not asking for recommendation which baseline correction to use): EMSC describes the spectrum as being composed of polynomial or other "mathematical" terms plus reference spectra. So you can very elegantly combine polynomial baseline terms with substrate or fluorescence spectra (you can even include spectra of what should not be subtracted as baseline). The huge advantage of this approach is that it allows to incorporate far more external nowledge/information than "the baseline should be a polynomial".