This is sorta a cross-post from my post in StackOverflow. Although, rather than seeking help with the code I'm here asking for help with a chemical/statistical problem associated with the code.

I'm trying to cluster infrared (IR) spectra so that spectra most similar to each other are grouped and processed differently. To do this I am implementing a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm on a 2D set of data representing the spectra. Since the IR spectra clusters need to have similar shapes and vertical height (the substance is absorbing the same amount of light for spectra in the cluster) I have adapted a 2D system with the following parameters

The mean of the spectra (the average %Transmission, this should correlate to the vertical height) and the mean of the first derivative of the spectra (this should correlate to the shape). These two parameters then give a 2D system that I can use DBSCAN to cluster the IR spectra, it works but it does not cluster them perfectly. Note that the DBSCAN algorithm typically clusters scatter plot points through their Euclidean distance from each other. This is why I have the two parameters which you can imagine being plotted on an X(mean) and Y(mean of derivative) plane, for the clusters algorithm to then cluster them.

I can already make logical sense of the fact that the average of the derivative will not be an accurate measure of the shape of the spectral line, but I can not figure out better method. How can I get a better parameter for representing the shape of a spectral line?

  • $\begingroup$ Can you give an example of such spectra, and how you expect them to differ? $\endgroup$ – Karl Nov 8 '20 at 20:50
  • 2
    $\begingroup$ Hey @Karl, this is tough since I am a little nervous of posting an image of the spectra as they are unpublished research. Would it help if I give an example using random made up spectra for the purpose of illustrating what I'm doing? $\endgroup$ – Cavenfish Nov 8 '20 at 21:09
  • 1
    $\begingroup$ Isn't there an established way to really compare the spectra for all their features? In my opinion you should ask also in Maths. Basically you want to group the plot of similar xy functions, they should know. The fact that the plot is actually a spectrum does not change anything. $\endgroup$ – Alchimista Nov 9 '20 at 8:08
  • $\begingroup$ Hey @Alchimista, I haven't seen any established ways of doing it yet but they probably exist. I will post this in Maths StackExchange as well, thankyou! $\endgroup$ – Cavenfish Nov 9 '20 at 18:44
  • $\begingroup$ @Karl I have added a minimum working example in the StackOverflow post that might better illustrate what I'm asking for help with. $\endgroup$ – Cavenfish Nov 9 '20 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.