In spectroscopy (and spectrophotometry), the position of a peak/trough is defined by the location of the tip of the peak or trough. This is why you see such precise values given for their locations.
The ranges you refer to (e.g., $3200$ to $3750\pu{ cm^{-1}}$ for the $\ce{-OH}$ stretch) are the ranges of wavenumber values where the tip of the relevant peak/trough is usually found, and not the range of values over which the peak/trough spreads.
Taking IR spectroscopy as representative, since that's the technique you're asking about, the following is from LibreTexts:
[This equation] gives the frequency of light that a molecule will absorb, and gives the frequency of vibration of the normal mode excited by that light.
$$\nu={1\over2\pi}\left({k\over\mu}\right)^{1\over 2} \\ \nu = \text{ frequency in cm}^{-1} \\ k = \text{ force constant in N/cm} \\ \mu = \text{ reduced mass in kg}$$
Thus, in theory a given vibration corresponds to an exact, specific wavelength of light that would be absorbed when recording an IR spectrum. However, due to effects like thermal excitation and solvation, we don't observe "infinitely narrow" absorption lines, but instead peaks of finite width. From that same LibreTexts reference:
In general, the width of infrared bands for solid and liquid samples is determined by the number of chemical environments which is related to the strength of intermolecular interactions such as hydrogen bonding. Figure 1 shows hydrogen bond in water molecules and these water molecules are in different chemical environments. Because the number and strength of hydrogen bonds differs with chemical environment, the force constant varies and the wavenumber differs at which these molecules absorb infrared light.
In any sample where hydrogen bonding occurs, the number and strength of intermolecular interactions varies greatly within the sample, causing the bands in these samples to be particularly broad. This is illustrated in the spectra of ethanol and hexanoic acid. When intermolecular interactions are weak, the number of chemical environments is small, and narrow infrared bands are observed.
Thus, if not for environmental effects, every absorption peak would be an absorption line. But, environmental effects cause the absorption to "spread" around that theoretical value. The peak is still considered to be located at that central point, though.
This helps to explain why, for example, the $\ce{-OH}$ stretch peak/band tends to be extraordinarily broad (lots of local variations in the hydrogen bonding environment), whereas the carbonyl $\ce{>C=O}$ stretch is often quite narrow (fewer environmental effects are able to perturb its frequency).