Can density functional approximations including dispersion correction perform worse than without it?

Question

I am trying correlate some experimental properties of a molecule to its calculated equilibrium geometry. To do so, I calculated the geometry and energy of some conformers. Now, I obtain mixed results when performing the same energetic calculation with dispersion-corrected (D3) and not corrected functional (B3LYP): when no correction is applied, a conformer has lower energy, while with D3 correction another conformer has lower energy. The idea of using D3 looked reasonable, as the molecule has two aromatic and two aliphatic wings, which from the D3-uncorrected geometry seemed to promote to some degree an aliphatic-aromatic stacking interaction.

The point is: my intuition drove me to the use of the dispersion correction, and indeed the D3-corrected result would agree wonderfully with spectroscopic results. But is there a case in which a D3-uncorrected result would be more accurate than the corrected one? Or, alternatively, is there a way to understand quantitatively if the uncorrected model does not describe correctly the system?

I saw a lot of reviews talking about the "benefits" of DFT-D3, but could it do worse than DFT alone?

Edit, to answer Marvin's questions: Yes, the molecule is organic. It all started with a "routine calculation set-up": b3lyp/6-31g*, but it later became evident that more refinement was required, so the current level might be seen as an expansion of the original calculation. Beyond the 6-31+g** level, I don't stress on Pople's basis anymore, and if more refinements will be required, Alrichs basis sets will be employed. Yes, I omitted it, but B-J damping was employed. At the moment I have optimized one structure with D3(BJ), the other one is being optimized. Of both the geometries optimized with B3LYP, the D3BJ-energy was calculated, together with the uncorrected one. The two molecules arise indeed from a semiempirical conformational search, followed by a fast and inaccurate hf/3-21g* optimization on 100 geometries. A dozen of low energy conformers was optimized with hf/6-31g*, and the two outliers/low lying conformers were subject to further study with the methods discussed. The molecule contains 90 atoms, and the calculated energy difference is less than 0.5 kcal/mol, but as NMR experiments show a net predominance of one geometry, it's likely that the low energy difference is coupled with an high isomerization barrier, which I still haven't calculated: at the moment I am focusing on the equilibrium geometry. As soon as the "new" geometry calculations will complete, I will probably do more calculations with a functional better suited for dispersion corrections, thank you very much!

Note: an NMR calculation is planned, but as good results require time and a strong basis set, it will probably be the last thing I will do, starting with better geometries.

Answer 1

tldr; I'll check the detailed results, but the overall answer is that dispersion correction is absolutely worth it

We just finished a benchmark over ~700 molecules with up to 10 conformers, so ~6700 single points at the DLPNO-CCSD(T) / def2-TZVP level. I will post a link to the preprint when it's available soon.

In the meantime, here's a quick summary for the effects of dispersion corrections:

Comparing un-corrected PBE, B3LYP, and ωB97X single-point energies to DLPNO-CCSD(T) illustrate a significant effect. The uncorrected median R² values drop by ~0.12, and the median Spearman correlations drop by ~0.08. For example, the median R² of B3LYP / TZ drops from 0.920 to 0.706 without the D3BJ dispersion correction.

The metric here is the correlation between a method (i.e. B3LYP) with the DLPNO-CCSD(T) energies.

I'll look to see if there's ever a case where the dispersion correction does worse. But there are many, many papers illustrating that conformer energetics are dominated by non-bonded interactions .. which are improved with dispersion-corrected density functional methods.

Your methodology seems solid, although if you want good energy evaluations of different conformers, I'd consider either RI-MP2 or DLPNO-CCSD(T) single-points. Our timings in the paper show that RI-MP2 can be about as fast as hybrid DFT methods, but more accurate at ranking conformers.