How are London Dispersion Forces generated?

Question

While talking about the gaseous state of matter we came to the topic of London Dispersion Forces caused by the generation of a dipole in one atom which induces a dipole in another. While talking about the cause of such interactions I stated that, since Schrödinger's equation for multi-electron atoms is time-dependent, at some moment a dipole may get generated (due to the increase in the probability of electrons in that region). Though now I am thinking that what I said was quite vague and may not be the reason for it.

So what is the reason for the generation of dipole moment in one atom to cause London forces?

Answer 1

Let me crash the party here.

TL;DR: The classical explanation of induced dipole attractions from electron densities "evading" each other does, by itself, not adequately or intuitively explain the actual charge density patterns that arise in these situations.

(I know that this must seem like an outrageous statement, and surely downvote fingers are itching now. Bear with me.)

---

Based on the Hellmann-Feynman theorem, it is known that the forces acting on a nucleus arise from two purely coulombic sources: Its attraction to its surrounding electron distribution, and its repulsion with other nuclei. Hence, the observation that e.g. the two atoms in a rare gas dimer are attracted to each other immediately implies that there is a concentration of electron density in between the nuclei, so that the resulting net forces pulls them "inwards" towards each other. As Feynman put it himself in 1939 (emphasis in the original):

... the charge distribution of each is distorted from central symmetry, a dipole moment of order $1/R^{7}$ being induced in each atom. The negative charge distribution of each atom has its center of gravity moved slightly toward the other. It is not the interaction of these dipoles which leads to van der Waals' force, but rather the attraction of each nucleus for the distorted charge distribution of its own electrons that gives the attractive $1/R^{7}$ force.

This is the complete opposite picture of the momentary effects in the induced-dipole explanation, where the electron densities "evade" each other by simultaneous displacements in the same direction to create attractive dipole interactions. The Feynman explanation is not very popular, but there are in fact some authors who have picked it up and commented on it, most notably perhaps Richard Bader (of Atoms-In-Molecules fame). Politzer and Murray have written a nice article on the topic.

---

But, is Feynman actually correct? If so, then we should be able to observe some accumulation of charge density between two neutral atoms that are bound by dispersion interactions, right? Indeed we do. The below image comes from a publication by Thonhauser et al. and shows the difference in charge density that arises when the dispersion interaction between to Ar atoms is "switched on" (highlight mine):

So, this may hold true for atoms, but what about entire molecules? Luckily, Hunt has shown in a very laborious 1990 paper that Feynman's picture holds true in that case as well:

... Feynman's statement concerning forces between well-separated atoms in S states generalizes to interacting molecules A and B of arbitrary symmetry (with A and B in their ground electronic states). To leading order, the dispersion force on each nucleus I in molecule A results from the attraction of I to the dispersion-induced change in polarization of the electronic charge cloud on molecule A itself.

---

Obviously, the "inwards polarization" effect must seem counter-intuitive at first. Why would the negatively charged electron clouds actually want to approach instead of evade each other? Thankfully, a straightforward rationalization of this effect for a rare gas dimer comes from another paper by Clark, Murray and Politzer:

What causes the polarization of the electronic densities toward each other (...)? This can be easily understood when it is noted that the electrostatic potential produced by the nucleus and electrons of any free ground-state atom is positive everywhere; the effect of the nucleus dominates over that of the dispersed electrons. This positive potential is what each atom “sees” of the other atom (...).

Of course, the astute reader may also voice another point of protest: "The fluctuating dipoles are variable in time, whereas Feynman's deformed charge density explanation is entirely static. How do we even compare the two? And what is the effect of the fluctuating dipoles when averaged over time?"

As it turns out, the two explanations are apparently consistent with each other, as detailed by several authors; Hunt herself in the paper mentioned above acknowledges fluctuating dipoles as a possible starting point, and a paper by Clark dedicates a full paragraph to the seeming dichotomy of the two pictures. At its core, however, the suggestive "electron-evading" nature of this explanation is very much misleading in light of the observable static effects of the "inwards" charge redistributions -- which, again, are actually required to create the resulting attractve interactions.

Answer 2

You are correct that simply stating that the Schrodinger equation is time dependent does not really answer the question. In fact, time-dependence is not a component of the derivation of the dispersion forces. At a surface level, they just represent how much the ground state energy of a system of polarizable molecules is lowered by their interactions. It's the same idea as any of the other intermolecular forces (coulomb, dipole-dipole, etc.) where some interaction between particles leads to a lower energy state than the free particles.

Generally, the London Dispersion interaction potential is not derived in a general chemistry class because it is purely quantum mechanical and rather involved. These slides provide a clear and thorough (albeit advanced) derivation, but I will explain the main points below.

Derivation Explanation

Let's imagine for this model that a polarizable atoms can be thought of as a simple harmonic oscillator with a positive charge ($+q$) on one end of a spring and a negative charge ($-q$) on the other end. The spring has an equilibrium length of 0, so that the atom has zero average dipole. This model of an polarizable atom captures the physics we want because applying a force of some kind to the spring will induce a dipole in this atom ($\vec{p} = q \vec{d}$).

So far we have only considered one of these atoms. What if we move to consider multiple of them together? Remember we're really interested in the energy difference between two non-interacting atoms and two interacting atoms, so let's start by finding the non-interacting ground state energy of two atoms. We'll let the two atoms have two different spring frequencies $\omega_1$ and $\omega_2$ respectively.

Setting $z_1$ and $z_2$ as the displacements of the springs from equilibrium for atoms 1 and 2 respectively and $m$ as the mass of an atom, we can write the total potential energy of the system as: $$ U_{\text{non-interact}} = \frac{1}{2}\left(m \omega_1^2z_1^2 + m \omega_2^2z_2^2\right) $$

Solving the Schrodinger equation for this potential yields the ground state energy for the two non-interacting harmonic oscillators: $$ E = \frac{\hbar}{2} (\omega_1 + \omega_2) $$

Now, we turn on the interaction and the problem becomes harder more interesting because the positions of the springs affect the magnitude of the interaction. Physically, because the dipole-dipole interaction can lower the energy, we expect that the interacting ground state of the two atoms will be a balance between minimizing the stretching of the springs (which increases the energy) and maximizing the amount of dipole-dipole interaction (which lowers the energy).

Setting $z$ as the distance between the two atoms, with some algebra one can show that the interaction energy between these two dipoles is given by $$ U_{\text{interact}}(z,z_1,z_2) = \frac{q^2}{2\pi\epsilon_0}\frac{z_1 z_2}{z^3} $$

Now, our total potential energy for the system is given by: $$ U_{\text{total}}(z,z_1,z_2) = U_{\text{non-interact}}(z_1,z_2) + U_{\text{interact}}(z,z_1,z_2) $$ While intimidating looking, this potential can also be solved exactly by changing variables to $z_+ = (z_1 + z_2)$ and $z_- = (z_1 - z_2$) and rearranging terms to make it look like the non-interacting potential energy. Once that is done the ground state energy can be found just as easily as in the non-interacting case.

With this done, we can find that the (one dimensional) energy stabilization is given by: $$ U(z) = \frac{-\alpha^2 I}{2}\frac{q^2}{(4\pi\epsilon_0)^2}\frac{1}{z^6} $$ Where the final step was to replace some of our less measurable parameters with their atomic counterparts: polarizability ($\alpha$) and ionization potential ($I$)