Why are excited Slater determinants used to describe electron correlation?

Question

Usually the first step in describing the electronic wave function of a molecule or atom is to describe it with a single Slater determinant. I get that this is an independent particle approximation, and that in the context of Hartree-Fock, each electron moves in the average field of the other electrons. That's fine.

But when we correlate the electrons, this is done by generating excitations out of the Slater determinant, and then taking weighted combinations of them. For example, configuration interaction mixes the ground state Slater determinant with singly, doubly, triply, etc excited determinants. A fancier way of doing this is coupled cluster, where cluster operators generate excitations out of the Slater determinant.

Hermann Kummel says that we can think about electron correlation this way:

"The first thing one may imagine happening is that two particles mutually interact, thereby lifting themselves out of the Fermi sea, so that after the interaction both are in unoccupied orbitals"

Kümmel, Hermann. "Origins of the coupled cluster method." Theoretica chimica acta 80.2-3 (1991): 81-89.

Why can we (or should we) think about electron correlation that way?

Answer 1

As I mentioned in comments, professor Jack Simons offers a relatively simple physical interpretation of the idea of mixing in excited detrminant for treating dynamical correlation in his book "Quantum Mechanics in Chemistry" (Chapter 8) freely available here, as well as in this video which is also authored by him.

---

An important mathematical finding is that a linear combination of a reference determinant and a doubly-exicted one can be expressed as linear combination of two other determinants. Namely,

\begin{multline} c_1 | \dotsc \ \phi_a \alpha \ \phi_a \beta \ \dotsc | - c_2 | \dotsc \ \phi_r \alpha \ \phi_r \beta \ \dotsc | = \\ \frac{c_1}{2} \Big( | \dotsc \ (\phi_a - c \phi_r) \alpha \ (\phi_a + c \phi_r) \beta \ \dotsc | - | \dotsc \ (\phi_a - c \phi_r) \beta \ (\phi_a + c \phi_r) \alpha \ \dotsc | \Big) \, , \end{multline} where $c = \sqrt{c_2/c_1}$.¹

Here, the determinants to the left differ by a doubly occupied spatial orbital $\phi_a$ being replaced by a doubly occupied spatial orbital $\phi_r$, while the determinants to the right describe the singlet $(\phi_a - c \phi_r)^1 (\phi_a + c \phi_r)^1$ state. So, a state created by adding the doubly-excited $| \dotsc \ \phi_r \alpha \ \phi_r \beta \ \dotsc |$ determinant to the reference $| \dotsc \ \phi_a \alpha \ \phi_a \beta \ \dotsc |$ one is equivalent to a state in which one electron occupies $\phi_a - c \phi_r$ spatial orbital (being in any spin state) while another electron occupies $\phi_a + c \phi_r$ spatial orbital (also being in any spin state). And this is the way electrons "avoid" each other: by occupying these different spatial orbitals.

---

For example, $\pi^2 \rightarrow \pi^{*2}$ configuration mixing in alkenes or $\mathrm{2s^2} \rightarrow \mathrm{2p^2}$ configuration mixing in alkaline earth atoms produce left-right polarized and top-bottom polarized spatial orbital pairs shown below.

Here one electron stays closer to the left carbon atom by occupying $\pi^2 + c \pi^{*2}$ orbital, while another avoids it staying closer to the right carbon atom by occupying $\pi^2 - c \pi^{*2}$ orbital.

In this case one electron stays closer to the top of an atom by occupying $\mathrm{2s} + c \mathrm{2p}$ orbital, while another avoids it staying closer to the bottom of the atom by occupying $\mathrm{2s} - c \mathrm{2p}$ orbital.

---

1) Here is the proof for $2 \times 2$ determinants, where for brevity $\phi_i = \phi_i \alpha$ and $\phi_i^* = \phi_i \beta$. \begin{align} c_1 | \phi_a \ \phi_a^* | - c_2 | \phi_r \ \phi_r^* | &= \frac{c_1}{2} \Big( 2 | \phi_a \ \phi_a^* | - 2 c^2 | \phi_r \ \phi_r^* | \Big) \\ &= \frac{c_1}{2} \Big( 2 (\color{red}{\phi_a \phi_a^*} - \color{green}{\phi_a^* \phi_a}) - 2 (\color{blue}{c^2 \phi_r \phi_r^*} - \color{purple}{c^2 \phi_r^* \phi_r}) \Big) \\ &= \frac{c_1}{2} \Big( 2 (\color{red}{\phi_a \phi_a^*} - \color{blue}{c^2 \phi_r \phi_r^*}) - 2 (\color{green}{\phi_a^* \phi_a} - \color{purple}{c^2 \phi_r^* \phi_r}) \Big) \\ &= \frac{c_1}{2} \Big( (\color{red}{\phi_a \phi_a^*} - \color{blue}{c^2 \phi_r \phi_r^*}) - (\color{green}{\phi_a^* \phi_a} - \color{purple}{c^2 \phi_r^* \phi_r}) \\ &\phantom{=\frac{c_1}{2}}- (\color{green}{\phi_a^* \phi_a} - \color{purple}{c^2 \phi_r^* \phi_r}) + (\color{red}{\phi_a \phi_a^*} - \color{blue}{c^2 \phi_r \phi_r^*}) \Big) \\ &= \frac{c_1}{2} \Big( (\color{red}{\phi_a \phi_a^*} + c \phi_a \phi_r^* - c \phi_r \phi_a^* - \color{blue}{c^2 \phi_r \phi_r^*}) - (\color{green}{\phi_a^* \phi_a} - c \phi_a^* \phi_r + c \phi_r^* \phi_a - \color{purple}{c^2 \phi_r^* \phi_r}) \\ &\phantom{=\frac{c_1}{2}}- (\color{green}{\phi_a^* \phi_a} + c \phi_a^* \phi_r - c \phi_r^* \phi_a - \color{purple}{c^2 \phi_r^* \phi_r}) + (\color{red}{\phi_a \phi_a^*} - c \phi_a \phi_r^* + c \phi_r \phi_a^* - \color{blue}{c^2 \phi_r \phi_r^*}) \Big) \\ &= \frac{c_1}{2} \Big( (\phi_a - c \phi_r) (\phi_a^* + c \phi_r^*) - (\phi_a^* + c \phi_r^*) (\phi_a - c \phi_r) \\ &\phantom{=\frac{c_1}{2}}- (\phi_a^* - c \phi_r^*) (\phi_a + c \phi_r) + (\phi_a + c \phi_r) (\phi_a^* - c \phi_r^* - c^2 \phi_r \phi_r^*) \Big) \\ &= \frac{c_1}{2} \Big( | (\phi_a - c \phi_r) \ (\phi_a^* + c \phi_r^*) | - | (\phi_a^* - c \phi_r^*) \ (\phi_a + c \phi_r) | \Big) \, . \end{align}

Answer 2

It perhaps isn’t a very satisfying answer, but mathematically the reason is that this is our only option!

The atomic spin-orbitals form a complete basis set: we are guaranteed to be able to represent the true wavefunction as some linear combination of them. Under Hartree-Fock, as you point out, this is a single Slater determinant. If we want to do better than this, the extra chemistry that we want to add is electron correlation. The extra maths we need to add to cope with this is more basis elements… and the way to add these is to mix in more Slater determinants. Which is why I say this is our only option: if we start with the HF ground state, and want to mix in more basis functions still, then by necessity we will have to mix in some HF excited states.

(There is one extra point that might help in understanding the Kümmel description, which is that if we could get a lower energy for the ground state by mixing in a single-electron excitation, then the HF procedure would already have done this. So we need to consider two-electron excitations, which as Kümmel says can be interpreted as interactions between these two electrons that raise them above the HF ground state.)

Answer 3

Actually there is a reason for this.

The exact mathematical way to expand a function of N variables $\Psi(x1, x2...)$ - in terms of a basis - is a multi-determinant expansion, given the requirement below.

The determinant appears "magically" when the function is required to be anti-symmetrical wrt the exchange of coordinates of any two electrons.

$$\Psi(1,2) = -\Psi(2,1)$$

See Szabo and Ostlund, formulas 2.70-2.73

---

Analogy: we can expand any well behaved, monoelectronic orbital (a single variable function) as a set of infinite basis functions, and we can expand any multi-electronic system as an infinite number of determinants.