2 of 9 added 468 characters in body

1. Perturbations

As already mentioned, the Jahn-Teller effect has its roots in group theory. The essence of the argument is that the energy of the compound is stabilised upon distortion to a lower-symmetry point group. This distortion may be considered to be a normal mode of vibration, with the corresponding vibrational coordinate $q$ labelling the "extent of distortion". There is one condition on the vibrational mode: it cannot transform as the totally symmetric irreducible representation of the molecular point group, as such a vibrational mode cannot bring about any distortion in the molecular geometry (it may lead to a change in equilibrium bond length, but not in the shape of the molecule). $\require{begingroup} \begingroup \newcommand{\En}[1]{E_n^{(#1)}} \newcommand{\ket}[1]{| #1 \rangle} \newcommand{\n}[1]{n^{(#1)}} \newcommand{\md}[0]{\mathrm{d}} \newcommand{\odiff}[2]{\frac{\md #1}{\md #2}}$

In the undistorted geometry (i.e. $q = 0$), the electronic Hamiltonian is denoted $H_0$. The corresponding unperturbed electronic wavefunction is $\ket{\n{0}}$, and the electronic energy is $\En{0}$. We therefore have

$$H_0 \ket{\n{0}} = \En{0}\ket{\n{0}} \tag{1}$$

Here, the Hamiltonian, wavefunction, and energy are all functions of $q$. We can expand them as Taylor series about $q = 0$:

$$\begin{align} H &= H_0 + q \left(\odiff{H}{q}\right) + \frac{q^2}{2}\left(\frac{\md^2 H}{\md q^2}\right) + \cdots \tag{2} \\ \ket{n} &= \ket{\n{0}} + q\ket{\n{1}} + \frac{q^2}{2}\ket{\n{2}} + \cdots \tag{3} \\ E_n &= \En{0} + q\En{1} + \frac{q^2}{2}\En{2} + \cdots \tag{4} \end{align}$$

In the new geometry (i.e. $q \neq 0$), the Schrodinger equation must still be obeyed and therefore

$$H\ket{n} = E_n \ket{n} \tag{5}$$

By substituting in equations $(2)$ through $(4)$ into equation $(5)$, one can compare coefficients of $q$ to reach the results:

$$\begin{align} \En{1} &= \left< \n{0} \middle| \odiff{H}{q} \middle| \n{0} \right> \tag{6} \\ \En{2} &= \left< \n{0} \middle| \frac{\md^2 H}{\md q^2} \middle| \n{0} \right> + 2\sum_{m \neq n}\frac{\left|\left<m^{(0)} \middle|(\md H/\md q)\middle|\n{0} \right>\right|^2}{\En{0} - E_m^{(0)}} \tag{7} \end{align}$$

The derivation of equations $(6)$ and $(7)$ will not be discussed further here.1

Distortions that arise from a negative value of $\En{1}$ are called first-order Jahn-Teller distortions, and distortions that arise from a negative value of $\En{2}$ are called second-order Jahn-Teller distortions.

2. The first-order Jahn-Teller effect

Recall that

$$E_n = \En{0} + q\En{1} + \cdots \tag{8}$$

Therefore, if $\En{1} > 0$, then stabilisation may be attained with a negative value of $q$; if $\En{1} < 0$, then stabilisation may be attained with a positive value of $q$. These simply represent distortions in opposite directions along a vibrational coordinate. A well-known example is the distortion of octahedral $\ce{Cu^2+}$: there are two possible choices, one involving axial compression, and one involving axial elongation. These two distortions arise from movement along the same vibrational coordinate, except that one has $q > 0$ and the other has $q < 0$.

In order for there to be a first-order Jahn-Teller distortion, we therefore require that

$$\En{1} = \left<\n{0}|(\md H/\md q)| \n{0}\right> \neq 0 \tag{9}$$

Within group theory, the condition for the integral to be nonzero is that the integrand must contain a component that transforms as the totally symmetric irreducible representation (TSIR). Mathematically,

$$\Gamma_{\text{TSIR}} \in \Gamma_n \otimes \Gamma_{(\md H/\md q)} \otimes \Gamma_n \tag{10}$$

We can simplify this slightly by noting that the Hamiltonian, $H$, itself transforms as the TSIR. Therefore, $\md H/\md q$ transforms as $\Gamma_q$, and the requirement is that

$$\Gamma_{\text{TSIR}} \in \Gamma_n \otimes \Gamma_q \otimes \Gamma_n \tag{11}$$

In all point groups, for any non-degenerate irrep $\Gamma_n$, $\Gamma_n \otimes \Gamma_n = \Gamma_{\text{TSIR}}$. Therefore, if $\Gamma_n$ is non-degenerate, then

$$\Gamma_n \otimes \Gamma_q \otimes \Gamma_n = \Gamma_q \neq \Gamma_{\text{TSIR}} \tag{12}$$

and the molecule is stable against a first-order Jahn-Teller distortion. Therefore, all closed-shell molecules ($\Gamma_n = \Gamma_{\text{TSIR}}$) do not undergo first-order Jahn-Teller distortions.

However, what happens if $\Gamma_n$ is non-degenerate? Now, the product $\Gamma_n \otimes \Gamma_n$ contains other irreps apart from the TSIR.2 If the molecule possesses a vibrational mode that transforms as one of these irreps, then the direct product $\Gamma_n \otimes \Gamma_q \otimes \Gamma_n$ will contain the TSIR.

In a rather inelegant article,3 Hermann Jahn and Edward Teller did the maths for every important point group and found that:

stability and degeneracy are not possible simultaneously unless the molecule is a linear one...

In other words, if a non-linear molecule has a degenerate ground state, then it is susceptible towards a (first-order) Jahn-Teller distortion.

Take, for example, octahedral $\ce{Cu^2+}$. This has a $\mathrm{^2E_g}$ term symbol (see this question) - which is doubly degenerate. The symmetric direct product $\mathrm{E_g \otimes E_g = A_{1g} \oplus E_g}$. Therefore, if we have a vibrational mode of symmetry $\mathrm{E_g}$, then distortion along this vibrational coordinate will occur to give a more stable compound. Recall that the vibrational mode cannot transform as the TSIR, so we can neglect the $\mathrm{A_{1g}}$ term.

What does an $\mathrm{e_g}$ vibrational mode look like? Here is a diagram:4

Vibrational modes of Oh complex

It's an axial elongation, which happens to match what we know of Cu(II). However, there is a catch. The vibrational mode is doubly degenerate (the other $\mathrm{e_g}$ mode is not shown), and any linear combination of these two degenerate vibrational modes also transforms as $\mathrm{e_g}$. Therefore, the exact form of the distortion can be any linear combination of these two degenerate modes. It can also involve negative coefficients too i.e. it might feature axial compression instead of elongation; there is no way to find that out using symmetry arguments).

On top of that, there's also no indication of how much distortion there is. That depends on (amongst other things) the value of $\En{1}$, and all we have said is that it is nonzero - we have not said how large it is.

This is what is meant by "impossible to predict the extent or the exact form of the distortion".

3. The second-order Jahn-Teller effect

Pearson has written an article on second-order Jahn-Teller effects.5 To be continued another time

Notes and references

(1) For more details, look up perturbation theory in your quantum mechanics book of choice. In such treatments, the perturbation is usually formulated slightly differently: e.g. $H$ is taken as $H_0 + \lambda V$, and the eigenstates and eigenvalues are expanded as a power series instead of a Taylor series. Notwithstanding that, the principles remain the same.

(2) There is a subtlety in that the symmetric direct product must be taken. For example, in the $D_\mathrm{\infty h}$ point group, we have $\Pi \otimes \Pi = \Sigma^+ + [\Sigma^-] + \Delta$. The antisymmetric direct product $\Sigma^-$ has to be discarded.

(3) Jahn, H. A.; Teller, E. Stability of Polyatomic Molecules in Degenerate Electronic States. I. Orbital Degeneracy. Proc. R. Soc. A 1937, 161 (905), 220–235. DOI: 10.1098/rspa.1937.0142.

(4) Albright, T. A.; Burdett, J. K.; Whangbo, M.-H. Orbital Interactions in Chemistry, 2nd ed.; Wiley: Hoboken, NJ, 2013.

(5) Pearson, R. G. The second-order Jahn-Teller effect. J. Mol. Struct.: THEOCHEM 1983, 103, 25–34. DOI: 10.1016/0166-1280(83)85006-4.