Is it true that the better the trial function, the better approximation for the energy in variational principle?

Question

While studying the variational principle in McQuarrie's Quantum Chemistry, I came across the following problem: to relate the difference between an approximation $\phi$ and the exact ground-state wave function $\psi_0$ with the difference between the approximate energy $E=\int\phi^* \hat H \phi \, \text{d} \tau$ and the exact energy $E_0=\int\psi_0^* \hat H \psi_0 \, \text{d} \tau$. For instance, if I know that $\phi$ and $\psi_0$ differ by at most, say, $O(\epsilon)$ over all space, can I say that $E-E_0 \approx O(\epsilon)$?

It is intuitive to think that the better approximation $\phi$ is, the better the energy approximation gets, but I cannot prove this.

Without loss of generality, suppose $\phi$ and $\psi_0$ normalized. By writing $\phi$ as a linear combination of $\psi_n$, we get $$ \phi=\sum_{n=0}^{\infty} c_n\psi_n $$ such that $\sum_{n=0}^{\infty} c_n^* c_n=1$.

Therefore $$ \hat H \phi=\sum_{n=0}^{\infty} c_n\hat H\psi_n = \sum_{n=0}^{\infty} c_nE_n\psi_n $$

Hence $E=\int\phi^* \hat H \phi \, \text{d} \tau=\sum_{n=0}^{\infty} c_n^* c_nE_n$.

From here, the author shows that $E \geq E_0$, but how can I estimate the difference between the approximate energy and the exact energy?

Answer 1

TL;DR

No this is not true. Although a "better approximation" might yield a lower energy, this is not always the case.

Full answer

There is a major issue in the formulation of your problem: You want

to relate the difference between an approximation $\phi$ and the exact ground-state wave function ψ0 with the difference between the approximate energy $E=\int\phi^* H\phi \mathrm{d}\tau$ and the exact energy

The second part of that statement includes a clear mathematical definition of the target property (energy $E=\int\phi^* H\phi \mathrm{d}\tau$). However, there is no clear definition of what a better approximation of the wave function actually is. How do you want to define or measure it, so it can be expressed in a mathematical way?

There a various options to define such a measure, which would allow comparing two trial wave functions and judge their quality. The most obvious and commonly used one is the total energy. But this is the property you want to compare with, so we need something else. Examples for further options are:

• overlap with the exact wave function (if available) $\langle \phi|\psi_0\rangle$
• expectation values other than total energy, e.g. $\langle r\rangle$ or $\langle r^2\rangle$
• any molecular properties of the chosen model system (e.g. dipole moment, excitation energies, etc.)

The next problem is: the choice of the measure for a good approximation might influence the result. For example the overlap $S_i=\langle \phi_i|\psi_0\rangle$ for two different trial wave functions $\phi_1$ and $\phi_2$ (possibly each with a different variational ansatz) might yield $S_1 > S_2$ (meaning $\phi_1$ would be the better approximation), while the total energy of the second trial wave function might be lower (and therefore close to the true ground state): $E_2 < E_1$.

The different measures on how good of an approximation a given trial wave function is do not correlate, therefore there is no general answer which one is actually better. It depends on the property you want to calculate.

There is a even paper available covering this issue and including a more detailed example. Sadly, it is paywalled although it is primarily intended for educational purposes: https://pubs.acs.org/doi/10.1021/acs.jchemed.8b00959

how can I estimate the difference between the approximate energy and the exact energy?

In essence, this is the central question quantum chemistry is concerned about. And the field developed a whole lot of different methods to achieve reasonable approximations. Thus, I am afraid there is no simple answer to this question.

Answer 2

Disclaimer: As Feodoran has correctly pointed out in his answer, a trail function $\phi$ with a bigger overlap $\langle \phi|\psi_0\rangle$, does not necessarily yield a better approximation for the ground state energy $E_0$. I will leave my answer nonetheless, since I believe it can help to understand the variational principle.

Let us assume a wave function $\phi$ describes a state for our system of interest. For simplification, we assume that $\phi$ is normalized. In order to determine the properties our system will have when it is in the state $\phi$, we apply the corresponding operator. For the total energy $E$, it is the Hamiltonian $\hat{H}$. The problem in QM is that in contrast to classical mechanics we don't necessarily get a single value for the state, but a discrete distribution of values. The possible values are the eigenvalues of the operator $E_n$. The square of the overlap (dot product) between $\phi$ and the corresponding eigenfunction $\psi_n$ gives the probability $p_n$, which is the probability for $E_n$ to be observed:

$$p_n = c_n^* c_n =\langle\phi|\psi_n\rangle\langle\psi_n|\phi\rangle$$

The last relation essentially already answers your question, since it directly connects the probability of observing $E_n$ with how close $\phi$ is to $\psi_n$. The dot product serves as the measure of similarity. In the special case where the state $\phi$ coincides with $\psi_n$, the two functions fully overlap and, due to orthogonality of the eigenfunctions, has no overlap with the other eigenfunctions $\psi_{m \neq n}$. Hence, $E_n$ is 100% observed.

We cannot calculate the $p_n$'s without knowing the eigenfunctions, but we can calculate the expectation value $\langle E \rangle$:

$$\langle E \rangle = \langle\phi|\hat{H}|\phi\rangle = \sum_n p_nE_n$$

and we have of course $\langle E \rangle \geq E_0$. The better $\phi$ is an approximation for $\psi_0$, the larger the overlap is and the more probable $E_0$ becomes. Hence, minimizing $\langle E \rangle$ by varying $\phi$ will gives us the best possible approximation to $E_0$, which is the entire premise of variational theory.