# Derivation of Maxwell–Boltzmann statistics

I have 2 questions regarding the derivation of the formula which calculates the probability of molecules having a particular amount of kinetic energy $E_x$ in a system of $N$ molecules. It states that: $$p(E_x) = \frac{\exp(-E_x/kT)}{\sum_{i=0}^\infty \exp(-E_i/kT)}$$ This formula is initially derived from finding the maximum number of permutations $\Omega$ with $N$ molecules distributed over $n$ energy compartments within a system according to: $$\Omega = \frac{N!}{n_1!n_2!n_3!...n_i!}$$ Question 1. It is explained that one must find the maximum number of permutations by differentiating this permutation function and solving for $0$. However, I can already see from the formula that the maximum number of permutations is achieved if the denominator is equal to 1 (it can't be less), which means that there should be 1 molecule in each energy compartment $n_i$. Why isn’t it possible to reason the maximum that way? I'm aware this would need $N$ number of energy compartments resulting in some having very improbable high energy levels, but can't the difference in energy levels between the compartments be very small?

Question 2. Suppose the molecules only possess translational kinetic energy. I understand that translational kinetic energy is continuous: it does not have discrete energy levels like rotational or vibrational kinetic energy. Since it is continuous, there are no energy compartments with exact energy values. The number of possible energy compartments within such a system should therefore be infinite and one could only derive probability densities. How is it possible nonetheless to calculate a limited number of permutations using a limited number of energy compartments? Does each energy compartment cover a certain range of energy levels?

• My stat mech is rusty so I'll avoid commenting on the first question, but will just note that as long as the particle is confined, its translational energy is quantised as well (think particle in a box); just that the gaps between the energy levels are so small such that behaviour is almost classical – orthocresol May 31 '18 at 0:05

What you say is a good idea, but is not quite correct because we must maximize the distribute subject to two constraints. I will reiterate the derivation following along with McQuarrie's Statistical Mechanics.

Constraints in the Ensemble:

The constraints I mention are actually the properties that determine the ensemble we are working with:

$$\sum_ja_j=A$$ $$\sum_j\epsilon_ja_j=E$$ The first constraint says that sum over all states of the population numbers for each state $j$, given by $a_j$, must equal the total number of members in the system $A$. This is the number of molecules if you like.

The second constraint says that the energy of each state, $\epsilon_j$, multiplied by the population of that state, $a_j$, must equal the total energy, $E$, of the system when the sum is taken over all $j$. These constraints are so intuitive physically that it is easy to forget they even have to be enforced.

By the way, we are not actually constraining the total energy. We only constrain that whatever the energy is, it can be found from the second constraint. In fact, in McQuarrie, this derivation is undertaken for the canonical (NVT) ensemble.

Method of the Most Probable Distribution:

Now, as you say, the distribution which we primarily observe will be the one which maximizes the number of ways to arrange the members of the ensemble. Notice I say primarily observe. This is discussed below. This number of ways is given by the equation you write down, $$W(\mathbf{a})=\frac{A!}{\prod_ka_k!}$$

There are many possible distributions that satisfy the constraints we have given. Given this indeterminate (but probably infinite) number of ensembles which satisfy our constraints, the probability of being found in state $j$ is given by, $$P_j=\frac{\bar{a}_j}{A}=\frac{1}{A}\frac{\sum_{\mathbf{a}}a_j(\mathbf{a})W(\mathbf{a})}{\sum_{\mathbf{a}}W(\mathbf{a})}$$ Notice that we are now summing over the vector of all the quantum states for each ensemble, $\mathbf{a}$, because we want to find $\bar{a}_j$, the average population of quantum state $j$ in each different ensemble satisfying our constraints.

This is clearly not very useful to us because there are so many possible ensembles we will never be able to do something with them. We can, however, take advantage of the form of $W(\mathbf{a})$. Namely, by just playing around with the numbers, you will find that the multinomial coefficients are very strongly peaked at their maximum when all of the $a_j$'s are large. Fortunately, this is exactly the regime we live in because we are thinking about something on the order of $10^{23}$ particles, so the population of states will be very large.

This means we can make an approximation, which is formally exact in the limit as $A\rightarrow\infty$, that $\bar{a}_j/A=a_j^*/A$ where $a_j^*$ is the $a_j$ in the distribution which maximizes $W(\mathbf{a})$. So, the fact we are going to maximize this distribution comes about through an approximation, albeit one which becomes exact in the limit near where we are working.

So, we can simplify our equation for the probabilities to, $$P_j=\frac{1}{A}\frac{a_j^*W(\mathbf{a^*})}{W(\mathbf{a^*})}=\frac{a^*_j}{A}$$

Method of Lagrange Multiplies:

We want to find $a_j^*$ which means we must perform a maximization subject to the two constraints we listed at the beginning. Maximizing something subject to constraints does not seem to be easy, but there is a very nice method for doing it called the method of Lagrange multipliers. Essentially, we introduce some undetermined coefficients to the maximization equation so that the constraints are included, but in reality we are just adding zero to the equation.

What I mean by this is that I can rewrite the constraints as follows: $$\alpha\left(\sum_ja_j-A\right)=0$$ $$\beta\left(\sum_j\epsilon_ja_j-E\right)=0$$ where $\alpha$ and $\beta$ are the so-called Lagrange multipliers. Then, we can simply maximize our distribution as normal: $$\frac{\partial}{\partial a_k}\left(\ln W(\mathbf{a^*})-\alpha\left(\sum_ja_j-A\right)-\beta\left(\sum_j\epsilon_ja_j-E\right)\right)=0$$ It really isn't obvious that this should maximize the system while satisfying the constraints, so you will have to read elsewhere why this works, but the fact is that it does work.

Notice, I have actually played a trick. Instead of maximizing $W(\mathbf{a^*})$, I am maximizing $\ln W(\mathbf{a^*})$. This is because $W(\mathbf{a^*})$ has factorials in it, so by taking the logarithm we can employ Stirling's approximation. This is also more or less exact in the limit of a large number of particles, which we always satisfy. The reason we can do this is because the logarithm of a function will have a maximum at the same coordinate as the function itself and the logarithm is smooth such that will not somehow create another maximum or something which would spoil the trick.

After taking the derivative, we lose the constant and every term in the sum where $k\ne j$. So, we have, $$-\ln a_j^*-\alpha-1-\beta E_j=0$$ Or, after rearranging and solving for $a_j^*$: $$a_j^*=e^{-(\alpha+1)}e^{-\beta E_j}$$

Putting it All Together:

So, all that remains is to solve for the undetermined mutlipliers and plug it all back into our probability statement, $P_j=a_J^*/A$.

This can be done very simply (and we actually don't care about the mutliplier $\alpha$). If we take the sum over all $j$ of the equation we wrote for $a_j^*$, we realize that the left side is equal to $A$ due to our first constraint and we can rearrange to write $\alpha$ in terms of $\beta$: $$e^{\alpha+1}=\frac{1}{A}\sum_je^{-\beta E_j}$$ Then, we can find the final probabilities from, $$P_j=\frac{a_j^*}{A}=\frac{e^{-(\alpha+1)}e^{-\beta E_j}}{A}=\frac{A}{\sum_je^{-\beta E_j}}\frac{e^{-\beta E_j}}{A}$$

We finally arrive at the final probability distribution which defines Boltzmann statistics:

$$P_j=\frac{e^{-\beta E_j}}{\sum_je^{-\beta E_j}}$$

Thus, the only one of these Lagrange multipliers we care about is $\beta$. We have not actually found what this multipliers is yet, but I will stop here as it should be no surprise that the multiplier is given by $\beta=1/k_bT$.

So, I believe the above answers your question 1. That is, you have to worry about the constraints associated with you ensemble so what you mention does not quite work.

As to your second question, it is not quite true that translational energy is continuous. Pretty much everyone pretends it is, but kinetic energy is quantized. It is, however, very rare that the boundaries a particle is confined to are sufficiently small (and impermeable) that quantization of the kinetic energy is noticeable. Also, because there are many of these states which are closely spaced, one must reach very low temperatures for the quantization to be important.

What I have described above is the origin of Boltzmann statistics (not Maxwell-Boltzmann as the title says). This is a common confusion because of the famous Maxwell-Boltzmann distribution of velocities. I only bring this up because the Maxwell-Boltzmann distribution of velocities makes exactly the approximation which you describe by pretending the kinetic energies are continuous.

That is, if we simply plug the kinetic energy of a particle $E=1/2mv_x^2$ into our final result (and note the deonominator is the partition function), we get,

$$P_{v_x}=\frac{e^{-\frac{mv_x^2}{2kT}}}{Q}$$ Then, the partition function, $Q$, is just a normalization constant. Because the states of different kinetic energies are so closely spaced, we can elevate $P_{v_x}$ to a function, $P(v_x)$, and integrate over all velocities to determine the probability of finding a particle with a velocity between $v_x$ and $v_x+dv_x$. I will leave this as an exercise. Notice this is just an integral over a gaussian function.

• Thank you so much for your detailed explanation. Is there a way to prove or reason that having 1 molecule in each energy compartment violates your mentioned constraints? Perhaps it's because there are discrete energy levels which forces such a large amount of compartments to exceed the maximum amount of allowed total energy? – JohnnyGui May 31 '18 at 13:05
• Ah. I think I understand now. What you have said assumes that each compartment gives an equal contribution to the energy which is not in fact the case. If one distributes the particles equally, there are only N (or N! if the particles are distinguishable) ways to do this. In reality, however, the bins have different energies associated with them so one has to find the balance between putting particles in lower energy bins and still allowing for many possible rearrangements. Also, you seem to say the number of states and particles are the same which need not be the case. – jheindel May 31 '18 at 18:07
• Yes, this is indeed what I meant; each bin containing one molecule with a different energy. I thought this would also give the maximum number of permutations (denominator is at its minimum at 1) while not violating your mentioned constraints. But by the way you're explaining it, an amount of N! possible ways is actually NOT the maximum amount? Perhaps I'm misunderstanding the way of how the permutations are meant to be imagined then. – JohnnyGui May 31 '18 at 20:10
• I believe what you have written does satisfy the constraints but we have a parameter to worry about which is the temperature of the system. As it stands, you have not said anything about the probability of occupying a particular state given a value of the temperature. Once you take this into consideration, you get the result I have given above which is that lower temperatures make low energy states more likely. The Boltzmann distribution adopts the configuration you suggest as $T/rightarrow/infty$ and all states become equally likely to occupy. – jheindel Jun 1 '18 at 1:21
• Sorry, I'm still not sure I understand how it is initally decided that it is impossible for each energy bin to contain only one molecule at a low temperature, which sounds possible to me if the energy bins don't differ very much from each other (very narrow energy range). Does there exist an amount of permutations higher than N!? – JohnnyGui Jun 1 '18 at 19:09

(1) You have to maximize $\Omega$ subject to the constraints that the total number of particles and the total energy is constant, which is more complicated than just taking the derivative and setting it equal to zero. You can do it using Lagrange multipliers or (which is more typical in an undergraduate course) move to the canonical ensemble where the requirement of constant energy can be relaxed.

(2) Translational kinetic energy is only continuous in the limit of a particle that has amplitude over the entire universe. For any finite-sized container, the energy eigenstates are quantized. Mind you, they are very closely spaced. Nevertheless, you'll find that the translational partition function contains the volume of the container, and in the limit of an infinite container goes to infinity. Fortunately, Z itself is only of theoretical interest, any practical calculation requires a derivative of Z, and these are all nice and finite.

• Thanks for the explanation. Regarding 1), I can't see how having 1 molecule in each compartment would violate the contraints. Is there a reason or mathematical proof for this? Or is N! not the maximum amount of possible permutations? – JohnnyGui Jun 2 '18 at 23:06
• That depends on the number of compartments, and on the energy of a particle in each. In other words, your approach will work for one specific state, but is not generalizable to all similar states in the microcanonical ensemble. To get the general answer, you need to either use the method of Lagrange multipliers or shift to the canonical ensemble where the problem is easier. – Christopher Grayce Jun 3 '18 at 20:30
• Ok, I thought the whole purpose was to find one specific state (not several) that can have the highest amount of permutations regarding which molecule can be in which energy bin, while not violating the mentioned constraints. – JohnnyGui Jun 6 '18 at 20:06

I think I now know what my misunderstanding was.

What I did not realise is that, to have a certain microstate, the system must have a certain total energy U and thus a certain temperature. Therefore, the maximum number of possible ways $Ω$ is a function of the temperature. In other words, each temperature has its own maximum number of possible ways $Ω$.

If I solve the equation for 1 particle per energylevel, then I basically have already decided which microstate the system is in and which temperature it has. I have not given the formula freedom to deploy itself as a function of temperature.

Please correct me if my reasoning is wrong.