Assuming the Van der WaalWaals equation with $b=0$, I agree with the solution to the equation:
$$\mathrm{PV}_m^2 - 24\mathrm{V}_m + 2 = 0\tag{1}$$$$pV_\mathrm{m}^2 - 24V_\mathrm{m} + 2 = 0\label{eqn:1}\tag{1}$$
I'll point out that I'm following the notation of the gievn solution, but $\mathrm{V}_m$ seems odd to me. I'd think that $\mathrm{V}_m$ would be reserved to mean the molar volume at STP.
I'll point out that I'm following the notation of the given solution, but $V_\mathrm{m}$ seems odd to me. I'd think that $V_\mathrm{m}$ would be reserved to mean the molar volume at STP.
But equation $(1)$\eqref{eqn:1} has two unknowns, $\mathrm{P}$$p$ and $\mathrm{V}_m$$V_\mathrm{m}$. Letting $a' = P$$a' = p$, $b' = -24$, and $c' = 2$ then the appropriate quadratic equation is of course
\begin{alignat}2\mathrm{V}_m &= \dfrac{-b' \pm\sqrt{b'^2 - 4a'c'}}{2a'}\tag{2}\\\mathrm{V}_m &= \dfrac{-(-24) \pm\sqrt{(-24)^2 - 4(\mathrm{P})(2)}}{2(\mathrm{P})}\tag{3}\\\mathrm{V}_m &= \dfrac{24 \pm\sqrt{(-24)^2 - 8\mathrm{P}}}{2\mathrm{P}}\tag{4}\\\mathrm{V}_m &= \dfrac{12 \pm\sqrt{144 - 2\mathrm{P}}}{\mathrm{P}}\tag{5}\end{alignat}\begin{alignat} 2V_\mathrm{m} &= \dfrac{-b' \pm\sqrt{b'^2 - 4a'c'}}{2a'}\tag{2}\\ V_\mathrm{m} &= \dfrac{-(-24) \pm\sqrt{(-24)^2 - 4×p×2}}{2p}\tag{3}\\ V_\mathrm{m} &= \dfrac{24 \pm\sqrt{(-24)^2 - 8p}}{2p}\tag{4}\\ V_\mathrm{m} &= \dfrac{12 \pm\sqrt{144 - 2p}}{p}\label{eqn:5}\tag{5} \end{alignat}
So we have two unknowns and one equation. Thus the problem is unsolvable without additional information. Notice that would still be the case if the simple ideal gas equation, $PV=nRT$, had been used. The ideal gas equation could be solved only to $\mathrm{PV}_m=24$$pV_\mathrm{m}=24$.
For equation $(5)$\eqref{eqn:5} there are three cases for the sqrt term $\sqrt{144 - 2\mathrm{P}}$$\sqrt{144 - 2p}$:
(1) If the term is negative:
- The term is negative.
If $144 - 2\mathrm{P}$$144 - 2p$ is negative, then both roots would be imaginary.
(2) If the sqrt term is equal to zero:
- The term is equal to zero.
We have $\sqrt{144 - 2\mathrm{P}} = 0\implies144 - 2\mathrm{P} = 0$$\sqrt{144 - 2p} = 0\implies 144 - 2p = 0,$ so $\mathrm{P} = 72$$p = 72$. Therefore, $\mathrm{V}_m = \dfrac{12}{72} = 0.167$.$V_\mathrm{m} = 12/72 = 0.167.$
(3) If the sqrt term is greater than 0:,
- The term is greater than zero.
Then $72 > P$$72 > p$ also, and mathematically $\mathrm{V}_m$$V_\mathrm{m}$ has two roots.
Now, again there are three possibilities.
(a) $\sqrt{144 - 2\mathrm{P}} < 12$$\sqrt{144 - 2p} < 12$
This is impossible since $\mathrm{V_m}$$V_\mathrm{m}$ would have two valid values.
(b) $\sqrt{144 - 2\mathrm{P}} = 12$$\sqrt{144 - 2p} = 12$
This is impossible since $\mathrm{V_m} =0$$V_\mathrm{m} = 0$ and the gas volume can't be $0$.
(c) $\sqrt{144 - 2\mathrm{P}} > 12$$\sqrt{144 - 2p} > 12$
This would yield one negative value and one positive value for $\mathrm{V_m}$$V_\mathrm{m}$, which is okOK for $\mathrm{V_m}$$V_\mathrm{m}$. However it also means that $\mathrm{P} < 0$$p < 0,$ which is nonsensical.
Thus the only reasonable solution is if the square root term is equal to zero.
EDIT -P.S. The comment by user Poutnikuser Poutnik made me look at this again, and I now understand the book's solution...solution…