This is just a guess... But I wonder if this is due to entropic considerations. In the equation you have written, there are two molecules of gas getting converted to three molecules of gas. $\Delta S$ is positive, so it decreases $\Delta G$ as the reaction temperature increases. However, in the reaction $\ce{(CN)2 + 2 O2 = 2 CO2 + N2}$ you have an equal number of gas molecules, so the entropic component of $\Delta G$ will be small.
I'm not really sure why that would matter. At very high temps, the first reaction will presumably have a very large, negative free energy. Of course, the $\Delta G$ for the $\ce{CO2}$ reaction is most likely negative, so you might have to control the feed stoichiometry carefully to ensure that you are getting only the first reaction.
EDIT:
I got the $\Delta G$ values from the NIST-JANAF website, and it doesn't look like the free energy for these reactions is much different at 4000 K.
$\Delta_f G_{(CN)_2}^{0} = 136.036\ \mathrm{kJ\ mol^{-1}}$
$\Delta_f G_{CO}^{0} = -446.485\ \mathrm{kJ\ mol^{-1}}$
$\Delta_f G_{CO_2}^{0} = -393.183\ \mathrm{kJ\ mol^{-1}}$
$\Delta G_{rxn1}^{0} = -1029.005\ \mathrm{kJ\ mol^{-1}}$
$\Delta G_{rxn2}^{0} = -922.397\ \mathrm{kJ\ mol^{-1}}$
EDIT 2:
So, I just read one of the original papers on this type of flame, and it sounds like the thermal stability of the products is an issue as well. I take it that at these very high temperatures, some of the energy can be transferred to bond-breaking processes, which would lower the total flame temperature. $\ce{CO}$ and $\ce{N2}$ have very strong covalent bonds, so perhaps they do not readily dissociate at these temps.