Determining Kohn-Sham and Hartree Fock virtual orbitals: The underlying field

In Frank Jensen Introduction to Computational Chemistry from 2007, I stumbled upon this paragraph about whether to assign meaning to the KS orbitals or not

Another difference is that the unoccupied orbital energies in Hartree–Fock theory are determined in the field of N electrons and therefore correspond to adding an electron, i.e. the electron affinity. The virtual orbitals in density functional theory, on the other hand, are determined in the field of N − 1 electrons and therefore correspond to exciting an electron, i.e. unoccupied orbitals in DFT tend to be significantly lower in energy than the corresponding HF ones, and the highest occupied molecular orbital–lowest unoccupied molecular orbital (HOMO–LUMO) gaps are therefore much smaller with DFT methods than for HF. This also means that orbital energy differences in DFT are reasonable estimates of excitation energies, in contrast to HF methods where excitation energies involve additional Coulomb and exchange integrals.

I can't seem to understand why the Kohn-Sham equations are solved in the presence of N-1 electrons and not N. From my understanding the Kohn Sham and the Hartree Fock equations are almost identical. Where does this difference come from?

That is true: in the Kohn–Sham model electrons both in occupied and in virtual orbitals are "moving" in the field $n-1$ electrons, while in the Hartee-Fock model electrons in occupied orbitals are "moving" in the field $n-1$ electrons, but electrons in virtual orbitals are "moving" in the field all $n$ electrons. And this is simply "by construction": you just have to look at the very definitions of these models. So, in short:
If you look at the Kohn–Sham equations $$\left(-{\frac {\hbar ^{2}}{2m}}\nabla ^{2}+v_{{{\rm {eff}}}}({\mathbf r})\right)\phi _{{i}}({\mathbf r})=\varepsilon _{{i}}\phi _{{i}}({\mathbf r}) \, ,$$ where $$v_{\rm {eff}}(\mathbf {r} )=v_{\rm {ext}}(\mathbf {r} )+e^{2}\int {\rho (\mathbf {r} ') \over |\mathbf {r} -\mathbf {r} '|}d\mathbf {r} '+v_{{{\rm {xc}}}} \, ,$$ it is last term of the Kohn–Sham effective potential, the exchange-correlation potential, $$v_{{{\rm {xc}}}}({\mathbf r})\equiv {\delta E_{{{\rm {xc}}}}[\rho ] \over \delta \rho ({\mathbf r})} \, ,$$ that causes the above mentioned feature of Kohn–Sham orbitals.
For the case of the Hartree-Fock equations $$\hat{F} \phi_i({\mathbf r}) = \epsilon_i \phi_i({\mathbf r}) \, ,$$ the summation in the Fock operator by the very definition of the model runs over all occupied orbitals only, $$\hat{F} = \hat{H}^{\text{core}} + \sum_{j=1}^{n}[\hat{J}_{j} - \hat{K}_{j}] \, .$$ For occupied orbitals, in the Hartree–Fock equation defining some particular (occupied) $i$-th orbital the Coulomb and exchange contribution from this orbital itself perfectly cancel each other, i.e. $\hat{J}_{j} \phi_i = \hat{K}_{j} \phi_i$ when $j=i$. Thus, in the Hartree-Fock model an electron from any occupied orbital is also moving in the field $n-1$ electrons. And this is of course physically more than reasonable: at the end of the day electron should not interact with itself.
But for virtual orbitals the situation in the Hartree-Fock model is different from that in the Kohn-Sham one: for any virtual orbital $\phi_k$ the sum in the Hartree-Fock equation defining it still runs over occupied orbitals only, but none of the terms $\hat{J}_{i} \phi_k$ and $\hat{K}_{i} \phi_k$ this time cancel each other since $k$ is simply grater than $n$.