I'm basing my attempt at an answer off of Modern Quantum Chemistry by Szabo and Ostlund, p.119-122.
The general development of Hartree-Fock is done via functional variation of the ground state energy $E_0=\left<\psi_0|\hat{H}|\psi_0\right>$. At the end of this, we obtain a differential equation
$$ f\left|\chi_a\right> = \sum_{b=1}^N \epsilon_{ba}\left|\chi_b\right> $$
where $f$ is the Fock operator and the sum is over all $N$ occupied spin orbitals. By appropriate unitary transformation (that which diagonalizes the matrix $\epsilon$), we obtain the equation in its canonical form
$$ f\left|\chi_a'\right> = \epsilon_{a}\left|\chi_a'\right> $$
where the $\chi_a'$ are the Hartree-Fock orbitals.
My understanding of why it would be difficult to use Koopmans theorem with other orbitals is that the Hartree-Fock orbitals are unique in putting the equation in canonical form. In the noncanonical forms, there isn't a clear value that could considered the orbital energy for a particular $\left|\chi_a\right>$ (i.e. they don't return an eigenvalue when acted on by the Fock operator).
