I'm basing my attempt at an answer off of Modern Quantum Chemistry by Szabo and Ostlund, p.119-122.
The general development of Hartree-Fock is done via functional variation of the ground state energy $E_0=\left<\psi_0\right|\!\hat{H}\!\left|\psi_0\right>$. At the end of this, we obtain the matrix form of a differential equation
$$ f\left|\chi_a\right> = \sum_{b\,=\,1}^N \epsilon_{ba}\left|\chi_b\right> $$
where $f$ is the Fock operator and the sum is over all $N$ occupied spin orbitals. By appropriate unitary transformation (that which diagonalizes the matrix $\epsilon$), we obtain the equation in its canonical form
$$ f\left|\chi_a'\right> = \epsilon_{a}\left|\chi_a'\right> $$
where the $\chi_a'$ are the canonical Hartree-Fock orbitals.
My understanding of why it would be difficult to use Koopmans' theorem with other orbitals is that the canonical Hartree-Fock orbitals are unique in putting the above matrix equation in diagonal form. In the noncanonical forms, there isn't a clear value that could considered the orbital energy for a particular $\left|\chi_a''\right>$ (i.e., they don't return an eigenvalue when acted on by the Fock operator).