Selection rules
The intensity of the transition from a state $\mathrm{i}$ to a state $\mathrm{f}$ is governed by the transition dipole moment $\mu_{\mathrm{fi}}$ (strictly, it is proportional to $|\mu_{\mathrm{fi}}|^2$):
$$\iint \Psi_\mathrm{f}^*\hat{\mu}\Psi_\mathrm{i}\,\mathrm{d}\tau \,\mathrm{d}\omega \tag{1}$$
where $\mathrm{d}\tau$ is the usual volume element (representing integration over spatial coordinates) and $\omega$ is an additional spin coordinate (since the states $\Psi$ also specify the spin). For an electronic transition, $\hat{\mu}$ is the electric dipole moment operator, and only depends on the spatial coordinates $x$, $y$, and $z$. (I've also ignored the vibrational component i.e. the Franck-Condon factor since it's not really relevant for the purposes of this answer.)
Normally in determining selection rules, we are more interested in whether the integral in $(1)$ is zero or not. If the integral is zero, the transition is said to be forbidden. The usual way to determine whether it is zero is by using symmetry: if the integrand is antisymmetric under any symmetry operation, then the integral is zero. (It is exactly analogous to the integral of an odd function $f(x)$ over a symmetric region; for example, $\int_{-a}^a \sin x \,\mathrm{d}x = 0$.)
If we assume that both the initial and final states can be separated into spin and spatial components, then we get:
$$\iint (\psi_\mathrm{f}^\mathrm{spin}\psi_\mathrm{f}^\mathrm{space})\hat{\mu}(\psi_\mathrm{i}^\mathrm{spin}\psi_\mathrm{i}^\mathrm{space})\,\mathrm{d}\tau\,\mathrm{d}\omega$$
(I also dropped the complex conjugates - they aren't important.) Since $\hat{\mu}$ does not operate on the spin coordinate, we can separate this multiple integral:
$$\left(\int \psi_\mathrm{f}^\mathrm{spin} \psi_\mathrm{i}^\mathrm{spin} \,\mathrm{d}\omega\right) \left(\int \psi_\mathrm{f}^\mathrm{space} \hat{\mu} \psi_\mathrm{i}^\mathrm{space} \,\mathrm{d}\tau\right)$$
The first integral is nonzero if $\Delta S = 0$. The quantum number $S$ is related to the eigenvalue of a hermitian operator (the actual eigenvalue is $S(S+1)\hbar^2$), and eigenfunctions of a hermitian operator corresponding to different eigenvalues are necessarily orthogonal, making the integral identically zero if $\Delta S \neq 0$. This is the spin selection rule.
The second integral is nonzero if $\psi_\mathrm{f}^\mathrm{space}$ and $\psi_\mathrm{i}^\mathrm{space}$ have a different character under the inversion operation, i.e. one is gerade and another is ungerade. This is because $\hat{\mu}$ itself is ungerade, so for the integrand to be gerade, we need $\psi_\mathrm{f}^\mathrm{space}\psi_\mathrm{i}^\mathrm{space}$ to be ungerade ($\mathrm{u \otimes u = g}$). In turn this means that $\psi_\mathrm{f}^\mathrm{space}$ and $\psi_\mathrm{i}^\mathrm{space}$ must have different symmetry. This is the Laporte selection rule. It can be relaxed when the geometry is not perfectly octahedral, or tetrahedral for that matter, since inversion is no longer a symmetry operation. However, I digress.
Spin-orbit coupling
The problems arise because the total states are not separable into spin and spatial components. This occurs because both electron spin, as well as orbital angular momentum, are both angular momenta. They therefore interact with each other, and this is the spin-orbit coupling spoken of. Mathematically, it is treated as a perturbation to the Hamiltonian; the result is that the transition dipole moment cannot be separated into two integrals and this leads to the invalidation of the selection rules.
When considering spin-orbit coupling, the term symbols such as $^3\!F$, $^1\!D$, which imply a simultaneous eigenstate of the total spin operator $\hat{S}$ and the total orbital angular momentum operator $\hat{L}$ are no longer meaningful since both angular momenta are coupled to each other (by $\vec{j} = \vec{l} + \vec{s}$) and behave as one "combined" source of angular momentum. The quantities $S$ and $L$ are called "nearly good" quantum numbers: they are no longer strictly conserved, but the "contamination" of different quantum numbers is relatively small. Mathematically we could express this as an admixture of different spin states into the original wavefunction:
$$^3\!F_J = c(^3\!F) + c_1(^1\!F) + \cdots$$
where $^3\!F_J$ denotes a state with spin-orbit coupling (not an eigenstate of $\hat{S}$ and $\hat{L}$), and $(^3\!F)$ denotes the original $^3\!F$ state before spin-orbit coupling is considered (which is a true eigenstate of $\hat{S}$ and $\hat{L}$). If $c_i \neq 0$, then one obtains a non-zero transition dipole moment between "states" of nominally different $S$.
The fact that the selection rules are still a thing is solely because spin-orbit coupling is a small effect ($c_i$ close to 0), so the compounds behave "almost" as if they obey the selection rules. More properly, the transition dipole moment between states of different $S$ is small, and therefore the d-d bands in $\mathrm{d^5}$ spectra are extremely weak. The magnitude of spin-orbit coupling, however, does vary with $Z^4$ ($Z$ being the atomic number). Therefore, while spin-orbit coupling is rather small in the 3d series, it becomes an extremely important mechanism by which the spin selection rule in the the 4d and 5d congeners, as well as the f-block elements, is relaxed.