First of all, let me state the obvious: Phosphorus is awesome.
After we got that out of the way we can focus on why.
There are many different modifications of phosphorus in nature. With increasing thermodynamic stability they are $$\ce{P_{white} -> P_{red} -> P_{violet} -> P_{black}}.$$
Apart from this there are many low molecular different allotropes, like $\ce{P4 (white)}$, $\ce{P6}$, $\ce{P8}$, $\ce{P10}$, $\ce{P12}$. And because that is not enough, there are chain like polymeric allotropes, too.
Apart from this it is possible to formulate many different cations and anions, that are derived from the above molecular structures. Just to name a few, there are $\ce{P3+, P5+, P7+, P9+}$ mainly observed in gaseous phase an $\ce{P^{3-},P2^{4-}, P3^{5-}, P4^{2-}, P7-,...,}$ usually in combination with alkali metals. Most amazingly it may form anionic polymer strains of the general form $\ce{[P7-]_{\infty}, [P15-]_{\infty}}$.
But now for the most important part, phosphorus has stable oxidation states in compounds, ranging from $\mathrm{-III}$ to $\mathrm{+V}$. Here are a few examples:
$$\ce{\overset{-III}{P}H3,\ \overset{-II}{P_2}H4,\ [\overset{-I}{P}H]_{n},\ \overset{\mathrm{\pm0}}{P4},\ H3\overset{\mathrm{+I}}{P}O2,\ H4\overset{\mathrm{+II}}{P2}O4,\ H3\overset{\mathrm{+III}}{P}O3,\ H2\overset{\mathrm{+IV}}{P2}O6,\ H3\overset{\mathrm{+V}}{P}O4}$$
While dealing with these compounds it is usually completely unnecessary to describe bonding with hybrid orbitals.
In case of phosphane $\ce{PH3}$ it would be wrong. Assuming $\ce{{}^{sp^3}P}$ one would expect $\angle(\ce{H-P-H})\approx109^\circ$, while it is found to be $\angle(\ce{H-P-H})=93.5^\circ$, which is almost the same angle as the $\ce{p}$ orbitals are having towards each other.
In general your assumption is correct, that it is possible to form only three covalent bonds to reach a stable configuration. And that will most likely be the case when phosphorus forms compounds with more electropositive elements.
Now dealing with oxygen, means dealing with a much more electronegative element, i.e. $\ce{En(O)}\approx3.4$, $\ce{En(P)}\approx2.2$. That also means that bonds are much more polarised towards the oxygen.
Analysing the phosphate anion $\ce{PO4^{3-}}$ it is crucial to recognise its symmetry, which is tetrahedral $T_\mathrm{d}$. In this arrangement it is perfectly safe (but not at all necessary) to describe phosphorus as $\ce{sp^3}$ hybridised.
A Natural Bond Orbital analysis (BP86/def2-TZVPP) reveals that there are 4 equal $\ce{P-O}~\sigma$ single bonds and each oxygen has three lone pairs. The contribution of the $\ce{d}$ orbitals to bonding is well below $1\%$ and can be interpreted as numerical noise (use as polarisation functions) of the DFT method.
\begin{array}{rlrr}\hline
& &\mathrm{\%P (hyb)} &\mathrm{\%O (hyb)}\\\hline
3\times&\ce{Bd(O-P)} & 24 (\ce{sp^3}) & 76 (\mathrm{sp^{2.3}})\\
&\ce{Lp(O)} & & 100 (\mathrm{sp^{0.4}})\\
2\times&\ce{Lp(O)} & & 100 (\ce{p})\\\hline
\end{array}
This is consistent with the partial charges, i.e. $q(\ce{P})=2.2$, $q(\ce{O})=-1.3$. Therefore a more accurate Lewis formula is with charge separation.
The corresponding NBO reflect the bonding picture one would expect given all details from above. It should be noted, that natural bond orbitals are a linear combination of the canonical orbitals and do not have a physical meaningful energy eigenvalue. The two top rows represent the $\ce{p}$ lone pair orbitals of oxygen, the third row represents the $\ce{sp^{0\!.4}}$ lone pair orbitals, the fourth row gives the $\sigma$ bonding orbitals. (The last row is the orientation of the molecule, core orbitals are not displayed.)
The corresponding canonical orbitals which have a physical meaningful eigenvalue are delocalised over the whole molecule, hence they are not providing a simple understandable bonding picture. While NBO fail to respect the symmetry point group, canonical orbitals are constructed to obey this principle. (Here shown from highest energy, top, to lowest energy, bottom, core orbitals not displayed.)