The whole problem comes from the high charge of $\ce{Al^{3+}}$ ion : $+3$. As such it attracts strongly $\ce{H2O}$ molecules from the solution. But the central Oxygen atom of these $\ce{H2O}$ molecules is more strongly attracted than the $\ce{H}$ atoms, due to the high electronegativity of $\ce{O}$ atoms. Hydrogen atoms from water are not so strongly attracted and they are even repelled by $\ce{Al^{3+}}$, due to the low electronegativityrepulsion of the $+3$ charge on $\ce{Al^{3+}}$ and the partial positive charge on $\ce{H}$ atom. As a consequence, the bond $\ce{O-H}$ which exists in water and in the structure $\ce{Al···O-H}$ will be broken between $\ce{O}$ and $\ce{H}$ when water is approaching $\ce{Al^{3+}}$. And this rupture is leaving a new bond $\ce{Al^{3+}- O^{-}}$ which is charged $2+$ and repels the $\ce{H^+}$ ion into the solution. Of course the triply charged $\ce{Al^{3+}}$ is stronger than $\ce{H^+}$ to hold the intermediate Oxygen atom. Matter of fact, all solutions of triply charged ions like $\ce{Al^{3+}}$ (or $\ce{Fe^{3+}}$, etc.) are hydrolyzed and produce a strongly acidic aqueous solution. As a consequence, if ammonia is introduced into such an acidic solution, it will immediately be neutralized and transformed into ammonium ions $\ce{NH4^+}$ which cannot make a complex, for lack of available doublet. This is the reason why $\ce{Al^{3+}}$ (and $\ce{Fe^{3+}}$) ions cannot make complexes with ammonia.