as you can see, there are three electron subshells above obviously inert Xe core. 5d-electons, that most of the time are far from the nuclei because of the form of their orbitals, feel dramatic increase in effective core charge through the third d-row. For example, for Ti effective charge, keeping its d-electrons is 4. For gold it is 11, so the electrons are well-held.
For s-electrons the form of the orbital puts very significant part of electronic density near the nuclei. This means, that for heavy elements (staring from as far as Ga) a so-known 'inert electron pair effect' is observed: in comparison with p- and d-electrons, due the form of the s-orbital, s-electrons are much less shielded from the charge of the nuclei by underlying, leading to much more strong bonding of said electrons. As effect, Bi is highly unstable in valence state +5, perbromates are much stronger oxidizing agent than perchlorates, but same is not true for bromates and chlorates, and Ag, Au and Hg have very tightly bound s-electons, leading to relative inertness of said metals.
For Au and Ag, however, strongly bound s-electrons participate in extremely powerful (due to relative small size of s-orbitals) s-s interactions, while for Hg only relatively weak s-p and d-p interactions are possible because of filled d- and s- subshells. Thus, metallic Ag and especially Au in metallic state have s-electrons involved into effective covalent interactions, leading to relatively inert nature. Beware, however, that they are far from unreactive: Ag have high affinity to sulfur, and both Ag and Au dissolve in cyanide solutions when oxygen is present, not saying about such agressive reagents as chlorine and bromine.
Still, Au, Ag and Cu share one trait with Ia (I main subgroup) elements. Due to only spherical s-orbitals involvement in atom bonding in metal state, all three metals have high plasticity. Other d-elements have significant d-d bonding in metal states, and d-orbitals are not spherically symmetrical, leading to much less plastic behavior.
