For the first part of your question, you are correct that Mg has 12 electrons, which are arranged in shells in the pattern 2 - 8 - 2. But magnesium is a typical metal in that it tends to lose electrons to form a cation that then forms solids by ionic bonding. The octet rule predicts that the resulting cation will be stablest if its outermost shell contains eight electrons. And indeed the magnesium ion is $\mathrm{Mg}^{2+}$, two electrons less than the atom, with an electron configuration of 2 - 8.

What would happen if you had asked about a non-metal atom in the third period, say Cl? This has an electron configuration of 2 - 8 - 7, and tends to form a single covalent bond (e.g., to form $\mathrm{Cl}_2$ or $\mathrm{HCl}$), giving a configuration of 2 - 8 - 8 rather than 2 - 8 - 18. The reason for this is that the first eight valence electrons occupy the $s$ (2 electrons) and $p$ (6 electrons) subshells, while any further electrons in the third shell must go into the $d$ subshell. These orbitals are both higher in energy than those from the first two subshells (which is why, as you go through the periodic table, the $4s$ shell is filled before the $3d$) and extend less far from the nucleus, so that they are much less likely to overlap other atoms' orbitals to form a covalent bond. So it is often energetically preferable not to put electrons into this subshell if possible, which is why the octet rule can still apply to elements in this period.

Of course, it is possible to put electrons into this subshell, which is why it is possible for the central atom to have more than an octet in compounds like $\mathrm{SF_6}$. This leads into the second part of your question, which, as Manishearth suggested, looks like a duplicate of Can an atom have more than 8 valence electrons? If not, why is 8 the limit?.

For the first part of your question, you are correct that Mg has 12 electrons, which are arranged in shells in the pattern 2 - 8 - 2. But magnesium is a typical metal in that it tends to lose electrons to form a cation that then forms solids by ionic bonding. The octet rule predicts that the resulting cation will be stablest if its outermost shell contains eight electrons. And indeed the magnesium ion is $\mathrm{Mg}^{2+}$, two electrons less than the atom, with an electron configuration of 2 - 8.

What would happen if you had asked about a non-metal atom in the third period, say Cl? This has an electron configuration of 2 - 8 - 7, and tends to form a single covalent bond (e.g., to form $\mathrm{Cl}_2$ or $\mathrm{HCl}$), giving a configuration of 2 - 8 - 8 rather than 2 - 8 - 18. The reason for this is that the first eight valence electrons occupy the $s$ (2 electrons) and $p$ (6 electrons) subshells, while any further electrons in the third shell must go into the $d$ subshell. These orbitals are both higher in energy than those from the first two subshells (which is why, as you go through the periodic table, the $4s$ shell is filled before the $3d$) and extend less far from the nucleus, so that they are much less likely to overlap other atoms' orbitals to form a covalent bond. So it is often energetically preferable not to put electrons into this subshell if possible, which is why the octet rule can still apply to elements in this period.

Of course, it is possible to put electrons into this subshell, which is why it is possible for the central atom to have more than an octet in compounds like $\mathrm{SF_6}$. This leads into the second part of your question, which, as Manishearth suggested, looks like a duplicate of Can an atom have more than 8 valence electrons? If not, why is 8 the limit?.