According to some chemistry textbooks, the maximum number of valence electrons for an atom is 8, but the reason for this is not explained.
So, can an atom have more than 8 valence electrons?
If this is not possible, why can't an atom have more than 8 valence electrons?
2017-10-27 Update
[NOTE: My earlier notation-focused answer, unchanged, is below this update.]
Yes. While having an octet of valence electrons creates an exceptionally deep energy minimum for most atoms, it is only a minimum, not a fundamental requirement. If there are sufficiently strong compensating energy factors, even atoms that strongly prefer octets can form stable compounds with more (or less) than the 8 valence shell electrons.
However, the same bonding mechanisms that enable the formation of greater-than-8 valence shells also enable alternative structural interpretations of such shells, depending mostly on whether such bonds are interpreted as ionic or covalent. Manishearth's excellent answer explores this issue in much greater detail than I do here.
Sulfur hexafluoride, $\ce{SF6}$, provides a delightful example of this ambiguity. As I described diagrammatically in my original answer, the central sulfur atom in $\ce{SF6}$ can be interpreted as either:
(a) A sulfur atom in which all 6 of its valence electrons have been fully ionized away by six fluorine atoms, or
(b) A sulfur atom with a stable, highly symmetric 12-electron valence shell that is both created and stabilized by six octahedrally located fluorine atoms, each of which covalently shares an electron pair with the central sulfur atom.
While both of these interpretations are plausible from a purely structural perspective, the ionization interpretation has serious problems.
The first and greatest problem is that fully ionizing all 6 of sulfur's valence electrons would require energy levels that are unrealistic ("astronomical” might be a more apt word).
A second issue is that the stability and clean octahedral symmetry of $\ce{SF6}$ strongly suggest that the 12 electrons around the sulfur atom have reached a stable, well-defined energy minimum that is different from its usual octet structure.
Both points imply that the simpler and more energetically accurate interpretation of the sulfur valence shell in $\ce{SF6}$ is that it has 12 electrons in a stable, non-octet configuration.
Notice also that for sulfur this 12- electron stable energy minimum is unrelated to the larger numbers of valence-related electrons seen in transition element shells, since sulfur simply does not have enough electrons to access those more complex orbitals. The 12 electron valence shell of $\ce{SF6}$ is instead a true bending of the rules for an atom that in nearly all other circumstances prefers to have an octet of valence electrons.
That is why my overall answer to this question is simply "yes".
Question: Why are octets special?
The flip side of whether stable non-octet valence shells exist is this: Why do octet shells provide an energy minimum that is so deep and universal that the entire periodic table is structured into rows that end (except for helium) with noble gases with octet valence shells?
In a nutshell, the reason is that for any energy level above the special case of the $n=1$ shell (helium), the "closed shell" orbital set $\{s, p_x, p_y, p_z\}$ is the only combination of orbitals whose angular momenta are (a) all mutually orthogonal, and (b) cover all such orthogonal possibilities for three-dimensional space.
It is this unique orthogonal partitioning of angular momentum options in 3D space that makes the $\{s, p_x, p_y, p_z\}$ orbital octet both especially deep and relevant even in the highest energy shells. We see the physical evidence of this in the striking stability of the noble gases.
The reason orthogonality of angular momentum states is so important at atomic scales is the Pauli exclusion principle, which requires that every electron have its own unique state. Having orthogonal angular momentum states provides a particularly clean and easy way to provide strong state separation between electron orbitals, and thus avoid the larger energy penalties imposed by Pauli exclusion.
Pauli exclusion conversely makes incompletely orthogonal sets of orbitals substantially less attractive energetically. Because they force more orbitals to share the same spherical space as the fully orthogonal $p_x$, $p_y$, and $p_d$ orbitals of the octet, the $d$, $f$, and higher orbitals are increasingly less orthogonal, and thus subject to increasing Pauli exclusion energy penalties.
A final note
I may later add another addendum to explain angular momentum orthogonality in terms of classical, satellite-type circular orbits. If I do, I'll also add a bit of explanation as to why the $p$ orbitals have such bizarrely different dumbell shapes.
(A hint: If you have ever watched people create two loops in a single skip rope, the equations behind such double loops have unexpected similarities to the equations behind $p$ orbitals.)
Original 2014-ish Answer (Unchanged)
This answer is intended to supplement Manishearth's earlier answer, rather than compete with it. My objective is to show how octet rules can be helpful even for molecules that contain more than the usual complement of eight electrons in their valence shell.
I call it donation notation, and it dates back to my high school days when none of the chemistry texts in my small-town library bothered to explain how those oxygen bonds worked in anions such as carbonate, chlorate, sulfate, nitrate, and phosphate.
The idea behind this notation is simple. You begin with the electron dot notation, then add arrows that show whether and how other atoms are "borrowing" each electron. A dot with an arrow means that the electron "belongs" mainly to the atom at the base of the arrow, but is being used by another atom to help complete that atom's octet. A simple arrow without any dot indicates that the electron has effectively left the original atom. In that case, the electron is no longer attached to the arrow at all but is instead shown as an increase in the number of valence electrons in the atoms at the end of the arrow.
Here are examples using table salt (ionic) and oxygen (covalent):
Notice that the ionic bond of $\ce{NaCl}$ shows up simply as an arrow, indicating that it has "donated" its outermost electron and fallen back to its inner octet of electrons to satisfy its own completion priorities. (Such inner octets are never shown.)
Covalent bonds happen when each atom contributes one electron to a bond. Donation notation shows both electrons, so doubly bonded oxygen winds up with four arrows between the atoms.
Donation notation is not really needed for simple covalent bonds, however. It's intended more for showing how bonding works in anions. Two closely related examples are calcium sulfate ($\ce{CaSO4}$, better known as gypsum) and calcium sulfite ($\ce{CaSO3}$, a common food preservative):
In these examples the calcium donates via a mostly ionic bond, so its contribution becomes a pair of arrows that donate two electrons to the core of the anion, completing the octet of the sulfur atom. The oxygen atoms then attach to the sulfur and "borrow" entire electrons pairs, without really contributing anything in return. This borrowing model is a major factor in why there can be more than one anion for elements such as sulfur (sulfates and sulfites) and nitrogen (nitrates and nitrites). Since the oxygen atoms are not needed for the central atom to establish a full octet, some of the pairs in the central octet can remain unattached. This results in less oxidized anions such as sulfites and nitrites.
Finally, a more ambiguous example is sulfur hexafluoride:
The figure shows two options. Should $\ce{SF6}$ be modeled as if the sulfur is a metal that gives up all of its electrons to the hyper-aggressive fluorine atoms (option a), or as a case where the octet rule gives way to a weaker but still workable 12-electron rule (option b)? There is some controversy even today about how such cases should be handled. The donation notation shows how an octet perspective can still be applied to such cases, though it is never a good idea to rely on first-order approximation models for such extreme cases.
2014-04-04 Update
Finally, if you are tired of dots and arrows and yearn for something closer to standard valence bond notation, these two equivalences come in handy:
The upper straight-line equivalence is trivial since the resulting line is identical in appearance and meaning to the standard covalent bond of organic chemistry.
The second u-bond notation is the novel one. I invented it out of frustration in high school back in the 1970s (yes I'm that old), but never did anything with it at the time.
The main advantage of u-bond notation is that it lets you prototype and assess non-standard bonding relationships while using only standard atomic valences. As with the straight-line covalent bond, the line that forms the u-bond represents a single pair of electrons. However, in a u-bond, it is the atom at the bottom of the U that donates both electrons in the pair. That atom gets nothing out of the deal, so none of its bonding needs are changed or satisfied. This lack of bond completion is represented by the absence of any line ends on that side of the u-bond.
The beggar atom at the top of the U gets to use both of the electrons for free, which in turn means that two of its valence-bond needs are met. Notationally, this is reflected by the fact that both of the line ends of the U are next to that atom.
Taken as a whole, the atom at the bottom of a u-bond is saying "I don't like it, but if you are that desperate for a pair of electrons, and if you promise to stay very close by, I'll let you latch onto a pair of electrons from my already-completed octet."
Carbon monoxide with its baffling "why does carbon suddenly have a valence of two?" structure nicely demonstrates how u-bonds interpret such compounds in terms of more traditional bonding numbers:
Notice that two of carbon's four bonds are resolved by standard covalent bonds with oxygen, while the remaining two carbon bonds are resolved by the formation of a u-bond that lets the beggar carbon "share" one of the electron pairs from oxygen's already-full octet. Carbon ends up with four-line ends, representing its four bonds, and oxygen ends up with two. Both atoms thus have their standard bonding numbers satisfied.
Another more subtle insight from this figure is that since a u-bond represents a single pair of electrons, the combination of one u-bond and two traditional covalent bonds between the carbon and oxygen atoms involves a total of six electrons, and so should have similarities to the six-electron triple bond between two nitrogen atoms. This small prediction turns out to be correct: nitrogen and carbon monoxide molecules are in fact electron configuration homologs, one of the consequences of which is that they have nearly identical physical chemistry properties.
Below are a few more examples of how u-bond notation can make anions, noble gas compounds, and odd organic compounds seem a bit less mysterious:
Yes, it can. We have molecules which contain "superoctet atoms". Examples:
$\ce{PBr5, XeF6, SF6, HClO4, Cl2O7, I3- , K4[Fe(CN)6], O=PPh3 }$
Almost all coordination compounds have a superoctet central atom.
Non-metals from Period 3 onwards are prone to this as well. The halogens, sulfur, and phosphorus are repeat offenders, while all noble gas compounds are superoctet. Thus sulfur can have a valency of +6, phosphorus +5, and the halogens +1, +3, +5, and +7. Note that these are still covalent compounds -valency applies to covalent bonds as well.
The reason why this isn't usually seen is as follows. We basically deduce it from the properties of atomic orbitals.
By the aufbau principle, electrons fill up in these orbitals for period $n$:
$n\mathrm{s}, (n-2)\mathrm{f},(n-1)\mathrm{d},n\mathrm{p}$
(theoretically, you'd have $(n-3)\mathrm{g}$ before the $\mathrm{f}$, and so on. But we don't have atoms with those orbitals, yet)
Now, the outermost shell is $n$. In each period, there are only eight slots to fill in this shell by the Aufbau principle - 2 in $n\mathrm{s}$, and 6 in $n\mathrm{p}$. Since our periodic table pretty much follows this principle, we don't see any superoctet atoms usually.
But, the $\mathrm{d,f}$ orbitals for that shell still exist (as empty orbitals) and can be filled if the need arises. By "exist", I mean that they are low enough in energy to be easily filled. The examples above consist of a central atom, that has taken these empty orbitals into its hybridisation, giving rise to a superoctet species(since the covalent bonds add an electron each)
I cooked up a periodic table with the shells marked. I've used the shell letters instead of numbers to avoid confusion. $K,L,M,N$ refer to shell 1,2,3,4 etc. When a slice of the table is marked "M9-M18", this means that the first element of that block "fills" the ninth electron in the M(third) shell, and the last element fills the eighteenth.
Click to enlarge:
(Derivative of this image)
Note that there are a few irregularities, with $\ce{Cu}$, $\ce{Cr}$, $\ce{Ag}$, and a whole bunch of others which I've not specially marked in the table.
Something worth adding to this discussion that I'm surprised hasn't been mentioned about such "hypervalent" molecules like $\ce{SF6}$.
One of my professors at university informed me that the common explanation (that the empty d-orbitals are empty and are thus accessible) is actually most likely incorrect. This is an old-model explanation that is out-of-date, but is for some reason continuously taught in schools. A quote from the Wikipedia article on orbital hybridisation:
In 1990, Magnusson published a seminal work definitively excluding the role of d-orbital hybridization in bonding in hypervalent compounds of second-row elements.
(J. Am. Chem. Soc. 1990, 112 (22), 7940–7951. DOI: 10.1021/ja00178a014.)
When you actually look at the numbers, the energy associated with those orbitals is significantly higher than the bonding energy found experimentally within molecules like $\ce{SF6}$, meaning that it is highly unlikely that the d-orbitals are involved at all in this type of molecular structure.
This leaves us stuck, in fact, with the octet rule. Since $\ce{S}$ cannot reach into its d-orbitals, it cannot have more than 8 electrons in its valence (see other discussions on this page for definitions of valence etc, but by the most basic definition, yes, only 8). The common explanation is the idea of a 3-centered 4-electron bond, which is essentially the idea that sulfur and two fluorines 180 degrees apart share only 4 electrons between their molecular orbitals.
One way of comprehending this is to consider a pair of resonance structures where sulfur is bonded covalently to one $\ce{F}$ and ionically to the other:
$$\ce{F^{-}\bond{...}^{+}S-F <-> F-S+\bond{...}F-}$$
When you average these two structures out, you will notice that sulfur maintains a positive charge and that each fluoride has a sort of "half" charge. Also, note that sulfur only has two electrons associated with it in both structures, meaning that it has successfully bonded to two fluorines while only accumulating two electrons. The reason they have to be 180 degrees apart is due to the geometry of the molecular orbitals, which is beyond the scope of this answer.
So, just to review, we've bonded to two fluorines to the sulfur accumulating two electrons and 1 positive charge on sulfur. If we bonded the remaining four fluorides from $\ce{SF6}$ in the normal covalent way, we'd still end up with 10 electrons around sulfur. So, by utilizing another 3-center-4 electron bond pair, we achieve 8 electrons (filling both the s and p valence orbitals) as well as a $+2$ charge on the sulfur and a $-2$ charge distributed around the four fluorines involved in the 3c4e bonding. (Of course, all of the fluorines have to be equivalent, so that charge will actually be distributed around all of the fluorines if you consider all of the resonance structures).
There actually is a lot of evidence to support this style of bonding, the simplest of which is observed by looking at bond lengths in molecules such as $\ce{ClF3}$ (T-shapes geometry), where the two fluorines 180 degrees apart from each other have a slightly longer bond length to chlorine than the other fluorines do, indicating a weakened amount of covalency in those two $\ce{Cl-F}$ bonds (a result of averaging out a covalent and ionic bond).
If you are interested in the details of the molecular orbitals involved, you may wish to read this answer.
TL;DR Hypervalency doesn't really exist, and having more than $\ce{8 e-}$ in non-transition metals is much harder than you would think.
In chemistry, and in science in general, there are many ways of explaining the same empirical rule. Here, I am giving an overview that is very light on quantum chemistry: it should be fairly readable at a novice level, but will not explain in its deepest way the reasons for the existence of electronic shells.
The “rule” you are citing is known as the octet rule, and one of its formulations is that
atoms of low (Z < 20) atomic number tend to combine in such a way that they each have eight electrons in their valence shells
You'll notice that it's not specifically about a maximal valence (i.e. the number of electrons in the valence shell), but a preferred valence in molecules. It is commonly used to determine the Lewis structure of molecules.
However, the octet rule is not the end of the story. If you look at hydrogen (H) and helium (He), you will see that do not prefer an eight-electron valence, but a two-electron valence: H forms e.g. H2, HF, H2O, He (which already has two electrons and doesn't form molecules). This is called the duet rule. Moreover, heavier elements including all transition metals follow the aptly-named 18-electron rule when they form metal complexes. This is because of the quantum nature of the atoms, where electrons are organized in shells: the first (named the K shell) has 2 electrons, the second (L-shell) has 8, the third (M shell) has 18. Atoms combine into molecules by trying in most cases to have valence electrons entirely filling a shell.
Finally, there are elements which, in some chemical compounds, break the duet/octet/18-electron rules. The main exception is the family of hypervalent molecules, in which a main group element nominally has more than 8 electrons in its valence shell. Phosphorus and sulfur are most commonly prone to form hypervalent molecules, including $\ce{PCl5}$, $\ce{SF6}$, $\ce{PO4^3-}$, $\ce{SO4^2-}$, and so on. Some other elements that can also behave in this way include iodine (e.g. in $\ce{IF7}$), xenon (in $\ce{XeF4}$), and chlorine (in $\ce{ClF5}$). (This list isn't exhaustive.)
Gavin Kramar's answer explains how such hypervalent molecules can come about despite apparently breaking the octet rule.
This question may be difficult to answer because there are a couple of definitions of valence electrons. Some books and dictionaries define valence electrons as "outer shell electrons that participate in chemical bonding" and by this definition, elements can have more than 8 valence electrons as explained by F'x.
Some books and dictionaries define valence electrons as "electrons in the highest principal energy level". By this definition an element would have only 8 valence electrons because the $n-1$ $d$ orbitals fill after the $n$ $s$ orbitals, and then the $n$ $p$ orbitals fill. So, the highest principal energy level, $n$, contains the valence electrons. By this definition, the transition metals all have either 1 or 2 valence electrons (depending on how many electrons are in the $s$ vs. $d$ orbitals).
Examples:
By the other definition, they could have more since they have more "outer shell" electrons (until the $d$ shell is filled).
Using the "highest principal energy level" definition for valence electrons allows you to correctly predict the paramagnetic behavior of transition metals ions because valence electrons (the $d$ electrons) are lost first when a transition metal forms an ion.
There is a big difference between a "rule" and a law of nature. The "octet rule" is a turn-of-the-last-century concept that somehow managed to get into introductory chemistry books and never got kicked out with the advent of modern quantum mechanics. (Circumstantial proof: it is impossible to identify individual electrons to label them "valence" or "not valence".)
Therefore, you won't find any answer based on physical evidence as to why/why not a rule that is not based on physical evidence will hold.
Atoms take their spatial configuration because it happens to be an electrostatically-favorable circumstance, not because electrons avail themselves like "slots".
Why 8? has not really been addressed by the above answers, and while tangential to the question, it is somewhat important to be considered. In general, but not always, atoms react to form complete quantum 'shells', with electrons interacting with all their orbitals.
The principal quantum number ($n$) determines the maximum azimuthal quantum number ($l$), in the sense that $l$ can only take values between $0$ and $n-1$. Thus for the first row, $n = 1$ and $l = 0$. For the second row, $n = 2$ so $l = 0,1$. For the third row, $n = 3$, so $l = 0, 1, 2$.
The azimuthal quantum number $l$ determines the range of possible magnetic quantum numbers ($m_l$), which lies in the range $-l \leq m_l \leq +l$. So for the first row, $m_l = 0$. For the second row, when $n = 2$ and $l = 1$, then $m_l = -1, 0, 1$. For the third row, $n = 3$, $l = 0, 1, 2$, $m_l = -2, -1, 0, 1, 2$.
Finally, the spin quantum number $m_s$ can be either $+1/2$ or $-1/2$.
The number of electrons that can fill each shell is equal to the number of combinations of quantum numbers. For $n=2$, this is
$$\begin{array}{cccc} n & l & m_l & m_s \\ \hline 2 & 0 & 0 & +1/2 \\ 2 & 0 & 0 & -1/2 \\ 2 & 1 & +1 & +1/2 \\ 2 & 1 & +1 & -1/2 \\ 2 & 1 & 0 & +1/2 \\ 2 & 1 & 0 & -1/2 \\ 2 & 1 & -1 & +1/2 \\ 2 & 1 & -1 & -1/2 \\ \end{array}$$
for a total of 8 electrons.
The second row contains 'organic compounds', of which millions are known, so there is frequently a bias in teaching chemistry to focus on the "octet rule". In fact, there is a duet rule to be considered for hydrogen, helium (and lithium which dimerizes in the gas phase), and the 'rule of 18' for transition metals. Where things get 'wonky' are silicon through chlorine. These atoms can form a complete quantum shell via the octet rule, or 'expand' their octets and be governed by the rule of 18. Or situations in-between, such as sulfur hexafluoride.
Bear in mind, this is a gross simplification, because these atomic orbitals mix to form molecular orbitals, but the counts of the atomic orbitals influence and directly correlate with the counts of the resulting molecular orbitals, so the combination of atomic quantum numbers still provides some interesting information.
Let's take a look at the periodic table: in the first row, there's only two elements: Hydrogen and Helium. They do not follow an octet rule. Hydrogen can only have a maximum of two electrons on the valence orbital. It turns out that the octet rule is not exclusive, meaning it is not the only rule that helps understand Lewis structure and electron configuration. Why do we use the octet rule, then?
Every period in the periodic table represents an energy shell of an atom. The first period represents the shell K, the first energy level, which has only the s orbital. Every orbital can only be filled with 2 electrons, both with a quantum spin towards opposite directions. Thus the maximum number of electrons possible for the first energy level shell, K, is 2. This is reflected in the fact that Helium is a noble gas, yet only contains 2. The second energy level shell, L, has the s orbital and the extra 3 p orbitals. Those add up to four orbitals or 8 electrons. Because the elements most commonly used are in the second and third period, the octet rule is in frequent use.
Elements of the third energy level are very similar. They still follow the octet rule, because even though now the have 5 d orbitals, no orbital needs to be filled. The electronic configuration shows that 4s is filled before 3d, so they don't need to fill the d orbital, thus they usually also obey the octet rule. However, third energy level shell elements, unlike second-row elements, (see Gavin's comment fir reference) are not limited to the octet rule. They can form hypervalent molecules in certain cases where the use that d orbital and fills — this is not the case with all apparent hypervalent molecules, SF6 is not hypervalent, it uses weak ionic bonds and polarity, but there still are hypervalent molecules out there. It will always depend on which state is more convenient in terms of electrostatics.
At the fourth energy level shell, there are f orbitals introduced, but we are not even close to filling them at that point because we first need to fill the d orbitals. The 5 d orbitals signify 10 electrons, plus the previous eight from the octet rule, sum up to 18. This is the reason why there are 18 columns in the periodic table. Now, a new rule superposes, and this is the well known 18-electron rule, which was mentioned above. Transition metals obey this rule with more frequency than not, though there are occasions in which they still obey the octet rule. At this point, with so many orbitals to fill, and with electrostatics playing a role in electronic configuration, we can obtain different cations from the same element with certain metals. That is also why they don't discuss oxidation state numbers with transition metals like they do with the first three rows of the table.
