The electron configuration of calcium is 2, 8, 8, 2, where up to that point each shell, asides from the first shell counts up to 8 - why then does scandium have an electron configuration of 2, 8, 9, 2?
What causes the 3rd shell to start filling up, rather than the fourth?
The relative energies of the electronic subshells have been calculated for atoms in the vicinity of $Z=20$ (J. Chem. Educ., 1994, 71 (6), 469), and the result is surprising:
Looking at this graph, by all means the electronic configuration of scandium should in fact be $\ce{1s^2 2s^2 2p^6 3s^2 3p^6}$ $\color{blue}{\ce{3d^3}}$ in order to minimize orbital energies! The graph does not show the energy of the $\ce{4p}$ subshell, but it would lie somewhat above both curves shown. However, there is an important effect not being considered here, which is the destabilizing interelectronic repulsions. Electrons in a same subshell tend to repel each other more (intrasubshell repulsion) than electrons in different subshells (intersubshell repulsion). It turns out (and no one can really qualitatively explain why, see below) the repulsion is such that in an atom with $Z=21$, the configuration $\ce{[Ar] 4s^2 3d^1}$ is lower in energy than other potential candidates, such as $\ce{[Ar] 3d^3}$ or $\ce{[Ar] 4s^1 3d^2}$. Theoretically, the odd configurations $\ce{[Ar] 4s^2 4p^1}$ or $\ce{[Ar] 4s^1 3d^1 4p^1}$ would have lower interelectron repulsion energies than the observed ground state, but in these cases the lowered interelectron repulsions do not compensate the requirement of populating the higher-energy $\ce{4p}$ subshell. The comparison of several possible electron configurations and the joint minimization of orbital energy and electron repulsion energies does not seem to be something that can be done without resorting to heavy calculations or direct experimental validation.
In addition to some material I linked in comments, this article from Eric Scerri's blog (a chemist who focuses on aspects of periodicity, including electronic distribution) states, regarding the electronic configuration of scandium compared to calcium and to the $\ce{Sc^3+}$ ion (emphasis mine):
This amounts to saying that all three of the final electrons enter $\ce{3d}$ but two of them are repelled into an energetically less favourable orbital, the $\ce{4s}$, because the overall result is more advantageous for the atom as a whole. But this is not something that can be predicted. Why is it 2 electrons, rather than one or even none? In cases like chromium and copper just one electron is pushed into the $\ce{4s}$ orbital. In an analogous case from the second transition series, the palladium atom, the competition occurs between the $\ce{5s}$ and $\ce{4d}$ orbitals. In this case none of the electrons are pushed up into the $\ce{5s}$ orbital and the resulting configuration has an outer shell of $\ce{[Kr] 4d^10}$.
None of this can be predicted in simple terms from a rule of thumb and so it seems almost worth masking this fact by claiming that the overall configuration can be predicted, at least as far as the cases in which two electrons are pushed up into the relevant s orbital. To those who like to present a rather triumphal image of science it is too much to admit that we cannot make these predictions. The use of the sloppy Aufbau seems to avoid this problem since it gives the correct overall configuration and hardly anybody smells a rat.
So unfortunately, it seems that even though most of the effects which combine to result in the observed electronic configurations are known, there is no qualitative way to predict where the configurations are going to mismatch with the Aufbau principle or the energy levels of the orbitals. I have read the statement that the Aufbau principle is most decidedly wrong for practically every atom with respect to the placement of the orbital energy levels, but incredibly it happens to predict the configuration of the valence shell for most atoms.
The electron configuration for scandium is:
$$\ce{1s^2 2s^2 2p^6 3s^2 3p^6 4s^2 3d^1}$$
The $\ce{4s}$ and $\ce{3d}$ orbitals are relatively close in energy, so there are a few anomalies in the order of filling. A fair question might be, why isn't the scandium electron configuration $\ce{[Ar] 3d^3 4s^0}$. There is a special stability associated with completely filled orbitals, so the $\ce{3d^1 4s^2}$ configuration is of lower energy than the alternative $\ce{3d^3 4s^0}$.
What is being looked at, is how to know or predict what an electronic configuration is going to be.
When some speak of filling up shells with electrons, one has to consider what that means. Does it mean a kind of cartoonish idea of take a particular element whose electronic configuration we know eg. Carbon atomic number 6, electronic configuration 2,4. Add a proton(making Nitrogen), and then add an electron, where will that electron go. And it goes in the last shell making Nitrogen 2,5.
Another way that might be meant by filling up is if you take the neutral atom, remove all or some of the electrons and start adding them in then what order would they fill up.
For example neutral scandium has 21 electrons, if you remove 3 electrons, to give 18 electrons which is Argon's configuration, so that goes from neutral scandium to Sc 3+. Then if you were to add electrons, where would the 19th, 20th and 21st electron go. Or, just for one electron, take a neutral atom, e.g. neutral Scandium, remove an electron, so we are looking at the 1+ cation, (maybe assuming that the electronic configuration is now that of the previous element, or maybe checking to find out what is the electronic configuration of that 1+ cation), then add the electron and consider where it's going to go.
Or for a simpler example, taking Carbon, imagine it has one less electron so has configuration 2,3 (Boron's configuration), then add an electron, and it becomes 2,4.
There's the idea that each shell is identified by a number n. The second goes further out than the first. The third goes further out than the second etc.
It'd be wrong/inaccurate to say that shells are filled inward to outwards, they aren't necessary.(Though the first and second shells do follow this). It'd be wrong/inaccurate to say that a shell fills to completion before the next shell fills, they aren't necessarily. (Though the first and second shells follow that). And it'd be wrong to think that the third shell has a max of 8, it doesn't.
Some basic material is excellent.
https://www.middleschoolchemistry.com/lessonplans/chapter4/lesson3
"When the third energy level has 8 electrons, the next 2 electrons go into the fourth energy level."
and
"The third energy level can actually hold up to 18 electrons, so it is not really filled when it has 8 electrons in it. But when the third level contains 8 electrons, the next 2 electrons go into the fourth level. Then, believe it or not, 10 more electrons continue to fill up the rest of the third level."
And you see the electronic configurations in that format here https://ptable.com/?lang=en#Properties e.g. Zinc 2,8,18,2
A view that explains a bit further than just looking at shells, is to use a model involving "subshells" rather than shells, and to say they are filled in order of energy level. That speaks of the Aufbau principle and order of energy levels and subshells, the madelung rule aka n+l rule. At 16-19 you might learn about it.
The electronic configurations for neutral atoms are shown in abbreviated form here https://sciencenotes.org/list-of-electron-configurations-of-elements/
That rule, the n+l rule, or madelung rule. Some might call it the afbau rule (though calling it afbau rule might be a bit ambiguous 'cos by that some just mean the idea of filling up in order of increasing energy level, whatever that order is).
The n+1 rule gives us the following order of subshells. "1s, 2s, 2p, 3s, 3p, 4s, 3d, 4p, 5s, 4d, 5p, 6s, 4f, 5d, 6p, 7s, 5f, 6d, and 7p". Those are all listed here https://chem.libretexts.org/Courses/Valley_City_State_University/Chem_115/Chapter_2%3A_Atomic_Structure/2.4_Electron_Configurations
n and l are quantum numbers.. n is the principal quantum number. Each subshell s,p,d,f respectively, have l values of 0,1,2,3 respectively. For The 4s orbital , n=4 and l=0, thus n+l=4+0=4. The 3d orbital has n=3 and l=2, thus n+l=3+2=5. So the n+l rule has 4s below 3d. 4s lower energy than 3d. The n+l rule always comes out with that order. It's a rule that helps to find the electronic configuration of neutral atoms.
When it comes to the transition metals, scandium onwards, technically, the order is a bit different. (i.e. not the n+l rule). Here, 3d is before 4s. (which helps to explain the electronic configuration of some of the ions but doesn't help to explain scandium's electronic configuration, that would take some further explanation).
The n+1 rule gives an easy "explanation" of scandium's electronic configuration. If we assume as the rule says that 4s is below 3d.
Also an explanation behind why 4s is below 3d, is looking at a radial probability distribution graph for the orbitals, "Why 4s before 3d" by mommachem https://www.youtube.com/watch?v=5hzv5KQ4LoM 4s is said to penetrate the nucleus more than 3d.
The n+l rule also answers a more foundational question to the one you ask. Of why is Potassium [Ar]4s1
You might have thought that the third shell has a max of 8 electrons, because some basic books wrongly state that(and are wrong in doing so). Infact a shell n, has a max of 2n^2 electrons. So the third shell has a max of 18 electrons.
But because 4s is less than 3d , (in energy level), for (neutral) Potassium and Calcium, the fourth shell is next to fill after 3p.
An s subshell can take a max of 2 electrons, a p subshell can take a max of 6 electrons, and a d subshell can take a max of 10 electrons.
Notice that before the third shell is maxed out. Specifically, after it has 8 electrons, (3s and 3p), then, the fourth shell fills up a bit(4s), getting two electrons, and then the third shell continues to fill(3d).
So that explains(or at least justifies!) why/how potassium, calcium and scandium have the electronic configurations they do.
And if you look at a periodic table with blocks drawn in, you see that. (e.g. looking at the fourth row left to right. 4s then 3d then 3p).
As you can see, an inner shell can have electrons added to it. And so that's where many books will put aside a simple view of just the shells, and from scandium onwards will tend to put away the format of electronic configurations of K,L,M,N like 2,8,9,2 And they'll go to showing electronic configurations as e.g. 1s2 2s2 2p6 3s2 4s1 e.t.c. And that model is in a way, simpler for elements from scandium onwards, and even Potassium onwards, in the sense that it's clear which subshell is going to fill up next.
When using the madelung rule(n+l rule), for getting the electronic configuration of neutral atoms, which is what it's for, then there are 21 exceptions. No exceptions in the first three rows. Two exceptions in the fourth row - Chromium and Copper. (Notice that Scandium is not an exception). And the rest of the exceptions are in the fifth sixth and seventh rows. No exceptions in the s and p blocks aka the main blocks.
Also, at AP level like 16-19, or A level in UK like 16-18, you might also be expected to know electronic configurations for ions of transition metals. Removing electrons and knowing the electronic configurations of various cations.
And that's where another rule comes into play. For working out electronic configuration of transition metal ions. That is that electrons come into 4s first, and go out of 4s first.
Some will explain that "into 4s first and out of 4s first", by saying that, assuming n+l order of energy levels with 4s below 3d, that for transition metals, electrons go in based on order of energy level, but they go out based on which shell is further out. So the fourth shell is further out, hence they leave 4s first. And 4s is lower in energy so they enter 4s first.
There is another explanation.. Which is that according to HF calculations, when it comes to neutral potassium and calcium, 4s is below 3d(so, like the n+l rule). But for transition metals scandium onwards, 3d is below 4s. So that then explains very well why electrons leave 4s first, i.e. leaves 4s before leaving 3d, it's because 4s is higher in energy than 3d. (So we wouldn't be using the n+l rule there).
Here is a graph based on HF calculations, that shows that , for Potassium and calcium 4s is below 3d. But for Scandium onwards, 3d is below 4s. So it's not just an explanation, it's well grounded.
https://pubs.acs.org/doi/abs/10.1021/ed071p469
Transition Metals and the Aufbau Principle
L. G. Vanquickenborne, K. Pierloot, and D. Devoghel
That then brings up another question, which is if 3d is below 4s in neutral transition metal atoms, scandium onwards, as HF calculations show, then why is Scandium's electronic configuration [Ar]4s2 3d1. Why isn't Scandium's electronic configuration [Ar] 3d3. And the explanation given, is that if we take Sc 3+ So scandium but with all fourth row electrons removed, so 3d and 4s electrons removed, to give Sc3+. Scandium has 21 electrons. Sc3+ has 18 electrons. And then we add three electrons, then the 19th electron goes into 3d. And the next two electrons initially go into 3d but then jump into 4s, because it's more stable so more preferable. This is explained nicely on Dr Eric Scerri's blog in these two articles here and here and on RSC here. And also explained by Jim Clark, an educator who has written some articles explaining that here and here
There is an article by a chemist, Geoffrey Neuss, here that tries to differ with Dr Scerri, and while Neuss makes some points that I think are fair and could be incorporated into Dr Scerri's explanation, such as that an ion doesn't necessarily have the same order of energy levels as a neutral atom (and perhaps shouldn't be assumed to be).. and that an ion's energy levels would be a bit different to a neutral atom. e.g. even if the order is the same, the distance between energy levels could differ. And Neuss also makes the good point, that when we see an electronic configuration lists 3d before 4s, that's just because it's listed in order of principal quantum number, it's spectroscopist notation and one shouldn't conclude from that, as some students might, that it is making a point about the order of 3d and 4s.
Neuss might not have been aware that the basis for Dr Scerri saying that 3d is below 4s, is HF calculations, and very strong. Neuss uses NIST data that looks at electron excitations and tries to conclude that neutral Scandium has 4s < 3d in energy level, like the n+1 rule has it. But that's flawed for Neuss to do that, because when a photon is fired in to cause an excitation, the energy levels are changed quite a bit.. And it's a different set of rules to work out what transition is going to happen, called "selection rules". So e.g. even looking at something simple like Hydrogen and the excitations of that, you can't get any list of orbital energy order. So i'd definitely go with Dr Scerri's explanation.
Orbitals fill up low energy first, then to higher energy. But since an electron could go into one orbital then jump to another, (as explained by Dr Scerri), we can't really always predict electronic configurations.
Jim's chemguide article mentions Vanadium. We can use NIST data to see the ground configurations of neutral atoms and of ions. For Vanadium V+ is [Ar]3d4 and V is [Ar]3d3 4s2
So we see that to get from V+ to V, when an electron is added to V+, one of the electrons jumps out of 3d into 4s. Or to look at it the other way, when an electron is removed from V to make it V+, an electron leaves 4s but then another electron jumps out of 4s and into 3d. The rule that electrons leave 4s first, would not predict V+ 'cos it didn't account for an electron jumping from one orbital into another!
There might still be some further advanced issues re the discussion of ordering of 3d and 4s, mentioned in the accepted answer here Why do 3d orbitals have lesser energy than 4s orbitals in transition metals?
At a very high level (and not relevant to the energy of orbitals subject), they'd say electrons are not particles, some say they are waves, or something with properties of each. And also, at a high level see this ACS paper, https://pubs.acs.org/doi/pdf/10.1021/ed200673w they say electrons don't "occupy" orbitals, they might even be spread out. And orbitals are just visual representations mapping to some Mathematics, not direct representations showing shapes of regions of space. But for the purposes of considering electronic configurations, people are fine with the concept *as a model*, of orbitals as regions and of electrons in orbitals.
