Carbon dioxide from the atmosphere (or from decaying matter in the ocean) reacts with water to form carbonic acid:
$$ \ce{ CO2 + H2O -> H_2CO_3 }$$
and this reacts with calcium ions to form calcium carbonate:
$$ \ce{H2CO3 + Ca^{2+} -> CaCO3 + 2H+ }$$
The solubility of calcium carbonate is about $13~\mathrm{mg \over L}$, so if the concentration of calcium carbonate is greater than this the excess will precipitate out as solid calcium carbonate.
Shell-forming organisms actively absorb calcium from the water around them, so they are able to increase the concentration of calcium carbonate to above $13~\mathrm{mg\over L}$ and precipitate the excess to form their shells. They can get the carbon dioxide from the water around them or from their own metabolism. The actual details of shell formation is far more complex than this as it's controlled by processes with the cells of the organism rather than just being uncontrolled precipitation. I'm not sure how well the details are understood even today.
Incidentally, this is why shelly fauna are not keen on the acidification of the oceans that results from increased atmospheric carbon dioxide. The solubility of calcium carbonate is strongly $\mathrm{pH}$-dependent and rises sharply as the water gets more acid. Given that the shell is usually intended to stop other animals eating you, having your shell dissolve is generally not good for life expectancy.