The unstability of alkaline earth bicarbonates can be traced back to their hypothetical crystal structure. Let us start with calcite CaCO3. There, each Ca2+ cation coordinate with 6 oxygen atoms in an octahedral shape. Besides, each oxygen coordinate with a C and two metal atoms. Thus, each carbonate group (containing three oxygens) coordinate with six metal atoms (valence 2+) without much distortion, creating a structure that is very stable.
Now, the bicarbonate structure is a distortion of CaCO3. There, on oxygen atom coordinates with three metals, another with two metals, and another with one metal plus a hydrogen atom. The latter is stabilized via hydrogen bonding with an oxygen atom which is coordinated with two metals. On average, you still have 1 metal atom (valence 1+) for each HCO3- group. Besides, both anion and cation are still hexacoordinated.
Li atoms are too small and can hardly be stabilized by coordinating with 6 oxygens. The anionic nature of such oxygens would make them "feel" and repel each other. This is the reason behind the instability of LiHCO3, previously mentioned by Oscar Lanzi.
Conversely, to create a solid bicarbonate of an alkaline earth metal, eg Ca(HCO3)2, each Ca atom would need to coordinate with 12 different oxygen atoms, which would be awfully unstable. Alternatively, each oxygen atom would need to "uncoordinate" with a metal atom, making them a lot less stabilized by electrostatic forces. In any case, the crystalline CaCO3 and their decomposition products (CO2, H2O) would be a more stable arrangement.