According to this Wikipedia solubility table, lithium bicarbonate is soluble in water to $\pu{5.7 g/100mL}$ at $\pu{20^oC}$, while this Wikipedia page gives the solubility of magnesium bicarbonate as $\pu{0.077 g/100mL}$.
However, another source (found by Oscar Lanzi in his good answer), "The Solubility of Magnesium Carbonate in Aqueous Solutions of Certain electrolytes, The Journal of Physical Chemistry, Volume 7, pub. 1903, reports making two measurements of the solubility of magnesium bicarbonate and finding $\pu{3.1 g/100mL}$ and $\pu{2.0 g/100 mL}$. Note that this source is from 1903, but also that the the Wikipedia link above does not give a source for their value at all. At least one reason for the dearth of information regarding magnesium bicarbonate is that it easily decomposes and it cannot readily be isolated outside of aqueous solutions. My feeling is that the Wikipedia value seems low, and that the values reported in the Journal of Physical Chemistry seem more reasonable.