"Insoluble (in water)" is a relative term, especially for ionic compounds, and so is "alkaline". That relativeness gives us enough wiggle room for two of our most common bases to slip through.
Roughly speaking, it takes a thousand volumes of water to react with and then dissolve one volume of calcium oxide, so the solubility of lime in water would not be evident from casual observation. Yet if you put an indicator into the water it reacts in a way that indicates the water has turned alkaline. The solubility of lime in water is hard to see with the eye, but it overwhelms the tiny intrinsic autoionization of water, so lime gives evidence of being both an "earth" and "alkaline". Ditto (despite the still lower solubility) for magnesia.
Lime and magnesia, then, are the quintessential alkaline earth's, far exceeding anything else with similar properties in their common occurrence. They have become the namesake of the all the "alkaline earth metals" in Group 2.