Berillium is toxic and quite hard to obtain, so it's off.
Lithium is more common but still quite rare. It still is used in some pyrotechnic compositions, but rarely in form of metal.
Magnesium IS used in some composition that are intended is light sources. At the dawn of photography photoflashes used magnesium-based compositions. But it is rarely used nowdays, both thanks to electrical flashes and because it generates too much UV light, so it may harm eyes.
BTW more recent photoflash compositions use aluminium to avoid avoid the bane of magnesium-based composition: extremely fine smoke.
BTW, to obtain any metal above a lot of electricity is required, and in a setting with such metals Yablochkov candle should be available and it has quite vintage/steampunk look.