I tried making a lead acid battery. After charging for about 10 minutes, I connect the voltmeter in parallel to the cell and see that there is around a 1.2V (which is quite lower than what I predicted). However, when connecting to a lightbulb, the voltage seems to be smaller. Is this supposed to happen? Shouldn't the voltage be the same with the cell providing an appropriate current to match the resistance of the bulb?
Yes, this happens because there is an internal resistance within the battery. Look up how batteries are modeled within circuits.