You are correct in saying an infrared thermometer would just measure the surface of the metal, and in addition infrared thermometers don't measure metal surfaces particularly well anyway (metals typically have a low emissivity). Measuring electrical resistance is better, but minor alloyed contamination would heavily skew your measurement.
Ultrasonic thermometry (as permiakra suggests) is one of the only ways to measure the internal temperature of anything without actually drilling into it. Technically it's mesuring the average temperature along the path of the sound, but that's still something.
You absolutely do not need to measure the internal temperature of these metal blocks. Metals are ridiculously good conductors of heat. If you have a good measurement of the temperature of the heating bath, you can consider the metal to be the same temperature after about a minute at most, more likely a dozen seconds. On the size scales you're working with even the most non-heat-conductive metals (bismuth, a few aerospace superalloys, plutonium...) will have a temperature within the error tolerance of the other parts of your setup (e.g. convection in the heating bath) within a matter of maybe a minute.
If you really need to convince yourself that metals are that heat conductive, go get a about half a foot of smooth coper pipe*, a piece of notebook paper, and a candle.
Light the candle.
Place the pipe in the middle of the paper and pick up the pipe using the paper as a sling.
Hold this assembly over the candle so that the flame touches the section of paper that is in contact with the pipe
As long as any part of the copper pipe is cool enough to touch, the paper touching any other part will not burn**. This is because as fast as the fire is dumping heat into the pipe, the pipe is spreading it away. Only once the whole pipe has risen significantly in temperature - far past the point where you can touch it without getting burned - will the paper start to burn.
* If the pipe is very thin-walled this might not work, however I have never found a copper pipe that thin.
** There may be a buildup of black soot from the candle on the paper, but this will only be on the side in contact with the flame, the opposite side will show no such buildup.