How can the vapor pressure increase until it reaches a value close enough to the atmospheric pressure to sublimate without reaching its melting point?
Compounds can sublimate even if the vapor pressure is much lower than the atmospheric pressure. This is how we can smell solids. If you put the solid substance in a closed container, the partial pressure of the substance in the gas phase will increase until it reaches the vapor pressure, and you will be at equilibrium. In an open system, it will continue to sublimate, with a concentration gradient in the gas phase (that is how the sense of smell is used to locate smelly objects).
they asked us to heat a beaker that contained camphor and to cover it with a watch glass with ice on it.
In your experimental setup, you never reach an equilibrium. The vapor pressure of the camphor in the beaker is higher than the vapor pressure of the camphor deposited on the cold watch glass. Accordingly, camphor will continue to sublimate from the solid on the bottom and continue to deposit on the camphor on the watch glass.
This seems weird. Does this happen to other substances?
Yes, you can see the snow sublimate in the winter when temperatures are below the freezing point of water. A dusting of snow on a drive way (or ice on a windshield) will "disappear" (sublimate) without liquid water present at any time. When food is freeze-dried, the same process is used. The key for this to work is that the air has to be dry (i.e. the partial pressure of water in the air has to be lower than the vapor pressure of ice).