Toxicities are often greatly influenced by how you are exposed and in what dose.
Mercury is a fascinating metal. You can find it as the pure metal, a salt or in an organic form. In its standard state it is a liquid and is rather volatile. Large amounts are naturally released into the atmosphere, but humans contribute as well, for example: burning fossil fuels, such as coal, will release it into the atmosphere.
The forms of mercury are not set in stone it is quite able to switch forms from the free metal, to the salt, to the organic by processes in nature.
Orally consuming some liquid metal isn't the worst possible route of exposure; mercury was at one time even used as a laxative. The only absorption in the GI tact is thought to be due to mercury vapor. Skin exposure is undesirable, but we know that inhaling the vapor is more toxic because it is soluble in lipids. Upon entering the bloodstream, some makes it way past the blood/brain barrier (hence its neurotoxicity) and some is converted to Hg(II) by catalase.
Mercury's inorganic salts have a long useful history where some were used to treat syphilis, some for lights and some for hat making and so on. "Mad as a hatter" won its name due to chronic exposure to Hg(NO$_3$)$_2$, which was used in hat making. This salt isn't very lipid soluble, but more absorption does occur when swallowed than the free metal. These salts tend to build up in the kidneys after absorption and high exposure will cause renal failure. Not much will pass through the blood/brain barrier, but chronic exposure will eventually allow some across and cause neurotoxicity. They also tend to bind with proteins containing sulfur and this can cause some cell death. As a general rule, Hg(I) salts are less toxic than Hg(II), presumably due to the decreased solubility of Hg(I). This does not mean they cannot cause harm: mercuric chloride was used in a factory and the wastes were discharged into a nearby sea, anaerobic bacteria methylated the mercury, this methylmercury was absorbed by fish that were later eaten and hurt many people.
As an organic, mercury compounds tend to be considerably more lipid soluble and therefore can readily be absorbed or cross the blood/brain barrier, in comparison to the free metal. A terrible example is dimethylmercury, which claimed the life of an inorganic chemist who was accidentally exposed to some she was using in an NMR experiment.
I do not know much about its amalgams aside from gold/silver. I believe most of the toxicity was seen in gold refineries due to the fact that the amalgam is heated to remove the mercury, which leaves behind gold, and results in the release of mercury vapor.