Why do the names of most chemical elements end with -um or -ium for both primordial and synthetic elements?
To expand on @BelieveInvis's answer -- in the early 19th century, when the Royal Society was really in the swing of things, the dominant language of scholarship was still Latin. Since Latin didn't have words for the new metallic elements, new words were coined from the existing terms for the substances and given Latinate endings.
From the OED's entry on -ium:
The Latin names of metals were in -um, e.g. aurum, argentum, ferrum; the names of sodium, potassium, and magnesium, derived from soda, potassa or potash, and magnesia, were given by Davy in 1807, with the derivative form -ium; and although some of the later metals have received names in -um, the general form is in -ium, as in cadmium, iridium, lithium, osmium, palladium, rhodium, titanium, uranium; in conformity with which aluminum has been altered to aluminium.
So, I think after that, other elements were simply given the suffix to fit the generally useful naming scheme, and then, metal names which were already in common use kept their common language names (e.g. gold as opposed to aurum) simply by force of usage.
The -ium suffix is a Latin suffix which forms abstract nouns, thus it is used to form chemical elements' name from its naming origin, such as minerals (calcium from calx) or person names (gadolinium).
To confirm and add a reference to the other two answers.
The deliberate naming of chemical elements such that they end in -um or -ium originates in $1811$. More specifically,
The Swedish chemist Berzelius, who recognised the value of logical naming, proposed (in $1811$) that the names of the elements should all be Latin in form.
Some elements had optained such a name prior to $1811$. Some were changed (albeit temporarily), and to some were simply added the suffix.
A number of names already given to metallic elements, e.g., uranium, chromium, barium, were of this form. He [Berzelius] used the Latin names ferrum, stannum and stibium for iron, tin and antimony, and gave zinc the "Latin" name zincum.
The names sodium and potassium were changed to natrium and kalium. All metallic elements discovered after this date ($1811$) have been given names ending in -ium (occasionally -um). Examples are cadmium, lanthanum, lithium, thallium, radium.
This is also why helium has the suffix.
The name helium is anomalous. This element, which was first discovered in the sun, was assumed to be an alkali metal, but when it was discovered terrestrially it was found to be an inert gas.
W. E. Flood. The Origins of Chemical Names. 1963, Oldbourne, London. (page xii)
Well, they end in -um because it's a good Latin neuter singular of the second declension: that is, it's a way you can refer to 'stuff' of the sun (helios), or of Magnesia (where magnesium and magnetic stuff was found), of lime (calx, calcis). Some elements have -ium because they're derived from stems that have an -i at the end, like the ones listed above. But others, like lithium, don't actually have an -i at the end of the stem etymologically (cf. Ancient Greek lithos). There, it seems instead that the people in charge of the naming reanalyzed the original pattern of adding a plain -um to the end of the stem, and instead added -ium on analogy to several elements formed from Greek or Latin -i stems.
And to make it official, the IUPAC (chemistry's governing body) now basically requires any new element to end in -ium (since 2002), so that's pretty much all we're going to see from here on out. (Source: http://media.iupac.org/publications/pac/2002/pdf/7405x0787.pdf)