What is safe very much depends on your experience with handling of dangerous substances, your equipment, storage conditions etc. It is hard to give a general answer, but from your question it appears that you have little experience with chemicals and assessing how hazardous they are in certain conditions. So my advice would be: Stay on the safe side and do not take any risks you cannot assess. However, I think it would be better if you made the decision yourself, rather than based on a list or recommendation by anonymous people on some random internet forum.
One approach could be to visit a website of one of the major chemical suppliers. They usually sell quite a few elements, but also are obliged to supply the material safety data sheet (MSDS), at least in Europe. Have a look at them, they are informative. For example, an MSDS for silver I just had a look at states on the first page:
Not a hazardous substance or mixture according to Regulation (EC) No 1272/2008.
You can sefely conclude that silver will not be dangerous. For lead, the MSDS says:
Reproductive toxicity (Category 1A), H360FD
Effects on or via lactation, H362
Specific target organ toxicity - repeated exposure, Oral (Category 1), Central nervous
system, Blood, Immune system, Kidney, H372
For the full text of the H-Statements mentioned in this Section, see Section 16.
So yes, lead is dangerous in some ways, especially if it enters your body, but for one big solid lump that does not happen accidentally so easily, in contrast to powder for example which you could breathe in. You can look at the H and P (hazard and precautionary) statements in the MSDS if you want to get a better impresssion. But for a proper assessment, you would have to find some competent person you trust. I would say that a piece of lead that you only look at and do not manipulate in any other way is harmless, but would suggest that you make up your own mind.
One note for MSDSs: They read rather drastic usually, and I assume this is on purpose. So if you want to be on the safe side, they are a good indicator.
Just to let you know: Cobalt, one of the elements you already have, reads like this:
Flammable solids (Category 1), H228
Eye irritation (Category 2), H319
Respiratory sensitization (Category 1), H334
Skin sensitization (Category 1), H317
Germ cell mutagenicity (Category 2), H341
Carcinogenicity (Category 1B), H350
Reproductive toxicity (Category 1B), H360F
Long-term (chronic) aquatic hazard (Category 4), H413
For the full text of the H-Statements mentioned in this Section, see Section 16.