It is a common fact that we have iron in our blood. But what if we had something different in its place? Like instead of iron what if it was copper, or silver, or what if we had gold in our blood stream instead? Theoretically, what would that do?
To start, transition metal complexes with large porphyrin-like molecules are so electronically complex that standard chemical intuition doesn't apply and you have to either do the experiment or run a (nearly impossible) calculation to determine what effect a non-physiological metal would have on the protein activities. In the case of hemoglobin, your best bet would be to think about metals which have two accessible oxidation states because the interconversion of Fe(II) and Fe(III) is critical to the function of hemoglobin.
To give a broader answer, let's think about the general question of substituting non-native metals into a protein. This is, in fact, an very interesting and open question. On smaller metalloproteins there have been studies probing whether nature 'picked' the best metal for a task or just a convenient/abundant one [1,2]. The focus of those studies is on catalysis, and there are cases where many non-physiological metals can function in the protein and even catalyze the reaction as good or even better than the wild type.