How can one quickly identify suspicious water molecules that might have been modelled wrong - or what one should be looking for if one suspects a particular water molecules to be something else.
You can check how many ligands the water has, and what the distances are. Most water molecules can safely be ignored, except for ones in an active site or so. When the protein is in solution, it is covered by water (and other particles making up the solvent) on the entire surface. In a crystal, you are missing some of these water because of crystal contacts to neighboring proteins, and because many water molecules are not sufficiently fixed in place ("ordered") to give significant electron density peaks.
Such 'density spheres' are in most cases water molecules, but could also contain a transition metal (e.g. Mg, Fe, Zn etc) or an ion such as SO4, Na, K or Cl depending on the crystallisation conditions.
As alluded to in the comments, most of your examples have much higher electron count than water, so they are unlikely to be confused with water. Sodium, magnesium, flouride and ammonium ions are closest in electron count, and you would have to use the nature and location of the neighboring atoms to make a call. See e.g. this paper.
[OP in comments] Thanks for the input! I just came across a couple of structure that I found to be poorly done when looking at the actual density maps - and this had made me a bit paranoid about structure quality when my structural analysis is important for an article. Its time consuming to look through the density maps of each individual structure when reviewing many different ones - and since B-factors and occupancy is available info in the pdb file, the process is more efficient if they could be used as an indicator of when one should also check the actual maps. But maybe the work just has to be done.
You should carefully look at those regions in the crystal structure (and density) that are important for your own work. The original paper might not have had your angle of interest, so there might be areas in the structure that were modelled with moderate attention (as opposed to the detailed scrutiny an active site would get when it comes to identifying bound ions that look like water). You should also look at which substances were used in crystallization. If there was a high concentration of - say - chloride, it is likely that you have some chloride ions bind somewhere on the protein surface.
Would it be correct to be suspicious of a water molecule with a very low B-factor, as this can indicated high coordination and thus most most likely should be a metal or ion of some kind instead of water?
Yes. It also could indicate an atom with more than 8 electrons (e.g. potassium).
If one suspect that an ion should be modelled in, how does one know if it should be Na, Cl or some other atom if there are many different ion's in the crystallisation condition? (The coordination of metals are not always perfect in proteins, and thus this might always be a speculation or more protein-class/fold dependant).
You look at the coordination sphere and the electrostatic potential. For cations you could have ligands like glutamate or aspartate, while you don't expect those for anions.
Could one say something about the water molecules based on occupancy with or within looking at the B-factor? Or are there some other good indicators one should be looking at?
Occupancies of water molecules below 1.0 are a red flag. You need very well-ordered waters in a high resolution structure to have enough data to support modelling of partially occupied sites. As for B-factors, you can order you water molecules by B-factor and number of hydrogen bonding partners. The lower the B-factor, the more consistent the B-factor of water and neighbors, and the higher the coordination number of the water, the more likely you would find a water there in reality (i.e. in solution in the cell).
[from the comments] You could check structures from PDB-REDO. They automatically remove waters that are not in the electron density map, and I think in some cases the removed waters are replaced with missing side chains.
If you go through the trouble and compare the deposited structure with the PDB-REDO database, I would not trust the PDB-REDO more than the deposited structure but rather treat all parts of the model that shows differences with caution.