As with all the periodic trends concerning differences inside a single group, it will have something to do with the size of the atoms involved.
You already noted that fluorine is the smallest of the halogen atoms and iodine is the largest. This is due to there being only two core electrons in fluorine while iodine has 46. While adding an electron to any of these will increase their size, the size differences between them will remain the same due to said core electrons; hence, fluoride will be the smallest and iodide will be the largest.
This means that fluoride has a high charge-per-volume ratio while iodide’s is much lower (compare that to Pearson’s HSAB concept — it’s the same idea). A higher negative charge density means that it draws cations towards it better and thus stabilises the crystal structure. A secondary effect is that the smaller size of fluoride means that it must have a lower coordination number in the crystal structure, thus having less positively charged neighbours, thus attracting them more.
Together, these influences mean that it is just harder to break up the high ordering of the crystal structure to melt the salts the higher up you are in the group.
A higher degree of covalent bonding for iodine may play a role, but I would deem it neglectable.