It would be tempting to argue that fluorine is so electronegative and holds its electrons so tightly that their polarizability is reduced, thus so are the dispersion forces in $\ce{F2}$. But upon further review, this does not stand up. We would expect nitrogen, being less electronegative than oxygen, to offer more polarizability still, yet molecular nitrogen boils at a temperature lower than both oxygen and fluorine (-196 °C). The real question is why $\ce{O2}$ boils higher than both neighboring diatomic molecules, $\ce{N2}$ and $\ce{F2}$.
Putting two and two together
What really distinguishes oxygen from its neighbors is its existence as a diradical, which arises from the degeneracy of its partially filled π molecular orbitals. This creates the possibility of interaction between unpaired electrons from different molecules.
Such an interaction is described, in terms of the magnetic properties of liquid oxygen, in this answer. Pairs of oxygen molecules tend to have "sticky collisions" in which they are indeed temporarily dimerized through bonding between their unpaired electrons. This counts as an attractive interaction that selectively occurs upon condensation and it is just enough to favor condensation at a slightly higher temperature than both neighboring, non-radical diatomics.
When one (and one) is enough
Another comparison is possible with the monoxides of carbon and nitrogen, versus $\ce{O2}$ and each other. Both $\ce{CO}$ and $\ce{NO}$ are weakly dipolar as well as having dispersion forces, but $\ce{NO}$ has the potential to dimerize like $\ce{O2}$ whereas $\ce{CO}$ does not. Here is how boiling points compare in this series:
Nitric oxide combines a weak dipole with the dimerization capability between molecules with an unpaired electron, forming a dimer. The tendency of nitric oxide to dimerize, though weak, is stronger than that in $\ce{O2}$, so this interaction produces a larger effect on boiling point with the nitrogen oxide.