Here is a chart of the solubility of the species in water and the log of the partition coefficient you noted.
solubility solubility
MW g/100ml water moles/liter log(P)
Benzene 78.11 0.179 (1) 0.0229 2.13
Flororbenzene 96.103 0.155 (2) 0.0161 2.27
Cholorbenzene 112.56 0.050 (2) 0.0044 2.84
Bromobenzene 157.01 0.041 (2) 0.0026 2.99
(1) Wikipedia
(2) Cation Complexes of Compounds Containing Carbon-Carbon Double Bonds. VI. The Argentation of Substituted Benzenes
L. J. Andrews, R. M. Keefer
J. Am. Chem. Soc., 1950, 72 (7), pp 3113–3116
DOI: 10.1021/ja01163a085
weblink http://pubs.acs.org/doi/abs/10.1021/ja01163a085
There is a Wikipedia chart in article on Ionic radii.
Ion radii (picometers)
F- 119
Cl- 167
Br- 182
The halide on a benzene ring is going to be relatively covalently bound. It definitely won't dissociate to form ions and only very moderately participate in hydrogen bonding.
So as the size of the halide increases the "volume" of the molecule increases. Thus the molecule would preferentially like to be in the non-polar phase.
Here is a scaled image of fluorobenzene molecule:

and a scaled image of the bromobenzene molecule.

Haloalkanes
The "anomalous" fluoroalkane is explained in a different manner. To get an n-alkane that has a boiling point above room temperature we need to go to n-pentane.
BP solubility solubility
(deg C) MW g/100ml water millimoles/liter
n-Pentane 36 72.15 0.040 (1) 0.55
1-Fluoropentane 63 90.14 1.270 (2) 14.1
1-Chloropentane 108 106.6 0.197 (3) 1.85
1-Bromopentane 130 151.05 0.127 (4) 0.838
(1) Wikipedia
(2) NIH
(3) Wikipedia
(4) Handbook of Aqueous Solubility Data, Second Edition
By Samuel H. Yalkowsky, Yan He, Parijat Jain
There are two competing effects.
(1) The size of atom bound to the alkane.
Br > Cl > F > H
Scaled model of Fluoroethane

(2) The second effect is the electronegativity of the X group in the $\text{Alkane}\text{X}$ bond.
F > Cl > Br > H
4.0 3.2 3.0 2.2
The fluorine atom has a slightly larger radius in a fluoroalkane than the corresponding C-H group. But the halide groups allow more dipole forces. So by "volume" the overall effect is:
fluoroalkane
alkane
choloroalkane
bromoalkane