Solomon and Fryhle $11^{th}$ edition says :
Order of reactivity of HX in the acid catalyzed conversion of alcohols to alkyl halides follows the order : HI>HBr>HCl
We can account for this by saying that the acidity order is also HI>HBr>HCl and hence the $-OH$ is easily protonated by the more acidic halide but the reaction is already acid catalyzed by a strong acid and the conversion of $-OH$ to $-H_2O^+$ is fast so the acidity of $HX$ should make a negligible difference.
The slow step of the reaction is the formation of the carbocation and that does not seem to involve the halide so how can we explain this order ?