There are three mechanisms to discuss here: $S_N2$, $S_N1$ for $RX$ and nucleophilic aromatic substitution by addition-elimination for $ArX$:
$S_N2$ and $S_N1$:
(Image from https://www.quora.com/Why-steric-hindrance-doesnt-affect-Sn1-reaction)
$S_N2$ is concerted (one step), so this one step is rate determining, or the slowest of the mechanism. In $S_N1$, formation of the carbocation is rate determining. In both rate determining steps, the leaving group is being kicked off. This means the easier it is for the $C-X$ bond to break, the faster the reaction goes. The bond enthalpy decreases as size of halogen increases, giving the order you stated above.
(Image from https://www.masterorganicchemistry.com/2018/08/20/nucleophilic-aromatic-substitution-nas/#seven)
In this case, step 1 is rate determining, which does not involve the leaving group departing. The halogen, however, can speed up the reaction by stabilising the negative charge formed during this step. The stronger the inductive effect of the halogen - which outweighs the +M mesomeric effect - the more it can draw the negative charge from the ring, stabilising it. The bond strength is irrelevant here, as the step during which it is cleaved is faster than the previous step regardless, and is always 'waiting around' for the first step. This gives the reverse order to the previous one.