TL;DR - Bond angle strain gets a lot worse in going from cyclpropyl chloride to the transition state for the 3-membered ring example, than for cyclopentyl chloride going to the corresponding transition state.
The rate for each reaction will be determined by the free energy barrier in each case. Cyclopropyl chloride is strained because the carbons "want" to be tetrahedral, with sp3-hybridized orbitals (109° bond angles), but are forced to have much more acute (60°) bond angles within the ring. However, at the transition state, the carbon undergoing nucleophilic attack "wants" to be trigonal planar, with sp2-hybridized orbitals (120°), but is still geometrically-constrained to 60° bond angles within the ring. This makes the free energy barrier for SN2 very large.

Cyclopentyl chloride has bond angles close to the ideal 109°. There will be some bond angle strain at the transition state, but the effect won't be nearly as bad as for the three-membered ring.

Now, can the effect be generalised? Mostly. As expected, cyclobutyl chloride reacts faster than cyclopropyl, but slower than 5 or 6-membered rings. Five vs six is a tricky one. Cyclopentyl chloride is actually slightly more reactive. If you look at cyclohexane, it is very stable in the chair conformation. All nice bond angles, no eclipsing interactions. At the transition state for SN2 with cyclohexyl chloride, bond angle strain is introduced and there is steric interaction between the incoming nucleophile and the adjacent axial hydrogens. Cyclopentyl chloride doesn't start off quite so perfect - there are some eclipsing interactions. Also, the transition state may actually even relieve some of these eclipsing interactions.
Beyond six-membered rings, things get very complicated quickly. You could apply the same approach, but the number of possible conformations quickly rises from 8-membered and above. It becomes difficult to take all the possible factors and conformations into account, so without high-level calculations, it is difficult to make predictions.
If you have access, the reactivity of 3-6 member cycloalkyl chlorides was included in a publication by Rablen et. al..[1] It includes calculated energy barriers for SN2 reactions of these cycloalkyl chlorides with cyanide ion as nucleophile.
Reference:
- Rablen, P. R.; Mclarney, B. D.; Karlow, B. J.; Schneider, J. E. How Alkyl Halide Structure Affects E2 and SN2 Reaction Barriers: E2 Reactions Are as Sensitive as SN2 Reactions. J. Org. Chem. 2014, 79 (3), 867–879 DOI: 10.1021/jo4026644.