Small and/or highly charged ions tend to be highly solvated in water. These solvated ions may still be smaller than the solvated ions of larger ions.
However, water molecules are not as free as the molecules of a gas. Water has a much higher viscosity, for instance. So think of water as a loose gel. A large (unsolvated, or poorly solvated) ion has an easier time slipping between gaps in this gel. A smaller, more highly solvated ion is, by definition, more tightly bound to the gel network, and moves more sluggishly, as if it has to carry more of the network with it. This approach is more explanatory than predictive (see figure).
There are lots of numbers, some agree, some disagree with any simple explanation. But a clean example is sodium vs potassium: sodium bare ion is smaller than potassium bare ion, but their hydrated ions reverse that (hydrated potassium ion is smaller than hydrated sodium ion - and check out lithium!). So potassium has a greater conductivity, for the same number of ions.