Le Châtelier does apply to pure solids and liquids - what happens is that pure solids and liquids experience almost no change in concentration[*] in chemical equilibria, so for all intents and purposes they are not affected by Le Châtelier considerations.
Heterogeneous systems
Imagine a system with two phases; for instance, a gas phase and a solid phase, and we have some sort of equilibrium between them; for instance if we put water in a closed container (remember that Le Châtelier only properly applies to closed systems), it reaches an equilibrium with its own vapour:
$\ce{H2O(l) <=> H2O(g)}$
Here we have two phases: a liquid phase and a gas phase. The molecules in both are the same but in different aggregation states, which crucially differ in a fundamental behaviour regarding volume: in the liquid, molecules occupy a fixed volume each in certain conditions. If you have twice the number of molecules, you get twice the volume, and the density remains constant. So, if the equilibrium above is displaced to the left, the concentration of water molecules in the liquid phase doesn't change: the size of the liquid phase does. You simply get more volume of liquid. The same is true about solids.
For the gas it's different: the gas phase will always occupy the whole free volume in the container, and what changes is its pressure (which is to say, its concentration). If you double the number of gas molecules, the volume doesn't change - it still fits the whole container - but their concentration doubles.
This is a simplified picture; formally, the concentration of real solids and liquids is not exactly constant. However, it is much, much less sensitive to conditions than gases or solutes, so for all practical purposes it's a constant. See the next section for a numerical example that also applies here.
Dilute homogeneous systems
When we deal with dilute homogeneous systems (only one phase), the concentration of the solvent can be safely treated as constant because it's so much larger than anything else that changes simply don't impact it.
Imagine that you are applying Le Châtelier to the self-ionisation equilibrium of water; if you have a sample of pure water, a very small fraction of the molecules dissociate per the following reaction:
$\ce{2H2O<=>H3O+ + OH-}$
If you have a pure water in equilibrium and you add $\ce{OH-}$ ions, Le Châtelier predicts that this will displace the equilibrium to the left, and some of the previously dissociated water molecules will associate again to form $\ce{H2O}$.
However, water itself is the medium of the reaction, so its concentration is pretty much constant. Let's calculate some actual figures to get an idea. The self-ionisation constant is $\mathrm{K_w=10^{-14}}$, so for a chemically pure sample of water we'd get these concentrations:
$[\ce{H3O+}] \approx 10^{-7} \mathrm{M}$
$[\ce{OH-}] \approx 10^{-7} \mathrm{M}$
$[\ce{H2O}] \approx 55.55 \mathrm{M}$
Now, imagine we add a lot of $\ce{OH-}$ ions - and we make their concentration a thousand times bigger than before - that will have a big impact on the concentrations of both $\ce{H3O+}$ and $\ce{OH-}$. When we reach equilibrium we'd have
$[\ce{H3O+}] \approx 10^{-10} \mathrm{M}$
$[\ce{OH-}] \approx 10^{-4} \mathrm{M}$
Note that this means that 99.9% of the dissociated molecules have been displaced to the left and formed water. However, this number of new water molecules (around 0.0000000999 M) is simply irrelevant compared with the previous concentration of water (55.55 + 0.0000000999 is still 55.55 for all practical purposes). Even further, as with heterogeneous systems, even if the addition of more water were relevant, its impact would be increasing the volume of the phase, not changing its concentration.
That's why the concentration of water isn't even included in the self-ionisation constant - yes, formally the equilibrium depends on $[\ce{H2O}]$, but since that's pretty much always the same, it's easier to factor it into the constant and write the equilibrium as $\mathrm{K_w=[\ce{H3O+}][\ce{OH-}]}$.
[*] More precisely, activity