Part of your confusion lies in the all-too common belief that it's the ratio of ligand to binding partner (e.g. substrate and enzyme) which determines the amount of binding. That's not the case. Instead, it's the absolute concentration of free ligand which determines how much is bound.
Let's take a microscopic view, and look at a single enzyme molecule. The enzyme doesn't know or care about what the other enzymes are doing, it only knows about its own local environment. To bind a substrate molecule, it needs to encounter (collide with) a substrate molecule in the correct orientation. How frequently it does that is directly proportional to the amount of free, bind-able substrate molecules there are in solution. For bound enzymes, (ignoring catalytic turn over for the moment), transitioning to the unbound state is a first-order process, independent of the concentration of anything else in the solution. (The total macroscopic rate of unbinding events depends on the number of bound complexes in solution, but for an individual bound enzyme it's concentration independent.) So for a single enzyme molecule, the fraction of the time it spends in the bound state is only dependent on the concentration of free ligand, so the time average of a large number of molecules is going to reflect that same distribution, by the ergodic hypothesis.
If binding is difficult but release is easy, then you're going to need a large amount of substrate in order to push the rate of binding to be comparable to the rate of release. If binding is easy but release takes a long time, you won't need as much substrate. That's what the KD of a binder (and roughly KM of an enzyme) measures. It's the concentration of the free ligand which makes the effective on-rate an the off-rate identical, and thus results in enzymes binding about half the time (or equivalently, at any one time half the enzymes are in the bound state).
So it doesn't matter if you have 100 molecules of substrate for every molecule of enzyme - if you're dilute enough, the concentration of substrate will be too low, the rate of enzyme/substrate encounters will fall off, and any enzyme-substrate complexes which form will fall apart long before some other one can form.
Enzyme/substrate (and ligand/binder) ratios are important, but mainly because it's the concentration of free substrate, rather than the concentration of total substrate, that's involved in the calculation. As your enzyme binds substrate, it depletes the pool of free substrate. If your enzyme concentration is large in comparison to your concentration of substrate, substrate going to the enzyme-bound state could cause a significant drop in free substrate concentrations, leading the nominal substrate concentration to be different from the actual free substrate concentration. (This is also part of why people primarily work with initial rates - the amount of free substrate drops over time due to catalysis.)
Caveat: KM is not quite the binding affinity of an enzyme for the substrate. In addition to the standard off rate of dissociation, there's also the "off rate" due to catalysis. There's also the lifetime of catalytic intermediates and the product-bound state which comes into play. With typical Michaelis-Menten assumptions - single step catalysis which is very slow compared to binding, with rapid product dissociation - the difference is minimal, but for many enzyme systems those assumptions don't hold.