Let's do a simple generic example. We have a reaction:
$$\ce{2A <=>[k_1][k_{-1}] B}$$
Experimentally we determine that the rates of the forward and reverse reactions are:
$$\text{Rate}_1=k_1[\ce{A}]^2$$
$$\text{Rate}_{-1}=k_{-1}[\ce{B}]$$
At equilibrium:
$$\text{Rate}_1=\text{Rate}_{-1}$$
$$k_1[\ce{A}]^2=k_{-1}[\ce{B}]$$
Since the rate constants are constants we can combine them into the equilibrium constant to generate the law of mass action:
$$K_{eq}=\frac{k_1}{k_{-1}}=\frac{[\ce{B}]_{eq}}{[\ce{A}]_{eq}^2}$$
Now, if $k_1=2k_{-1}$, we have a value of the equilibrium constant, which gives us the relationship between $[\ce{A}]_{eq}$ and $[\ce{B}]_{eq}$ that defines equilibrium:
$$K_{eq}=\frac{k_1}{k_{-1}}=\frac{2k_{-1}}{k_{-1}}=2$$
$$K_{eq}=\frac{[\ce{B}]_{eq}}{[\ce{A}]_{eq}^2}=2$$
$$2{[\ce{A}]_{eq}^2}=[\ce{B}]$$
In this case, only concentrations of $\ce{A}$ and $\ce{B}$ that satisfy $2{[\ce{A}]_{eq}^2}=[\ce{B}]$ constitute equilibrium. Other combinations of concentrations are not at equilibrium and therefore the forward and reverse rates are not equal.
Let's see how this works. An equilibrium situation is $[\ce{A}]=1\text{ M}$ and $[\ce{B}]=2\text{ M}$ because:
$$\frac{[\ce{B}]_{eq}}{[\ce{A}]_{eq}^2}=\frac{2}{1^2}=2$$
The rates are:
$$\text{Rate}_1=k_1[\ce{A}]^2=2k_{-1}[\ce{A}]^2=2k_{-1}(1^2)=2k_{-1}$$
$$\text{Rate}_{-1}=k_{-1}[\ce{B}]=k_{-1}(2)=2k_{-1}$$
However, what if $[\ce{A}]=[\ce{B}]=2\text{ M}$?
$$\frac{[\ce{B}]_{eq}}{[\ce{A}]_{eq}^2}=\frac{2}{2^2}=\frac{1}{2}\ne K_{eq}$$
$$\text{Rate}_1=k_1[\ce{A}]^2=2k_{-1}[\ce{A}]^2=2k_{-1}(2^2)=8k_{-1}$$
$$\text{Rate}_{-1}=k_{-1}[\ce{B}]=k_{-1}(2)=2k_{-1}$$
The concentrations, when plugged into the mass action expression, do not match the relationship defined by the equilibrium constant. More importantly, the rates are no longer equal. In this example, the rate of the forward reaction is now 4 times the rate of the reverse reaction. This is how equilibrium is reestablished. The forward reaction is going faster, but as it does so, $\ce{A}$ is consumed (slowing down the forward reaction) and $\ce{B}$ is produced (speeding up the reverse reaction) until the rates are equal again.
We can even do a bit of algebra to figure out what the concentrations of $\ce{A}$ and $\ce{B}$ will be once equilibrium is reestablished. Let's have $x$ represent the change in concentration of $\ce{B}$, so the change in the concentration of $\ce{A}$ is $-2x$.
$$K_{eq}=2=\frac{[\ce{B}]}{[\ce{A}]^2}=\frac{2+x}{(2-2x)^2}=\frac{2+x}{4-8x+4x^2}$$
$$2(4-8x+4x^2)=2+x$$
$$8-16x+8x^2=2+x$$
$$6-17x+8x^2=0$$
$$x=0.44695..., 1.6781...$$
In our case, we must use $x=0.44695$ since the other value would lead to a negative concentration for $\ce{A}$. Thus:
$$[\ce{A}]_{eq}=2.00-2(0.44695)=1.11$$
$$[\ce{B}]_{eq}=2.00+0.44695=2.45$$