The main issue I see here is that there's too much fixation on how the equation normally looks. Instead, you would find much more insight into looking at the physical meaning conveyed by the equation.
For example, you can plug in initial and final values to see if the function describes what we expect.
At the beginning of the reaction, $x=0$. Your equation gives $\frac{\mathrm{d}x}{\mathrm{d}t} = k\ce{[A]}_{0}\ce{[B]}_{0}$. This is exactly what's expected, as this is the maximum value of the rate of product formation.
Let's arbitrarily assume that $\ce{[A]}\geq \ce{[B]}$. Then the reaction is done when $x = \ce{[A]}_{0}$. If you plug this value into your equation, you find that $\frac{\mathrm{d}x}{\mathrm{d}t} = 0$ which is also correct.
And as Rodrigo pointed out, the main reason you have sign confusion is that you've chosen to represent the concentration of product as opposed to the concentration of reactant, and since one is produced while the other is consumed, you get an extra minus sign.