I guess the mistake you are doing is in the equation for $$K_a$$ for $$\ce{CH3COOH}$$. See if the following makes sense.
$$\frac{[\ce{H+}]^2}{[\ce{CH3COOH}]}=K_a(\ce{CH3COOH})$$ Let $$x$$ be the $$[\ce{H+}]$$ from acetic acid, and $$y$$ from the other organic acid. $$\frac{10^{-2}}{0.1-x}=10^{-5}$$ $$0.09+x+y=0.1 \text{ (Balancing for H+)}$$ $$\frac{y^2}{[\ce{CHCl2COOH}]}=K_a(\ce{CHCl2COOH})$$