Intuitively, yes, increasing the concentration of $\ce{O2}$ will drive the (quasi-)equilibrium of the fast reaction $\ce{O3 <=> O2 + O}$ towards the left, decreasing the concentration of the intermediate species $\ce{O}$, and thus slowing down the rate-determining slow reaction $\ce{O + O3 -> 2 O2}$.
A minor complication here is that the rate of the slow reaction is also proportional to the concentration of $\ce{O3}$, which driving the fast reaction towards the left side will increase. Indeed, in principle, the rate of the slow reaction would be maximized when the fast quasi-equilibrium concentration of $\ce{O}$ was equal to that of $\ce{O3}$, with any deviation from this in either direction slowing down the reaction.
The missing piece of the puzzle is that the combined reaction is called "the decomposition of ozone", not "the dimerization of monatomic oxygen". It is thus reasonable to assume that $\ce{O3}$ is at least initially present in considerable excess of $\ce{O}$. Indeed, given the extreme reactivity of monatomic oxygen, it's never likely to be present at more than trace quantities under typical conditions. Thus, converting some $\ce{O}$ back into $\ce{O3}$ by reacting it with $\ce{O2}$ will reduce the (already very low) $\ce{O}$ concentration by a much larger factor than it will increase the (comparatively high) $\ce{O3}$ concentration, and will thus slow down the rate-determining step of the reaction.
In any case, even if we did somehow (e.g. by running the reaction in near vacuum) contrive a situation where the quasi-equilibrium of the fast reaction was shifted so far to the right that the concentration of $\ce{O}$ became comparable to that of $\ce{O3}$, then we would also need to consider the additional reaction $\ce{2 O -> O2}$ whose rate depends (quadratically!) only on the $\ce{O}$ concentration. I would expect that a more thorough analysis (which would, of course, have to take into account the relative rate constants of the different competing reactions) should turn up no situation where adding $\ce{O2}$ would actually speed up the overall decomposition of $\ce{O3}$. Of course, I could be wrong; these things are not always intuitive.
$...$
to get italics. Use Markdown emphasis*...*
instead. $\endgroup$ – zwol Mar 12 '17 at 14:15