Ozone is produced naturally when highly energetic UV radiation breaks down oxygen molecules and cause them to split apart in a process called photolysis. If a free oxygen atom collides with another oxygen moleule, it joins up, forming ozone.
$$\ce{O2 + UV -> 2 O}$$
$$\ce{O + O2 -> O3}$$
So, why most of the ozone resides in stratosphere?
The equilibrium concentration of ozone depends on the altitude and is maximum at 25km which is just above troposphere.
- Only a small amount of ozone accumulates in the troposphere because other UV photons break down ozone molecules, turning them back into oxygen. When ozone is being broken down as fast as it is formed, its concentration remains constant. The exact height where the ozone concentration is maximum is 25km.
- Ozone absorbs many of the high-energy UV photons, converting their energy into heat. This energy heats the air, and the ozone rich air rises above the troposphere. This higher temperature differentiates the stratosphere from the troposphere.
- Most of the high-energy UV photons is used up by stratospheric oxygen and no significant amount of these photons reach in the lower part of the atmosphere.
There are certainly other factors that plays a role. You can find more information in this question post from EarthScience.SE.