Why fossil fuel is too dangerous?
Combustion of fossil fuel does not only release $\ce{CO2}$ but many other harmful substances(pollutants) like oxides of sulphur ($\ce{SO_{x}}$), oxides of nitrogen($\ce{NO_{x}}$), Carbon Monoxide($\ce{CO}$), etc...
oxides of sulphur ($\ce{SO_{x}}$):
When sulphur containing fossil fuel is burnt, then oxides of sulphur is produced. Sulphur Dioxide($\ce{SO2}$) is generally produced.
Low concentration of $\ce{SO2}$ causes diseases like asthma, swelling and irritation of respiratory tract.
High concentration $\ce{SO2}$ leads to stiffness of flower buds, which eventually fall off!
Oxides of Nitrogen($\ce{NO_{x}}$):
When in automobile engine, fossil fuel is burnt at high temperature significant amount of nitric oxide($\ce{NO}$) and nitrogen dioxide($\ce{NO2}$) is released.
$\ce{NO2}$ is lung irritant. it is highly toxic to living tissue.
$\ce{NO2}$ decrease the rate of photosynthesis, cause respiratory diseases in children, etc.
Carbon monoxide($\ce{CO}$):
$\ce{CO}$ is produced due to incomplete combustion of carbon. It is major constituent in smoke of vehicles.
$\ce{CO}$ binds with haemoglobin and form carboxyhemoglobin, which is 300 times more stable than usual oxygen-haemoglobin! If concentration of carboxyhemoglobin reaches 3 to 4% then oxygen carrying capacity of blood reduces.
Few more!
$\ce{SO2}$ with smoke and fog is responsible for classical smog(London smog)
$\ce{NO2}$ and $\ce{NO}$ is responsible foe photochemical smog(Los-Angeles smog)
Update:
I was reading more about environmental chemistry and i discovered that $\ce{NO_{x}}$ is responsible for 40% depletion of concentration of Ozone layer!
why are scientist more concerned about fossil fuel instead of CFC?
As commented by LDC3 The production of the more hazardous CFC's has been banned.
There are 95 derivatives of chloro and bromo fluorocarbon which are listed as ODS(Ozone Depleting Substances).
On 16th September, 1987 93 countries sign Montreal Protocol and accept not to use ODS.
Reference: GSEB textbook pdf