Just an observation I notice when around smoker friends or when I visit Las Vegas - polyester clothing doesn't seem to stay smelling of smoke for the rest of the day like my cotton shirts do (or also my hair for that matter).
Adsorption is a very complicated phenomenon which involves so many interactions. Smoke has thousands of chemicals too, therefore one cannot pin-point a single cause. It is just like the concept of stains. Some stains on cotton never come off easily, which means that molecular interaction is very strong between the dye in the stain and cotton fiber. As a rough estimate, the -OH groups on cellulose may be providing the hydrogen bonding sites to nitrogen containing aromatic (in a chemical sense) compounds, hence stick for a long time. I am drawing an crude analogy because silica which has plenty of silanols (Si-OH) groups bind amines too strongly.