Oxygen tries to draw electron density from the two moieties it's bound to, thus making both bonds more polarized than the normal carbon-carbon bond. Some functional groups offer more resistance than others to this polarization depending on their electronegativity. For this reason, if one of the groups "puts up more of a fight" than the other, oxygen will draw more electron density from the weaker one.
In your case, hydrogen is always the same since we're always talking about alcohols. Alkyl substituents donate electron density to the oxygen atom, so it is less eager to draw electron denstity from the hydrogen bond and will make it less polar. Change the alkyl for a primary alkyl chloride for example and the effect will be reversed.