With an analogous mechanism: C bonded to X has a δ+ charge and attacks the pi electrons to form a carbocation intermediate, with X- as a leaving group. X- then adds to the carbocation to neutralize it.
Why doesn't this occur?
If steric hinderance is the answer, what about MeX adding to a terminal alkyne?
RCH2X + OH- → RCH2OH + X- proceeds (with comparable/less hinderance?) so it seems unlikely that this the reason.
H-X bonds are more polar than C-X, but the electronegativity difference between C and H is small enough that we consider C-H bonds nonpolar. Electron-deficient C can pull some electron density from adjacent carbons if there are any - but again, what about MeX? Is the presence of 3 hydrogens enough to stabilize the C-X bond to the extent that the addition doesn't take place?