Recall that SN1 heavily favors protic solvents (polar/H+). In such cases, the nucleophile to-be-leaving-group (to be Nu-) is going to be attracted to the positively charged molecules of the protic solvent (...H+) and away from the molecule it is attached to. The power of the attraction comes from the dipole between C-Nu, where Nu has the negative bit of the dipole.
At the end of this step, when the nucleophile attached group leaves, you often see "Nu-" hanging around, but really it's with the solvent, e.g. H3O+Br- , and it's fairly common to omit the solvent here and in other similar ionic situations, if I'm not mistaken.
Examples that invoke hydrogen bonding make this more apparent.
F-C...(tertiary) + H2O
The dipole here looks like (δ-)F-C...(δ+) because F is more electronegative
The F is strongly attracted to the closer H: (δ+)H-(δ-)O-H(δ+)(δ-)F-C...(δ+) (pretend the HOH is bent :)