Seems like the silicon atom is more oxophilic as compared to the phosphorus atom. This could in principal be explained by the higher strength of the $\mathrm{Si-O}$ bond, as indicated by the dissociation energies $D$ (see http://www.wiredchemist.com/chemistry/data/bond_energies_lengths.html): $$ \mathrm{Si-O}:~~~~D=\mathrm{452~kJ/mol};~~~~r_b=163~\mathrm{pm} $$
$$ \mathrm{P-O}:~~~~D=\mathrm{335~kJ/mol};~~~~r_b=\mathrm{163~pm} $$ The $\mathrm{Si-O}$ bond is stronger thus given an explanation why the Peterson olefination is favoured. It seems that the bond lengths $r_b$ are the same, though.
By the way, a nice discussion of oxophilicity was given recently in Inorg. Chem., 2016, 55 (18), 9461.