Yes, your book is correct.
To rationalize it, throw away hydridization and electron exitation and work with concept of three-center four-electron bond. The idea is that in $\ce{XeF2}$ the real structure is a mix of $\ce{F-Xe+...F^-}$ and $\ce{F^-...Xe+-F}$, so the bonding is half-covalent and half-ionic. By packing more fluorine around the xenone atom, we force xenone into state with higher effective atomic charge, so the ionic part of the bonding becomes stronger as $\ce{Xe^{3+}}$ pulls fluorine anions much stronger.
same logic applies to many other cases, in particular sulfur oxides and sulfur fluorides.
For context, I quote experimental bond lengths from paper. Note, the paper itself quotes the values from other sources with slightly different values from different sources.
XeF2 ... 1.977
XeF4 ... 1.94
XeF6 ... 1.94 & 1.850 (two bond types)