I only want to add some data and quotes to supplement long's answer. I am also indebted to long since the reference he provided was what helped me to find these pieces of information.
As already mentioned in long's answer, the chemical shift $\sigma$ arises from several components. Chief amongst them are the (local) diamagnetic term $\sigma_\mathrm d$ and the (local) paramagnetic term $\sigma_\mathrm p$. The diamagnetic shielding is generally proportional to charge density and is given by the Lamb formula. The paramagnetic shielding is generally proportional to $\left<r^{-3}\right>$ (the expectation value of $r^{-3}$ for valence p or d electrons) and is inversely proportional to $\Delta E$, which is an averaged electronic excitation energy. The origins of these terms and formulae to (approximately) evaluate them are discussed by Ramsey in a seminal paper.1 Due to several issues which made the terms difficult to calculate, modifications to this model were made by adding several other terms.2–5 However, the subdivision still conceptually stands.
The variations in chemical shifts of nuclei do not depend directly on the absolute magnitude of either $\sigma_\mathrm d$ or $\sigma_\mathrm p$. They depend, instead, on the extent to which these quantities vary. Of course, there is the issue that if $\sigma_\mathrm d$ is small, then it cannot vary much. This is the case for hydrogen, in which $\sigma_\mathrm d$ is small (low electron density) and $\sigma_\mathrm p$ is small (high $\mathrm{1s}\to\mathrm{2p}$ excitation energy). However, it turns out that for the large majority of nuclei, the variation in paramagnetic shielding controls the variation in chemical shifts.
The remainder will just be quotes. Günther's NMR Spectroscopy states the trend:6
12. Selected Heteronuclei (p 434)
Existing experimental data show that chemical shift ranges tend to increase with the atomic number $Z$ across a given row of the Periodic Table and down a given group. As discussed earlier for carbon-13, the significant contribution to the shielding of heavier nuclei comes from the paramagnetic term, $\sigma_\mathrm p$, in the general equation for the shielding constant, $\sigma$.
More detailed discussion is to be found in older books, some of which are probably out of print. In NMR and the Periodic Table, edited by Harris and Mann, there is a lot of information.7
3B.5 Chemical Shift Ranges (pp 61–62)
A summary of some of the chemical shifts of $\ce{^13C}$, $\ce{^14N}$, $\ce{^17O}$ and $\ce{^19F}$, relative to the bare nuclei, is presented in Fig. 3.3. The ranges of observed shifts for these nuclei are about 650, 930, 700 and 800 ppm respectively, whilst that for boron is about 200 ppm. A somewhat similar situation arises amongst the second row nuclei, where the approximate ranges for $\ce{^27Al}$, $\ce{^29Si}$, $\ce{^31P}$, $\ce{^33S}$ and $\ce{^35Cl}$ are 270, 400, 700, 600 and 820 ppm. Thus although the nuclei with the lower values of $\left<r^{-3}\right>_{np}$ tend to have the smaller chemical shift ranges in each row, the ranges do not follow the trends of $\left<r^{-3}\right>_{np}$ shown in Fig 3.1. The greater ranges shown by $\ce{^14N}$, $\ce{^17O}$ and $\ce{^19F}$, compared with $\ce{^11B}$ and $\ce{^13C}$, reflect the importance of lone-pair electrons. In general these electrons give rise to low-energy $n \to \pi^*$ transitions and thus to large values of $\sigma_\mathrm p$. If the lone pair becomes involved in bonding their contribution to $\sigma_\mathrm p$ is removed resulting in increased nuclear screening. [...]
The much smaller ranges observed for $\ce{^7Li}$ and $\ce{^23Na}$, about 10 and 15 ppm respectively, imply that only $\sigma_\mathrm d$ is of importance for these nuclei, since $\sigma_\mathrm p$ requires the presence of p valence electrons.
Some of the heavier nuclei have very large chemical shift ranges; for example $\ce{^59Co}$ and $\ce{^205Tl}$ have ranges of about 12000 and 34000 ppm respectively. These undoubtedly reflect the greater polarisability of the higher p and d valence orbitals and their varied commitments to chemical bonding.
Some values of $\sigma_\mathrm d$ and $\sigma_\mathrm p$ in compounds of second-row elements are also mentioned (p 55). Notice how small the variations in $\sigma_\mathrm d$ for a given nucleus are.
$$\begin{array}{cccccc} \hline
\text{Molecule} & \text{Nucleus} & \sigma_{\mathrm{d}}(\text{loc}) & \sigma_{\mathrm{d}}(\text{non-loc}) & \sigma_{\mathrm{p}}(\text{loc}) & \sigma_{\mathrm{p}}(\text{non-loc}) \\ \hline
\ce{CO} & \ce{C} & 259.36 & 0.03 & -206.24 & -4.92 \\
& \ce{O} & 395.39 & 0.64 & -367.39 & -7.25 \\
\ce{(CH3)2CO} & \ce{C} & 257.60 & -0.12 & -212.14 & 4.99 \\
& \ce{O} & 398.47 & 0.10 & -580.07 & 2.42 \\
\ce{H2CO} & \ce{C} & 257.77 & -0.10 & -208.24 & 5.61 \\
& \ce{O} & 397.80 & 0.10 & -651.47 & 1.85 \\
\ce{CO2} & \ce{C} & 255.16 & -0.25 & -126.79 & -6.94 \\
& \ce{O} & 397.32 & 0.12 & -269.51 & -3.55 \\
\ce{CH3CN} & \ce{C} & 258.94 & 0.07 & -159.96 & -5.91 \\
& \ce{N} & 326.75 & 0.07 & -300.57 & -5.27 \\
\ce{HCN} & \ce{C} & 259.08 & 0.11 & -157.53 & -6.10 \\
& \ce{N} & 326.70 & 0.06 & -301.30 & -5.04 \\
\ce{N2} & \ce{N} & 324.39 & 0.24 & 0382.84 & -8.62 \\ \hline
\end{array}$$
Multinuclear NMR, edited by Joan Mason, also has a wealth of information.8 A graph of the chemical shift ranges against $\left<a_0^3/r^3\right>$ is presented on p 61. I was too lazy to scan the book, so here's a photograph of it (which is actually pretty decent):

The periodic variation with atomic number of the ranges of chemical shifts observed for the nuclei of main-group and post-transition elements was pointed out in 1964 by Jameson and Gutowsky.9 The tendency of the ranges to increase across a given row of the Periodic Table, and down a given group, was compared with the similar variation of the radial term $\left<r^{-3}\right>$ for the valence p electrons in the free atom, since the paramagnetic shielding term $\sigma_\mathrm p$ has this radial dependence. Figure 1 shows the periodicity of the ranges as now known and the corresponding periodicity of $\left<r^{-3}\right>$ values. The relatively large shift ranges for the first row elements compared with heavier elements can be linked with well-known "first row anomalies" such as the readiness to form pπ bonds (C=O, C=C, N=N, etc.) and relatively high electronegativity.
One might rightly ask, "what is the physical basis for this?" The reason is that $\sigma_\mathrm d$ is a sum over all orbitals, and orbitals close to the nucleus contribute much more than orbitals distant from the nucleus; in particular, the 1s orbital is a very major contributor. However, since changes in diamagnetic shielding in molecules arise primarily due to the movement of valence electrons, the values of $\sigma_\mathrm d$ do not change much. This argument was already put forth extremely early, by Dickinson in 1950.10 $H'(0)/H$ in this paper is equal to what we now know as $\sigma_\mathrm d$.

From Table III it is seen that the outer or valence electrons give a very small contribution to $H'(0)/H$. [...] Thus it would be expected, except perhaps for the lightest atoms, that molecular perturbations of the outer electrons would have little effect on $H'(0)/H$ [...] The observed shifts of nuclear resonances from one molecule to another are often as large as, or greater than, the entire atomic diamagnetic correction for the atom concerned. The diamagnetic field at the nucleus for both molecules would be closely the same and hence the shift must be attributed chiefly to a difference in the second-order paramagnetic field in the two molecules.
References
(1) Ramsey, N. F. Magnetic Shielding of Nuclei in Molecules. Phys. Rev. 1950, 78 (6), 600–703. DOI: 10.1103/PhysRev.78.699.
(2) Saika, A.; Slichter, C. P. A Note on the Fluorine Resonance Shifts. J. Chem. Phys. 1954, 22 (1), 26–28. DOI: 10.1063/1.1739849.
(3) Pople, J. A. The Theory of Chemical Shifts in Nuclear Magnetic Resonance. I. Induced Current Densities. Proc. R. Soc. London, Ser. A 1957, 239 (1219), 541–549. DOI: 10.1098/rspa.1957.0060.
(4) Pople, J. A. The Theory of Chemical Shifts in Nuclear Magnetic Resonance. II. Interpretation of Proton Shifts. Proc. R. Soc. London, Ser. A 1957, 239 (1219), 550–556. DOI: 10.1098/rspa.1957.0061.
(5) Flygare, W. H. Magnetic Interactions in Molecules and An Analysis of Molecular Electronic Charge Distribution From Magnetic Parameters. Chem. Rev. 1974, 74 (6), 653–687. DOI: 10.1021/cr60292a003
(6) Günther, H. NMR Spectroscopy, 3rd ed.; Wiley-VCH: Weinheim, Germany, 2013.
(7) NMR and the Periodic Table; Harris, R. K.; Mann, B. E., Eds.; Academic Press, London, 1978.
(8) Multinuclear NMR; Mason, J., Ed.; Plenum, New York, 1987.
(9) Jameson, C. J.; Gutowsky, H. S. Calculation of Chemical Shifts. I. General Formulation and the Z Dependence. J. Chem. Phys. 1964, 40 (6), 1714–1724. DOI: 10.1063/1.1725387.
(10) Dickinson, W. C. Hartree Computation of the Internal Diamagnetic Field for Atoms. Phys. Rev. 1950, 80 (4), 563–566. DOI: 10.1103/PhysRev.80.563.