To a certain extent, you can predict the relative solubilities of two salts, especially when there is a large difference in solubility. The existing answer mentions Fajans' rules and obviously it is quite easy to predict which of $\ce{NaCl}$ and $\ce{AgI}$ is more soluble.
But I'm inclined to be pessimistic and say no, this is not generally applicable.
You're perhaps on the right track. The dissolution of the salt arises from a couple of measurable quantities: the lattice enthalpy of the salt, and hydration enthalpies of the ions. You could measure both, and try to come up with a trend.
However that's often far easier said than done. The trends are sometimes quite subtle, and even if you have all the relevant data and limit yourself to a "simple" system like the alkali metal halides, you'll find that the interplay between two factors leads to a very delicate balance, and the resulting trend is quite complex. I don't think it's something that we can predict solely in our heads without doing any measurements.
For example, if you simply compare $\ce{NaF}$ and $\ce{NaI}$, $\ce{NaI}$ has a smaller lattice enthalpy (favours dissolution) but the $\ce{I-}$ ion also has a smaller hydration enthalpy (disfavours dissolution). So which factor wins out? There's no simple heuristic or rule of thumb that applies here, not any that I know of, at least.
How about two obviously more different salts? Let's try $\ce{NaF}$ and $\ce{MgF2}$. $\ce{NaF}$ has a smaller lattice enthalpy (favours dissolution) but also the sum of the hydration enthalpies is smaller than $\ce{MgF2}$ (more ions and $\ce{Mg^2+}$ is more charged than $\ce{Na+}$ - which disfavours dissolution of $\ce{NaF}$). Again we have the same dilemma.
In fact, that's just the enthalpic changes, so only half the story. There's also entropy to care about, which can either be positive or negative (and it's extremely difficult to predict this!).