The first pair is not stereoisomers. You have correctly explain why. When you are dealing with stereoisomers, that's what you have to do first: Identify stereo centers. Accordingly, each compound of second pair has two stereo centers. Then, mark (R,S) configurations of all stereo centers. As I marked, the first compound has (R,S) configuration, while second compound has (R,R) configuration.
Fact: The mirror image of any stereo center has the opposite configuration. For example, mirror image of (R)-glyceride is (S)-glyceride.
Now, let's look at second compound of this pair:
The configurations of this compound's two stereo centers (right structure) are (R,R). Thus, the configurations of two stereo centers of its mirror image should be (S,S). Thus, the stereoisomers with (R,R) and (S,S) configurations are enanthiomers. All other stereoisomers related to them are diastereomers. Therefore, the pair is diastereomers since they are (R,R) and (R,S).
I also want to emphasize one more point. When a compound has two or more stereocenters, it is beneficial to see any symmetry within them. Each stereo center of the pair contains identical set of groups attached to it ($\ce{H, OH, COOH}$). Thus, if these two stereo centers have opposite configurations, each stereo center is the mirror image of the other (see the left structure). Therefore, it has a plane of symmetry and hence, optically inactive. These compounds are called meso-compounds (They do not have their relevant enanthiomer).