It comes down to the history of their discovery, natural occurrence and applications. It's a fascinating story that you can read about in here:
The rare earths (i.e. lanthanides) were discovered over a few decades in the late 1800s. They were laboriously separated from "rare earths" - the name used back then to describe the rare minerals (silicates, oxides, phosphates, etc) that included these elements. These are all naturally occurring rocks. You can literally pick up from the earth a mineral that has rare earths in it. They are also stable (apart from Pm). Therefore, they are used in industry. There are rare earth mines, rare earth magnets, rare earth lasers. You get the stuff from the earth.
Actinides, were mostly discovered in the early to mid 1900s, after rare earths were a "thing". Other than U and Th, you do not find them in nature. You do not mine them from the earth, and they do not exist in "earths". Their radioactivity precludes their use in industrial applications other than a few niche uses. Although their chemical properties are similar to the lanthanides, there are many differences. They are simply not rare earths. Almost, but not quite.
I'll also add that the definition of "rare earths" is a loose one. The broad chemical definition includes the lanthanides La–Lu, Y and Sc. As far as geologists care, Sc is not a rare earth element, and Y is only "honorary". Industry and mining usually consider the lanthanide oxides as REO (rare earth oxides) and use REY (rare earths + yttrium) when they are talking also about Y, because it's not obvious to them that it is a rare earth.