Both alkoxides and hydroxides are not great leaving groups, but since we are just comparing their leaving group character...
First of all, leaving group tendency does not depend on nucleophilicity. Rather, it depends on basicity. Both of these are often confused to be the same thing. Yes, it's true that coincidentally both of these, in most cases, suggest similar things, but they are not the same. Let us compare the two for OH and OR.
Nucleophilicity:
OH is a better nucleophile than OR since it has more mobility(due to its small size)than OR. A good nucleophile should be small in order to attack sites easily.
Basicity:
This does not depend on mobility. It only depends on electron donating tendency (inversely proportional). Here, OR is a weaker base, hence it is a better leaving group