Could density be defined as the measure of matter per unit volume instead of mass per unit volume because mass is a measurement of matter?
It depends, there are different kinds of densities: mass density, particle density, charge density, probability density ...
Depending on how your measure your matter, mass, number of particles, charge (e.g. for electrons), you could use the corresponding density.
"Matter per unit volume" would only work in a broader sense, since it does not specifically state what kind of measure you want to use. Most of the time one simply defaults on mass, but this depends on the specific field.