I realised where my confusion lies - so I'm leaving an answer so that other people learning chemistry for the first time are able to fully understand what's actually going on about Avogadro's number.
We have to make a clear distinction between Avogadro's number and Avogadro's constant.
The former is a unitless real number, defined as $6.02214076\times 10^{23}$.
The latter is a physical (or shall I call chemical?) constant, and thus has a unit. As @Poutnik and other people have commented on my question, the fact that $mol$ does not have a dimension doesn't mean it can't be a unit (the same applies to radian as a measure of angles); and that means there should exist a constant called the Avogadro's constant with the dimension $mol^{-1}$ so that the product of the two is dimensionless, i.e.
$$n \ [mol] \ \times N_A \ [mol^{-1}] = N \ [particles]$$
Here it's trivial that [particles] is not a unit, simply because we did not feel the need to (and thus chose not to) assign any unit, dimension, and whatever onto that thing called number of particles.
Again, although $mol$ is dimensionless it should still be treated as a unit so in order to cancel that out to produce a unitless number $N$ we can conclude that we need a number with value $6.02214076\times 10^{23}$ and unit $mol^{-1}$, or namely Avogadro's constant.