Here it is stated that a molecule with a rotation axis cannot have dipole moment perpendicular to the axis, as any dipole moment would be changed upon rotation about this axis (which preserves the molecular geometry and hence the dipole).
The ammonia molecule inverts itself roughly 25 billion times per second (https://en.wikipedia.org/wiki/Nitrogen_inversion , http://courses.washington.edu/phys432/NH3/ammonia_inversion.pdf). Thus averaged over a long time the dipole moment of an individual ammonia molecule will be zero, as for every moment there is a dipole pointing in one direction, this will be cancelled at a different time by a dipole in the opposite direction (as the system is charge neutral the dipole takes the same value independent of where you place the origin, so molecular motion won't affect this)
However note the word "long" in the above. Any process that happens on a short time scale will not experience this long time averaging of the property, but rather the instantaneous, non-zero, value of the dipole. So what is short and what is long in this context? Well 25 billion inversions per second means that 1 inversion takes 0.04ns. How far does a molecule move in that time? Well taking an order of magnitude value from https://en.wikipedia.org/wiki/Thermal_velocity of 400m/s a molecule moves 160 Angstroms in the time it takes one inversion of ammonia to take place. This is a HUGE distance for a molecule, thus the motion is dictated by the instantaneous, non-zero, value of the dipole. Thus we talk of Ammonia as a polar species.
At the molecular level, ammonia has a dipole that is parallel to the axis of rotation. At any point in time, the molecule is trigonal pyramidal. Although it can undergo inversion, the trigonal planar structure, with no dipole, is a transition state, with infinitesimal life-time.