I (accidentally) stumbled upon the following statement in Atkins' "Elements of Physical Chemistry" (p378):
We represent dipole moments by an arrow with a length proportional to $\pmb{\mu}$ and pointing from the negative charge to the positive charge (1). (Be careful with this convention: for historical reasons the opposite convention is still widely used.)
Unfortunately he does not go into more detail. And I know this does not really answer your question.
The definition from the IUPAC is the same as the one used by Atkins:
electric dipole moment, $\mathbf{p}$
Vector quantity, the vector product of which with the electric field strength, $\mathbf{E}$, of a homogeneous field is equal to the torque. $\mathbf{T} = \mathbf{p} \times \mathbf{E}$. The direction of the dipole moment is from the negative to the positive charge.
The source quoted there is from 1993, so you can probably understand my surprise, when I did a little more searching and found in C. Párkányi's "Theoretical Organic Chemistry" (1997, p239):
[...] in organic chemistry the positive direction of the dipole moment is normally defined as the direction from the center of the positive charge towards the center of the negative charge. This convention prevails in physical organic chemistry and in inorganic chemistry. However, while the dipole still points from the positive charge to the negative charge, in physical chemistry and in chemical physics the positive direction of the dipole moment is defined in the opposite way, i.e., from the negative charge to the positive charge.
There are also a few sources given, but I currently have not enough time to look them up. I find this statement (definition) highly confusing and it also does not give another reason why this is the direction it uses. Please forgive me for not going into more detail here, I really don't want to add any more to the confusion.
Conclusion
Don't add to the confusion. Use
$$\huge\ominus \overset{\mathbf{p}}{\longrightarrow}\oplus$$
as your definition from now on. Popular use does not make it right. Help using it consistently in the correct way and flush out the historic remnants that are still being taught. However, keep Atkins' warning in mind when you read books, paper, etc.. In the literature you will find both versions.
If anyone wants to argue with you about this, give the following derivation. The dipole moment operator $\mathbf{P}$ is a vector operator that is the sum of the position vectors $\mathbf{r}$ of all $N$ charged particles weighted with their charge $q$.[goldbook]
$$\mathbf{P} = \sum_i^N q_i \mathbf{r}_i$$
For a molecule (neutral by definition)
we find
$$\begin{align}
0& =\sum_i^N q_i &\Leftrightarrow&&
0&=\sum_{i_+}^{N_+}q_{i_+} + \sum_{i_-}^{N_-}q_{i_-} &\Leftrightarrow&&
\sum_{i_+}^{N_+}q_{i_+} &= - \sum_{i_-}^{N_-}q_{i_-} &.
\end{align}$$
This can be transformed into
$$\sum_{i_+}^{N_+}|q_{i_+}|\mathrm{e} =
- \sum_{i_-}^{N_-}|q_{i_-}|(\mathrm{-e}).$$
Therefore we can write
$$\begin{align}
\mathbf{P} &= \sum_{i_+}^{N_+} q_{i_+} \mathbf{r}_{i_+}
+ \sum_{i_-}^{N_-} q_{i_-} \mathbf{r}_{i_-}
&\Leftrightarrow&&
\mathbf{P} &= \mathrm{e}\left(
\sum_{i_+}^{N_+} |q_{i_+}| \mathbf{r}_{i_+}
- \sum_{i_-}^{N_-} |q_{i_-}| \mathbf{r}_{i_-}
\right)&.
\end{align}$$
In the parenthesis the first term is a linear combination of vectors for all positive charges with the resulting vector $\mathbf{r}_+$ and the second term is a linear combination for all negative charges with the resulting vector $\mathbf{r}_-$. The dipole operator is therefore equivalent to
$$\begin{align}
\mathbf{P} &= \mathrm{e}\left(\mathbf{r}_{i_+} - \mathbf{r}_{i_-}\right),
\end{align}$$
which is a linear combination of vectors that points from negative to positive.