The diethylamide "unit" dramatically potentiates lysergic acid. LS-Dimethylamide is significantly less potent, as are other/different amide or amine units. The differences in the amide or amine units also result in different experiences (spiritual, visual, auditory, etc.). Of course, it's different for everyone, but a loose analogy could be to the differences between beer, wine, bourbon, and moonshine.
Now, as the DrAzulene noted, there is an indole associated with many hallucinogens. Indoles are pretty important, but mescaline and ecstacy (for example) don't have an indole group, whereas serotonin, tryptophan, wellbutrin, and many other indole-containing molecules are not hallucinogenic. But mescaline and ecstacy (again, for example) have amine and methylamine groups, respectively. Other derivatives of these (and other) psychedelic drugs (designer drugs) may use diethylamide, ethylmethylamine, or other amine groups.
So, in my humble estimation, the xyz-amine group seems to act like a key to interaction with receptors, to allow communication or attachment, and by default, delivery of the affecting "body" of the molecule.
The combination of interactions with specific and a diverse array of neural receptors is what imparts the profile of hallucinatory characteristics. Another loose analogy would be like how sugar, salt, and MSG all interact with different taste receptors. Aspartame, for example, interacts strongly with the sweet-receptors. If one molecule could interact strongly with these three flavor receptors, it would have a very pronounced impact on your tongue.
LSD interacts strongly with at least 21 different neural receptors, and chief among them, all (five known) dopamine receptors as well as many (at least 14) serotonin receptors, including the classical (3 known) 5-HT2 receptors: 5-HT2a, 5-HT2b, and 5-HT2c. The 5-HT2 receptor familiy has the most pronouced impact on psychedelia - as well as depression and schizophrenia.
Phenylalkylamines (that don't contain indoles), like ecstacy and mescaline, primarily interact with only the 5-HT2b receptor and none of the dopamine receptors (interesting because phenylalkylamines tend to resemble dopamine). They interact with serotonin receptors and other miscellaneous neural receptors, instead. But generally, the list of receptors is shorter (less than 10), and the primary way that they work (by blocking serotonin reuptake so that the brain releases more serotonin) are mechanisms that cause release of the natural chemicals from the brain to produce euphoria, so they are (pound for pound) less euphoric and less psychedellic. The way that brain produces feeling of euphoria in response to phenylalkyamines is due to the flood of serotonin (produced by your own brain), resulting from phenylalkyamines' blocking the reuptake of serotonin. The increased levels of serotonin cause a release of dopamine (from your own brain). Dopamine is generally the source of euphoria and the surge of dopamine resulting from the surge of serotonin is essentiall the reward stimulation.
Again, LSD intacts directly with all five kinds of dopamine receptors, all three psychedellic receptors, and many more... and there's a very good probability that there are receptors still undiscovered.