Is the lone electron pair of the nitrogen atom part of the conjugated π system in the α,β-unsaturated amide pictured below?
Is there a general rule for choosing when to include a nitrogen atom in a Hückel matrix?
Setting aside the arguments over umlauts, yes you can add the nitrogen atom to the Hückel matrix; in fact, theoretically you should do so and allow the matrix eigenfunctions and eigenvalues to decide whether the nitrogen atom in this case is really conjugated. The carbonyl oxygen can also be coupled in.
But there is a catch. Since the nitrogen or oxygen atom has different atomic orbital energy levels from the carbon atoms, and the orbital overlap with other atoms will be different, you cannot just put in the same numbers that you would with a carbon atom there. Various estimates are available for these heteroatom parameters, for instance this one given by Texas A&M University which includes common heteroatoms for the diagonal elements and their bonds with carbon for the off-diagonal elements.
Once you have decided upon the matrix elements you can solve the matrix for its eigenfunctions (molecular orbitals) and eigenvalues (energy levels) the same way you would with an all-carbon system, but the equations will probably be more complex and require numerical methods to solve efficiently.
RAlt
"
u
for letter ü orRAlt
*
p
for letter π (cc @ToddMinehardt). $\endgroup$CTRL+SHIFT+U<unicode>ENTER
on Linux. $\endgroup$