How can one determine whether an organic compound is planar or not?
My idea was that for a compound to be planar the hybridization of all the atoms must be the same. But I came across the compound cyclooctatetraene, which has the same hybridization of all its carbon atoms but is not planar. How do I explain above exception?
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
This is rather a broad question and there are not may ways to approach this a priori. However, there are a few guidelines that can help you make a call:
π bonds must include up to six atoms in one plane for the π overlap to be maximum. Thus, you can conclude that $\ce{C2H4}$ must be planar.
Three atoms denote a plane and thus must always be planar. From this and the previous bullet point, you can conclude that $\ce{C2H2O}$ (oxirine) must be planar.
If you have a non-cyclic contiguous π system — that usually means alternating single and double bonds but can include an atom carrying a lone pair in-between — the system has a desire to be planar to create a better overlap and thus reduction in energy. This refers to all systems which do not form a contiguous π circle, whether the circular part is on its own or not. Thus, conclude that acrolein $\ce{C3H4O}$ is planar.
If you have a cyclic π system that consists of a single circular arrangement, refer to the Hückel rules: cyclic systems with $4n + 2$ π electrons are aromatic and thus often planar. Cyclic systems with $4n$ π electrons would be antiaromatic. With the exception of butadiene (see the first and second bullet points) these distort to a non-planar state so that the double bonds may be regarded as isolated rather than conjugated. Of course in larger systems ($\ce{C12H12}$ and above), it would be entirely conceivable for parts of the π system to remain planar but there would always be at least two breaking points.
If you have a polycyclic π system or cyclic with extensions, these simple rules don’t work anymore. In most cases, they will be planar and display some kind of semi-charge separation to keep with Hückel’s rules.
Finally, there are cases which stericly just cannot be planar. The helicenes jump to mind: If they were planar, the hydrogens on one end of the helix would be inside the other end. Thus, these systems must distort to exist. Similar points can be made for 2,6-disubstituted styrenes, bisphenyl and its derivatives and more.
Honorary mention: Any molecule that contains a tetrasubstituted atom or a trisubstituted atom that is not in the thirteenth or fourteenth group or nitrogen cannot be entirely planar. Thus, dimethyl sulfoxide is not planar, nor is $\ce{PH3}$ or methane. Nitrogen compounds such as ammonia are a difficult call: typically, they would also be trigonal-pyramidal but they can undergo nitrogen inversion which comes with a planar transition.
