Working out the structure of a compound (organic, organometallic or organomain group) is not a simple matter. It is something which can be very intellectually taxing. I have done it many times in my life and I can tell you that for some thing you can do it in a matter of minutes while for other things it can take weeks, months or even years)
The first problem is that you need to know that your substance is sufficiently pure that the peaks in spectra are due to your compound of interest rather than some impurity.
For an organic compound if it is a pure substance which you have plenty of, my advice is normally.
Get an infrared spectrum of the substance so that you can identify some of the functional groups. Look for multiple bonds, you can look for evidence of ketones, amides, esters, carboxylic acids, nitriles, metal carbonyls, benzene rings and alkenes. In my experience it is sometimes hard to see the C C stretch for acetylenes but you might get lucky. This will give you a list of mental jigsaw bits.
Get a proton NMR of the substance, this will either be a great help or close to useless. If you are lucky you will be able to identify parts of the structure. For example if you have a phenyl group, a methyl close to TMS and two multiplets which appear to be all coupled to each other. Then using the 3J couplings you could tell if you have a propyl group.
Alternatively if you had a isobutyl group then the integration heights would be different and the coupling pattern different, for the isobutyl you would have doublets for the methyls instead of the triplets you see in the propyl.
You should also look for signs of symmetry, for example if you have benzopheone (diphenyl ketone) then you will have seen in the IR a ketone (which is slightly lower than 1700 as a result of conjugation) and only one carbon / hydrogen group. As a result of symmetry both phenyls are magnetically identical.
Equally if you have 1,3,5-trichlorobenzene you will have a single singlet in the aromatic part of the proton spectrum. Compare this with the multiplets you would have for 1,2,4-trichlorobenzene.
If you see a very large H H coupling then it is possible you have a ring or some other thing where the conformation is locked. This would be a substance with a 2J coupling. Note this it could be a great clue. For example in the structure of some penicillin derivatives you can see 2J couplings where two different hydrogen on the same carbon have different magnetic environments.
Look in the proton for anything which is not usual, for example cyclopropanes have a very distinctive shift which is close to zero ppm or even at - ppm.
If the proton was little use or you can not understand it then my advice is to try some more things.
If the proton NMR is very congested, then I would suggest running it again in a different solvent. Commonly people use CDCl3 for NMR. I always say that if the spectrum in chloroform is impossible to understand as a result of being congested then I run a sample dissolved in D6 benzene. The change of solvent will alter the solvation and the proton environments oftein change in their chemical shift, this can make some areas of the spectrum more easy to understand. In this way you may be able to get a clearer look at it and as a result you might be able to understand the structure better.
Another trick which makes proton NMR more simple is to use double irradation, this can allow you to make the spectrum more simple by removing one coupling effect at a time. This will make the spectrum more simple. If it solves you problem then be thankful otherwise note down any new insights you have into the substance and move on.
I would also say that getting a carbon-13 NMR can be a great help, this will help you understand the substance more. You will learn quickly how many unique carbon centres there are in the substance.
You should also consider the question of will a 2D NMR experiment help you, for example you can use a COSY or a CH correlation. For NOSEY you need to exclude all oxygen from the NMR sample. For this you will need to use deoxygenated chloroform and work on a vacuum line and seal up the tube with no air getting inside. This will take some skill and time to do.
If you are stuck you should consider getting a mass spectrum of the substance, this can give you an idea of how large the thing is. Be careful with mass spectra. It is sometimes possible to get a nice spectrum off a tiny impurity which is present in the sample while the main compound does not give a strong nice spectrum.
The mass spectrum will give you an idea of the formula, you can get the formula weight. One thing which is worth doing is to look at the isotope splitting pattern of the molecular ion. For example bromine is present in the form of two isotopes with about equal abundance so for a monobromo compound you should expect a doublet. If you can see such splitting in the molecular ion then it is a good sign you have a heavy atom in the substance. Keep in mind iodine only has a single stable isotope.
Next trick I have is to use a sodium fusion test, now this test has been banned in a lot of universities as undergrads has made loud bangs or started fires with it. But when used with care it is perfectly safe. I like the test as a method of detecting nitrogen, sulfur or halides in an organic substance. Heat a SMALL pellet of sodium until it melts. Then add some of your thing and carefully reheat. There is a great debate on how best to do this, and it takes some practise. Heat the mixture up until it glows and plunge it into cold water. It may go bang, pop, fizz etc when you do this. Then boil the water mixed with broken glass and filter it.
Test the filtrate for halide with silver nitrate, if it goes cloudy then add ammonia. If the solid redissolves then it is likely to be a chloride while if it stays cloudy then it is likely to be either a bromide or iodide.
Test the filtrate by adding ferrous sulfate, boil it and then add acid. If it goes blue on adding the acid then you have made Prussian blue and you had some nitrogen in the organic substance.
If you can detect either thiocyanate or sulfide in the water then you had sulfur in the organic compound.
Add these new facts to the list of facts you have and see if you can work out a structure which makes sense for the organic molecule. Sometimes you can work out several possible structures based on NMR and IR. For example toluene and methyl benzoate will look very similar in the proton NMR.
Next take the possible structures and see if you can distinguish between them by molecular weight and which heteroatoms you have found in them. For example diphenyl sulfide and 4,4'-dihydroxybiphenyl would have the same molecular weight. But one will have sulfur in while the other will not. Also the fine detail in the mass spectrum on the molecular ion will help sort these two out.
Now if you have stared at it for a while and you still can not solve it, then it might be time to get a UV / vis spectrum. This is good when you have a conjugated system present in the molecule. It can be used to give you an idea of the size of a pi system.
If you see any strange couplings in the proton NMR spectrum where you can see one thing which couples to something which can not be seen then you might have a phosphorus atom present. Some years ago a student showed me a strange NMR spectrum with a doublet they could not understand. I spotted that the sample contained some HMPA, the doublet was in the right place for HMPA.
If you have phosphorus or another heteroatoms which couples such as selenium or platinum then you can try getting more NMR. For example consider the phosphorus spectrum you would have if you had a Pt binding to a triphenyl phosphine and to a dppe (1,2-bis diphenylphosphino ethane) and a chloride. You would expect to have three different phosphorus environments. Also three different coupling constants.
If you have a Pt with two PPh3 ligands and a dppe, then you would have two P environments and a very complex splitting pattern. Keep in mind also that the Pt-195 will couple with the P nuclei giving sidebands on the P signals.
The key thing to do is to keep gathering more data and trying to look at the clues from the different bits of the data to see if it will fit together to give a structure. If you have multiple structures then you can attempt to use physical properties to distingish between them. If the melting points are very different then you can use a copy of the rubber book, a melting point rig and your sample to work out which it is.
In the old days the gold standard was to take an authentic sample of the thing and mix it with your suspected sample of the same thing. Then to go and do the melting point at the same time as pure samples of the authentic sample and your sample. This is the idea of a mixed melting point. If all three samples melt sharply at the same temperature then you have identified your thing.
Now if you do not have a winner, then there are several things you can do. You can try to form derivatives or degrade your compound in some predictable way. This might make new compounds which can be identified with greater ease. This will give you clues to what you have. For example treatment with ozone can cut up a molecule by cleaving alkenes, this will make it more easy to spot some fragments.
Now if you can grow a crystal of your thing, then my advice is to try for single crystal diffraction methods. Just be careful of the rouge crystal problem, even if you get a pretty picture of the atoms you will need to check that the spectroscopic properties of your bulk sample are right.
You can get some nasty shocks, I once handed in crystals of a organic phosphorous sulfur compound to discover to my horror that the crystals were of some sulfur (S8) which was present as an impurity. The S8 is impossible to see with NMR or IR, so I thought that I was handing over a pure sample to the crystallographer. That was embarrassing.
There are other tricks you can try but I think for most things that the rather long post I have given will sort most things out.