I am trying to wrap my head around the flame test, and for the most part, it is making sense. My question is their a way to figure out the colour of a flame of an unknown element given the emission spectrum. In other words, how does the eye combine the mess of an emission spectrum into a single colour? I have been researching for a while now, so I am prepared for some pretty heavy maths. Some links would also be helpful for further reading.
It is not simple -- for example, to many people, red + blue spectral lines appear purple/violet, i.e., shorter wavelength than pure blue! Though you could assign a color name to a pure spectral line, as you state, the appearance of mixed spectral lines becomes much more difficult to describe.
In this color wheel, names are assigned to mixed colors.
An interesting case is didymium glass, used in photography and by glass-workers to block the bright sodium yellow lines. Under incandescent illumination, the glass has a distinctly pink appearance, and under skylight, it's decidedly mauve.
So take description of spectral appearance with a grain of salt... oops, that's definitely yellow!
As I understand your issue, you need to convert spectra into RGB triplets. Quantitative colorimetry theory addressing that issue is quite thoroughly stated in this wikipedia article on CIE 1931 color space
In other words, how does the eye combine the mess of an emission spectrum into a single colour?
Although your query is related to vision and others have addressed it, let me clarify a couple of things that might be causing confusion. First, you are talking about simple flame tests for alkali metals, alkaline earth metals, and a few elements like boron and copper. Bunsen flame spectra are not a forest of emission lines but a very simple spectrum (indeed not a mess at all). You may have been misled by Google images search. Most of them are just cartoons and fake. Bunsen flame is a low-temperature flame. Not many lines are expected.
If you have a pocket spectroscope or even a homemade CD spectroscope, you will see that sodium flame is just one orange-yellow line around 589 nm. Similarly, potassium flame spectrum is just one or two lines. If you are lucky with sensitive eyes, you might the deep red and violet lines.
See this excellent video of the true flame colors Flame emission spectra. The video shows the simple spectra. You will also note that the yellow sodium line is usually present in most spectra...Na is everywhere.
You can try making a CD spectroscope to try it yourself. Internet is full of such easy constructions, but the spectra on Google Images is disappointly cartoonish (or rather software generated) from at a first glance.
Another misconception to be noticed is that many people think that the Bunsen flame colors are always due to atomic emission. This point is not valid. Ca, Ba, Sr, Cu, B, flame colors are due to small molecular compounds of those elements, not their atoms. Only the alkali metal emission is atomic emission, and the rest are molecular emission in a Bunsen burner.
For real analytical work, Bunsen flame color is useless. We have to use a high-temperature from a spark or electric discharge or a plasma around 5000 K to see true atomic emission of most of the elements in the periodic table. Then, indeed, the emission spectrum is a mess, mainly in the ultraviolet, and you certainly need a device to separate wavelengths. The concept of color then vanishes because we cannot see ultraviolet.