The color is in fact caused by the elemental iodine. At the 'yellow' stage of reaction the concentration of iodide is low, so the triodide anion (yellow at low concentration and red-brown at high) does not form, hence there is no formation of the blue triodide-amylose complex.
The OP wonders how the iodine concetration can be high enough to give the solution its yellow coloration, given the low solubility of elemental iodine in water?
First let me make a remark that crystalline iodine is very sluggish to dissolve in water to reach its maximal concentration. Even when making potassium iodide - iodine solutions - it takes a lot of stirring, anyone who worked in an analytical lab knows that.
But this is not the answer to the question. You have to consider that the iodine in solution does not come from a solid phase going to the liquid phase - it is generated in a homogenous chemical reaction. Therefore, it can reach concentrations far beyond the maximal solubility, forming a metastable supersaturated solution. This stage of the oscillation does not last long enough to allow nucleation and precipitation of the elemental iodine. The subsequent increase of iodide concentration causes formation of triodide, which along with starch gives the blue color and so on.