No, it won’t burn immediately. As your electrodes are generally separated by some distance, you should be able to isolate the produced oxygen and hydrogen gas by catching the respective bubble streams under a submerged test tube or the like. Alone, neither oxygen nor hydrogen will burn, however combining the oxygen and hydrogen gas will produce an explosive mixture. Mixing the hydrogen with air will also produce an explosive mixture as the atmosphere is about 22 % oxygen.
To get the hydrogen to burn, you need to supply the necessary activation energy, which can be supplied by a spark or flame. After this, the reaction will be self-sustaining due to its exothermicity. There are in fact entirely self-contained apparatus which can be purchased from scientific teaching shops that can electrolytically split water and then combust the hydrogen and oxygen using a piezoelectric ignition system, the ensuing tiny explosion causing water to spurt out of a small hole. Wikipedia mentions that the autoignition temperature of hydrogen in air is 500 degrees Celsius.
On a small scale, this reaction is quite safe, however you should always take safety precautions such as wearing protective eyewear and having fire-suppression equipment at hand.
Here’s an illustration of a reasonable water-splitting setup. Remember that your water should contain an electrolyte such as dissolved salt. Keep in mind that if your potential difference is high enough you will start to produce small quantities of chlorine gas and aqueous sodium hydroxide, both of which are hazardous in high concentrations.
I should draw attention to the fact that the $\ce{H2}$ headspace is about twice the volume of the $\ce{O2}$ headspace. This comes down to the stoichiometry of the reaction, and also means that the $\ce{H2}$ volume will increase at twice the rate of $\ce{O2}$. The wires I’ve illustrated connect to the terminals of a battery or lab PSU.
Addendum: Here’s a nice guide on how to construct and safely operate a water-splitting/combusting apparatus using household equipment, aimed at secondary school students.