When an electron is added to cesium, why is energy released?
All elements have a positive (first) electron affinity, except noble gases, for which it has not been measured conclusively, but might be either slightly positive or slightly negative.
You can more clearly see the reason why electron affinity is positive if you think of the opposite process: detaching an electron from the negative ion X– to form neutral atom X. Because the anion X– is stable, it takes energy to take away the electron.
I guess Isaac’s main problem arises from the usual chemical experience dealing about ions in water. Then of course Cs- is not a real reaction product. Electron affinities deal about reactions of atoms and electrons in vacuum! The cesium atom "well" to plunge in is not very deep, but better than nothing and there is no better alternative for the electron in a vacuum. The true reason is the atomic orbital energy for an additional electron on cesium, not easy to calculate.
Atoms want to become more stable, which is normally characterised by energy release as they achieve a more stable, lower energy. If cesium accepting that electron makes it more stable, then energy is released. As F'x mentioned in his answer though, not all electron affinities are exothermic. For instance, Beryllium's first electron affinity is positive, i.e. it is an endothermic process, because Beryllium's outermost occupied shell is an $s^2$ orbital. Hence, the additional electron would have to be added to the $p^6$ orbital instead, which is further away from the nucleus and therefore does not experience as much electrostatic attraction. Furthermore, the $s^1$ and $s^2$ are also "shielding" the nucleus, such that the outermost electron experiences even less electrostatic attraction.
For understanding why most atoms prefer are at a lower energy when more stable, I picture a person that has consumed a significant amount of caffiene and is hyperactive since he has a lot of energy. He becomes less hyperactive or "more stable" as he loses that energy. Remember, this is only an analogy and analogies are not perfect!
I would also like to point out that I, and I assume your question as well, am referring to to first electron affinities.