When two oppositely charged particles are brought closer together under a Coulombic potential, the *potential* energy of the pair *decreases*. However, conservation of energy requires that this potential energy be converted into some other form (total energy is conserved). In a simple classical description we say that it is converted into kinetic energy, ie $$-\Delta E_{Coulomb} = \Delta E_{kin}$$
In an adiabatic setting that increase in kinetic energy would amount to an increase in temperature. In an isothermal setting it results in an exothermic process as the thermally excited system relaxes to the temperature of the surroundings. 

Consider first a thought experiment. If you take a proton and an electron with no kinetic energy and at infinite separation and allow them to associate into a hydrogen atom, the resulting (unstable) atom has an energy of 13.6 eV or 1 313 kJ/mol, equivalent to a temperature of 105 300 K for an ideal gas of such "atoms" (that's an absurdly high T, but then the temperature of the sun's [core][1] is presumed to be ~15.7 million K). This description mixes classical and QM ideas (the temperature of a gas, interconversion of potential and kinetic energy, and the ionization energy of hydrogen) as a thought experiment. The point is that for a nucleus to bind an electron into a stable electronic arrangement, the resulting atom has to get rid of some energy, which it can pass on to the surroundings as heat.

Consider a more practical situation. The [photosphere][2] of the sun has a much lower temperature, ~6000 K, and its spectrum likens that of an ideal blackbody. There hydrogen atoms associate with electrons to form hydride ions, with a drop in electronic energy: 
$$\ce{H^{\cdot}  + e- -> H- +h\nu}, ~~\ \ EA =0.\pu{754 eV} $$
(note I use the *opposite* sign convention for EA as in the OP, here positive EA is lower in energy) The energy decrease involves a transition between electronic quantum states that is accompanied by release of excess energy by the ion as light. This is very nicely explained in the Wikipedia. Note that the [hydride ion][3] has a closed shell. The formation of a closed shell is associated with a positive electron affinity EA, that is, hydrogen can attract electrons to form a stable closed shell anion. 

A non-mathematical explanation for the origin of the positive EA is that QM dictates the most stable arrangements of electrons about nuclei, and only particular arrangements are stable. It is largely a geometric problem involving a balance between electron-electron repulsions, electron-nuclear attraction, the wave nature of matter evident particularly at small scales, and odd effects such as Pauli exclusion (the impossibility of two electrons having identical properties). 

Considering now your list in more detail:

1. It doesn't always do so. Sometimes it's impossible. Not all combinations are stable. Only when the electron affinity is consistent with electron attraction is a stable combination possible. This is largely addressed above. Assuming it is possible to form a stable atom or ion, the combined particles have to shed energy to the surroundings, otherwise they risk falling apart again. When it gives off energy the combination relaxes from an excited state to a lower energy state. It might only shed some energy and remain in a highly excited (reactive) state, however. 


2. This is addressed in another answer to your post. The argument is somewhat circular. Higher energy is less stable. Basically you are creating an unstable (excited) system when you attempt to remove an electron from a stable atom, so you have to add energy to do so. 

3. This is addressed by the hand-wavy QM argument at the end of my dissertation. The combination of charge attraction and repulsion with the wave nature of subatomic particles and the Pauli exclusion principle lead to particular particle combinations and geometric arrangements being stable. This is the subject of a more rigorous QM course.


  [1]: https://en.wikipedia.org/wiki/Sun#Core
  [2]: https://en.wikipedia.org/wiki/Sun#Photosphere
  [3]: https://en.wikipedia.org/wiki/Hydrogen_anion