Firstly, I would replace the word "feasible" with "favorable". I'm also going to replace $E_{cell}$ with $E_{OCV}$, where the open circuit voltage (OCV) is the potential of the cell without any applied electric potential. So you're right in that if the $E_{OCV}$ is positive, the net reaction is thermodynamically favorable: it will occur spontaneously if allowed to and will produce energy that can be used by the load connected to the battery (i.e. the cell can do work).
You're correct in saying that the reverse reaction is thermodynamically unfavorable, but that doesn't mean the reaction won't happen, just that you need to do work on the battery in order for it to happen. In most cases, this means applying a bias potential ($E_{cell}$) that is more positive than the $E_{OCV}$. When that happens, the current flowing across this bias potential takes energy into the cell and recharges it.
However, not every reaction will proceed in this way. In many cases, if a high potential is applied, another reaction will occur. If you have water in your battery and apply a high potential, you may electrolyze water, splitting into elemental hydrogen and oxygen. This will then reduce the usable capacity and/or cause rupture of the battery.
Lastly, all of this ignores the role of kinetics. Just because a reaction is thermodynamically favorable doesn't mean that it happens at a rate that useful. If you have a cell with $E_{OCV} = 3.60V$, applying $3.601V$ will begin to charge the cell but at such a slow rate than you'll never notice. You generally need to apply at least a few tens of millivolts of over-potential to a lithium ion battery in order to get any appreciable current. So in the case on non-rechargeable (chemically non-reversible) batteries, the over potential needed to reverse the reaction will be big enough that you will instead hit the potential for some other, undesirable reaction.
