When a battery is performing work on a charge and moves it to the higher potential, why doesn't the charge move through the battery and return to the low potential? Why does it go through the circuit?
Batteries are constructed with an internal barrier between the two electrolytes that does not conduct electricity (i.e., electrons can't pass), but that does allow ions to pass through. See the figure below (taken from Wikimedia, open source), of a simple alkaline dry cell (e.g., an AA battery), in particular the part labeled "ION CONDUCTING SEPARATOR."
The reason batteries are made this way is because in order for the anode electrolyte and the cathode electrolyte to come to thermochemical equilibrium, there has to be a transfer of both electrons and ions. By putting the separator in there that only conducts ions, you block equilibration of electrons except in the case where you have it connected to an external circuit. If you conduct an imaginary experiment where you * SNAP * your fingers and make that ion conducting separator disappear, the electrons would indeed surge from one electrolyte to the other as you describe, as fast as mass transfer would allow. (You would then having to imagine the battery either exploding or melting, because this would make a LOT of heat.)
The charge doesn't "move back through the battery" because in order to do that, it would have to pass directly through the chemical reactions that put it there in the first place. It moves through the external circuit, because that's the only path available to it.
To take it to the next level of detail, the half-reactions that occur at the two plates inside the battery are actually occurring in both directions all of the time. When the battery is open-circuit, both reactions are occurring at the same rate. The forward reactions are slowed down by the electric field created by the separation of charge, and the reverse reactions are accelerated by it, and the battery maintains equilibrium.
When an external path for current is created across the battery terminals, some of the charge flows through it reducing the voltage and the E-field slightly, and this allows the forward reactions within the battery to occur at a higher rate than the reverse reactions, and this is what eventually dicharges the battery as the reactants are consumed.
The answer is: it will! When no salt bridge or similar conducting element is present between the two half cells, the reaction will quickly run to equilibrium. The state in which the potential force of the difference in electron affinity between the two electrodes more or less equals the Coulomb force of the opposite charges that accumulate in them. However, if you make use of a salt bridge this charge difference accumulation can be removed, since an electrolyte, which sort of completes the circuit. Usually salt bridges are constructed in a manner that prevents intermixing of the ions in both half cells, so that there is some certainty about what redox reaction is actually going on, and how. An example of a common salt bridge used in high school chemistry experiments is a filter paper soaked in a solution of potassium chloride.