[OP] But how can, say, A becoming B reverse in entropic favourability purely based off the amounts of A and B?
The entropy of reaction is dependent on the reaction coefficient Q (so on the activities or concentrations of the reactants and products, $\frac{[\ce{B}]}{[\ce{A}]}$ for the example given by the OP).
As the reaction proceeds in the forward direction, the entropy of reaction decreases (and the Gibbs energy of reaction increases) until the reaction reaches equilibrium. No matter what the standard entropy of reaction $\Delta_r S^\circ$, the actual entropy of reaction $\Delta_r S$ will be positive in the absence of products (at the “beginning” of the reaction) and negative when a reactant runs out (if the reaction were to go to completion).
$$\Delta_r S = \Delta_r S^\circ - R \ln{Q}$$
If you plug this into the defining equation for the Gibbs energy, you will find that the Gibbs energy of reaction is also dependent on Q.
$$\Delta_r G = \Delta_r G^\circ + R T \ln{Q}$$
This makes sense because otherwise, we could not explain how a reaction reaches equilibrium as concentrations change.
Finally, for those who have learned about the relationship of $Q$ and $K$ to determine in which direction equilibrium lies, you can express $\Delta_r G^\circ$ as $ - R T \ln{K}$ to get the following expression for the Gibbs energy of reaction:
$$\Delta_r G = R T \ln{\frac{Q}{K}}$$
This is a direct way of relating the ratio of $Q$ and $K$ to the Gibbs energy of reaction. The expression yields zero when $Q$ is equal to $K$, and it shows the concentration-dependence (because $Q$ is part of the expression).