A common misconception is that the more acidic component will always protonate the less acidic componend and displace it from its salts. That misconception would be exemplified by equation $(1)$
$$\ce{HBr + KH2PO4 -> KBr + H3PO4}\tag{1}$$
This is incorrect. There is a constant acid-base equilibrium and even bromide ions will be protonated to a non-neglegible extent. The principal difference between $\ce{HBr}$ and $\ce{H3PO4}$ is that the former is a gas at room temperature and standard pressure. Thus, in an open vessel, any $\ce{HBr}$ in solution is again automatically in equilibrium with $\ce{HBr}$ in the gas phase. Thus, the equilibrium should better be written in form $(2)$:
$$\begin{multline}\ce{KBr (aq) + H3PO4 (aq) <<=>\\ <<=> HBr (aq) + KH2PO4 (aq) <=> \\ <=> HBr (g) ^ + KH2PO4}\tag{2}\end{multline}$$
As you can see, the gaseous $\ce{HBr}$ can diffuse away. This reduces the partial pressure of $\ce{HBr}$ above the solution, which affects the right-hand side equilibrium by Le Chatelier’s principle, reducing the concentration of $\ce{HBr}$ in solution, which in turn affects the left-hand side equilibrium, reducing the concentration of $\ce{KBr}$. Thus, if left to equilibrate in a well-vented area, large portions of $\ce{HBr}$ will diffuse away leaving a solution of potassium phosphate or potassium hydrogen phosphate.
The principle alluded to in the first sentence is thus correct, if rather than using acid strength it is quoted as:
The less volatile acid will displace the more volatile acid from its salts.
It is worth noting that an acid-base equilibrium $(3)$ also is observeable in your first reaction. However, the redox reaction is faster than the removal of hydrogen bromide. It is also irreversible, hence those are the final products obtained.
$$\ce{KBr + H2SO4 <=> HBr + KHSO4}\tag{3}$$