The bromine is also not resonance donating/activating
You can't have it both ways. If the bromine can resonance interact with the intermediate, then it can also resonance interact in the starting bromobenzene.
Let's back up to the starting bromobenzene.
Bromobenzene will likely react, but at a slower rate than plain
benzene, because the bromine is inductively withdrawing
That's correct, but let's compare further. The oxygen in phenol is electron withdrawing (oxygen's electronegativity is even greater than bromine's), yet phenol is a very reactive aromatic compound towards electrophilic aromatic substitution (it will decolorize a solution of bromine, it doesn't even need a Lewis acid catalyst). The difference between bromobenzene and phenol is that the phenolic oxygen can interact strongly via resonance and donate electrons into the aromatic ring, while the bromine in bromobenzene also resonance interacts with the aromatic ring, but much less efficiently (the larger size of bromine's $\ce{4p}$ orbitals makes overlap much less efficient with carbon's 2p orbitals). Although the interaction is much less efficient, there still is some resonance interaction between the bromine substituent and the ring. Look at the analogous resonance structures for bromobenzene that you drew for the intermediate, in the contest between inductive deactivation and weak resonance activation, the ortho and para positions are the least deactivated.

When bromobenzene undergoes electrophilic aromatic substitution, the whole ring is deactivated because (unlike phenol, where the resonance effect overwhelms the inductive effect) the weak resonance effect doesn't offset the strong inductive effect. However, since the bromine substituent does interact with the ortho and para positions by resonance, these two positions are less deactivated that the meta position which cannot interact via resonance with the bromine substituent.