When it comes to acidity, stability of the counter-ion is one of the main considerations. The nitro-group acts to stabilize the negative charge of the resulting nitrobenzoate by pulling the negative charge onto itself.
In the case of 2-nitrobenzoic acid, it is much nearer and therefore able to more effectively 'remove' the negative charge from the carboxylate group. In the case of 4-nitrobenzoic acid, it's still able to do so but less effectively as it is further away.
The hydrogen bonding effect that you describe does exist in 2-nitrobenzoic acid but this does not act to make it less acidic. While one might reason that the hydrogen bonding "holds the proton in place" and therefore makes it harder to leave, the presence of the hydrogen also means that both the nitro and the carboxylate group are highly unlikely to be simultaneously coplanar with the benzene ring (it's not obvious from the image because there's no bond between O and H). The energetics in this case are likely to favor the release of H+ in exchange for a coplanar configuration.