I routinely use a positive pressure of argon with a balloon in an otherwise sealed reaction setup where all joints are (greased) ground glass joints, potentially secured with clips. It is rather rare for them to give in to the balloon pressure but at least once the weakest link popped open (a single stopper that was not secured properly). Depending on your ...
If I understand the set up and purpose, just use a rubber stopper with a needle (any) through it.
I do expect that nothing you put in hydrogen peroxide shall react violently, as for you are doing compatibility tests.
Ground-glass joints are notorious for seizing at the wrong time, particularly if reactants are alkaline.
If pressure might change, I'd add some relief mechanism, such as a water-trap or rupture disk. Without knowing your setup, something as simple as a balloon might work, if not attacked by reactants.
If I didn't miss any point, you want to make sure that:
-the system is not in direct contact with the environment, in order to keep it clean
-pressure doesn't build up dangerously
-the non-gaseous product is not lost
What about a reflux condenser? Even unstoppered, it makes loss of material very unlikely. If you are concerned about contamination, you can ...