Images are from Wikipedia.
The first obvious thing that you can see is that N2O4 is planar but N2H4 is not. And the primary reason starts here.
N2O4 can be viewed as two nitrogen dioxide molecules joined together. If you draw a Lewis structure of NO2, you will recognize two things. Firstly, it has one unpaired electron. So imagine when two NO2 molecules come near together, they will share their two unpaired electrons (like how radicals react that you might be familiar with in organic chemistry) to form a single sigma bond. This is the very bond you are asking about.
And secondly, it is sp2 hybridized at the nitrogen atom. Thus, when you join the two NO2 molecules together, taking the relative orientation of the sp2 orbitals and how they should overlap to form a sigma bond in mind, you can see why N2O4 is planar. Because the N=O bonds (I write the "=" symbol for double bond for simplicity, it should be about 1.5 order due to resonance) heavily repulse each other when they are in the planar position, it weakens the N-N bond. Indeed this bond is so weak that it is homolytically dissociated at near below room temperature or above, creating basis for the popular equilibria between NO2 and N2O4.
The situation is even worse for N2O3. N2O3 can be viewed as formed from a NO (itself has an unpaired electron and a sp2 nitrogen) and a NO2 molecule, with the exact mechanism as above. And guess what? N2O3 has a lone electron pair on the nitrogen (in place of the missing oxygen compared to N2O4). Lone pairs repulse much, much more heavily than the bonding pairs do, and this further weakens the N2O3 N-N bond. This is illustrated by the easily disproportionation of N2O3 into NO and NO2.
Then why N2H4's bond is stronger? Because the nitrogen atoms are sp3 hybridized in this case (note that oxygen is divalent but hydrogen is only monovalent), thus they form a tetrahedral electronic structure. Thus they can comfortably adopt a skew conformation to minimize the repulsion. Either N2O3 or N2O4 could achieve this because the sp2 hybridization confines them to planar geometry.
Using Pauling's formula with the 1.14 A triple bond in N2 as a reference:
D(n) = D(m) - 0.6log(n/m)
With D(n, m) is the length of the bond with order n, m respectively.
The N-N bond order in N2O4, N2O3 and N2H4 can be estimated to be 0.44, 0.19 and 0.85 respectively.