Simple prefixes are arranged alphabetically disregarding any multiplicative prefixes. The multiplicative prefixes are inserted later and do not alter the alphabetical order.
For example, ‘1,2-dibromo-’ is considered to begin with ‘b’.
However, the name of a compound substituent is considered to begin with the first letter of its complete name.
For example, ‘1,2-dibromobutyl-’ is considered to begin with ‘d’.
On this matter, the current version of Nomenclature of Organic Chemistry – IUPAC Recommendations and Preferred Names 2013 (Blue Book) reads as follows:
P-14.5 ALPHANUMERICAL ORDER
Alphanumerical order has been commonly called ‘alphabetical order’. As these ordering principles do involve ordering both letters and numbers, in a strict sense, it is best called ‘alphanumerical order’ in order to convey the message that both letters and numbers are involved
Alphanumerical order is used to establish the order of citation of detachable substituent prefixes (not the detachable saturation prefixes, hydro and dehydro), and the numbering of a chain, ring, or ring system when a choice is possible.
P-14.5.1 Simple prefixes (i.e., those describing atoms and unsubstituted substituents) are arranged alphabetically; multiplicative prefixes, if necessary, are then inserted and do not alter the alphabetical order already established.
P-14.5.2 The name of a prefix for a substituent is considered to begin with the first letter of its complete name.
Therefore, the correct alphanumerical order corresponds to the name 7-(1,2-dimethylpentyl)-5-ethyltridecane (not 5-ethyl-7-(1,2-dimethylpentyl)tridecane) since the compound substituent name ‘dimethylpentyl’ starts with ‘d’, and ‘d’ is earlier alphabetically than ‘e’.
(Nevertheless, the preferred IUPAC name is 5-ethyl-7-(3-methylhexan-2-yl)tridecane.)
In your second example, the correct alphanumerical order corresponds to the name 6-(1,1-dimethylbutyl)-5,5-dimethyl-6-pentyltridecane (not 5,5-dimethyl-6-(1,1-dimethylbutyl)-6-pentyltridecane) since the compound substituent name ‘dimethylbutyl’ starts with ‘d’, and ‘d’ is earlier alphabetically than ‘m’.
(Nevertheless, the preferred IUPAC name is 5,5-dimethyl-6-(2-methylpentan-2-yl)-6-pentyltridecane.)