I mean won't the common ion effect operate?
The common ion effect always applies for pure substances. With the added sodium chloride, you have changed the effective solvent and changed the ions.
Or may be this is due to some kind of complex formation as NaCl solution is highly concentrated?
Exactly, before your ionic equation was:
$$\ce{Ag+ (aq) + Cl- (aq) <=> AgCl (s)}$$
but with brine, the ionic equation is:
$$\ce{AgCl (s) + Cl- (aq) <=> [AgCl2]- (aq)}$$
Which has a higher solubility constant, and where as before more chloride made the $\ce{AgCl}$ less soluble, if you note that you have switched regimes and $\ce{Cl-}$ has switched from precipitating a silver ion to dissolving a silver chloride molecule, which means now with increased chloride concentration, silver chloride becomes more soluble.