TL;DR
Note that the passive layer forms on the surface, there needn't be any change to lattice constant. Chromium needn't migrate , the Cr present on the surface will form the layer to protect it. The key point is how the layer develops from say a single-atom layer of oxide to the usual/maximum width by migration of electron and oxygen in the oxide layer. The process at molecular and atomic level is more complex than you think.
From wikipeddia article on Passivation
There has been much interest in determining the mechanisms which describe how the thickness of the oxide layer on a material increases with time. Some of the important issues include: the relative volume of the oxide compared to the parent metal, the mechanism by which oxygen diffuses through the metal oxide to the metal - oxide interface and the relative chemical potential for the oxide to form. Boundaries between micro grains, if the oxide layer is crystalline, form an important pathway for oxygen to reach the unoxidized metal below. For this reason, vitreous oxide coatings – which lack grain boundaries – can retard oxidation. The conditions necessary (but not sufficient) for passivation are recorded in Pourbaix diagrams.
Some corrosion inhibitors help the formation of a passivation layer on the surface of the metals to which they are applied. Some compounds, dissolving in solutions (chromates , molybdates) form non-reactive and low solubility films on metal surfaces.
The mechanism
All passivating‐type inhibitors are oxidizing agents. However, not all oxidizing agents are inhibitors. A passivating ‐type inhibitor functions by producing local‐action current which anodically polarizes a metal into the passive potential region and thereby provides the means for obtaining a noble mixed potential. This mechanism is independent of whether passivity is caused primarily by oxide or adsorbed films.
The major factors which determine whether a particular system will exhibit passivity are:
(a) reversible potential of the redox system created by the oxidizing inhibitor;
(b) electrochemical parameters for reduction of the inhibitor on the metal surface (exchange current, Tafel slope, and limiting diffusion current), and
(c) critical anodic current and Flade potential(The potential of a passive metal immediately preceding a final step fall from the passive to the active region.) of the metal. If oxygen is present in the system, some chemicals may function by changing its reduction kinetics.
It is further shown that the amount of inhibitor found associated with the surface is not necessarily related to adsorption. Data are presented for passivation of stainless steel and titanium. The mechanism is discussed in terms of various oxidizing agents including oxygen, chromate, molybdate, and pertechnetate.
Some theories I could find for passivation are Kinetic Theory of Inhibition and Passivation in Electrochemical Reactions and Scaling theory in a model of corrosion and passivation
