"Real" crystals do not grow by stacking unit cells, but by atoms, ions or molecules adsorbing to the surface of the existing crystallite nucleus.
The different possible surfaces of the crystal (those that are different, depending on orientation towards the unit cell (symmetry!) and current outer layer) each have some probability to start growing (first adsorption into a new plane) and some other to keep growing.
These probabilities (i.e. growth rates) depend on kinetics and thermodynamics, and thus are somewhat hard to predict. They also depend not only on the temperature and pressure, but also on the composition of the solution or melt. Their ratios dictate the macroscopic form of the crystal that grows.
Note that there are a lot more planes in a single crystal than the unit cell has outer faces. Typically the very "odd" planes are not so favourable for growth, but that is the reason why a crystal does not have to resemble the form of its unit cell.