As you add $\rm AgNO_3$ to $\rm KCl$, solid $\rm AgCl$ forms. The silver ion you add will at first be completely consumed by the chloride. Once all of the chloride is locked up in the precipitate, the silver ion concentration will rapidly increase. The Nernst equation shows how the potential is expected to change when this happens. If you're using a silver/silver chloride electrode with a reference electrode, the indicator potential is
$$\rm E = E^\circ_{\rm AgCl,Ag} - \frac{R T}{\cal F}\ln\frac{K_{sp}}{a_{\rm Ag^+}}$$
To see whether you'll get a drop or a rise in $E$, ask yourself what happens when silver concentration rises. If you've got the leads hooked up to your multimeter properly (negative lead to the silver cathode, positive lead to your anode), shouldn't you be seeing an increase in potential? What exactly are your $x$ and $y$ axes on that graph?
You'll get a bigger change when you form $\rm AgI$, because $\rm AgCl$ is about a thousand times more soluble than $\rm AgI$. The $\rm AgI$ will lock up relatively more ions when it forms (the $K_{sp}$'s of $\rm AgCl$ and $\rm AgI$ are $1.8\times 10^{-10}$ and $1.5\times 10^{-16}$, respectively).
Addendum 1: You say you're just using a silver electrode; the indicator potential is
$$\rm E = E^\circ_{\rm Ag^+,Ag} - \frac{R T}{\cal F}\ln\frac{1}{a_{\rm Ag^+}}$$
(but if you dip a silver wire into a chloride solution you'll get a very thin layer of AgCl and it'll begin to be a silver/silver chloride electrode (Source: G. D. Christian, Analytical Chemistry, 4th ed., Wiley 1986, p. 282)).
You also say you don't have a true reference electrode, just a graphite electrode... the fact that its potential can change as well complicates things. Your cell potential will depend on what the oxidation process is at the anode.
Addendum 2: You asked for a source for the $K_{sp}$'s. CRC Handbook 58th edition gives $K_{sp}$'s at $25^\circ$C for AgCl and AgI as $1.56\times 10^{-10}$ and $1.5\times 10^{-16}$; the 84th edition gives $1.77\times 10^{-10}$ and $8.52\times 10^{-17}$; Keller's Basic Tables in Chemistry gives them as $1.0\times 10^{-10}$ and $1.0\times 10^{-16}$ for "temperatures near 298 K". They're very temperature dependent and they depend on conditions like ionic strength as well, so it's not surprising that the values you're finding on the web are all over the place. I took the values above from a genchem text that happened to be lying on my desk.