For instance, the potential needed for water splitting is $\pu{1.23V}$. However, there is current flow even at lower potentials (say $\pu{0.5V}$). Clearly, there is no redox reaction happening in the cell as evidenced by lack of bubbles. What is the cause of this current?
As you noticed there is a small amount of current which keeps on flowing when there is no electrolytic decomposition. This "problem" is aptly named as non-Faradaic current. Faradaic current refers to current due to oxidation and reduction. Here is a detailed review: The difference between Faradaic and Nonfaradaic processes in Electrochemistry