Yes, what you are describing is a system approaching thermal equilibrium.
The temperature of the water approaches the temperature of it's surroundings as heat is transferred from the warmer object (either the air or the water) to the colder. This is basically true of any two objects in contact with each other; the temperature difference drives the flow of heat from the warmer object to the colder one. At first, when the temperature difference is large, this happens quickly, then more slowly as the difference in temperature gets small.
One complication to this simple idea is that the water is also evaporating, which is a cooling process. Particularly if the surrounding air is dry, the water can evaporate quickly enough to become a fair bit colder than it's surroundings.