I agree with you that generally, vapor pressure of a solution depends on, not only concentrations of its ingredients, but also on their chemical composition.
Addition of ethanol to water will decrease the boiling point of solution relative to water. While, adding acetic acid will increase the boiling point of the solution. One may think a solute that is more volatile than water(as a solvent), will always decrease the solution boiling point. This is what raoult's law predicts. But, this isn't always true and depends on the ideality of the solution. For example, consider Hydrochloric acid. The boiling point of liquid Hydrogen chloride is -85 degrees of centigrade and is more volatile than water. But after it dissolves in water, the boiling point of solution can even increases up to 108 degrees(in some concentrations) This is because the interactions between particles in Hydrochloric acid is stronger than pure water. This stronger bonding arises from dissociation of Hydrogen chloride molecules into ions. When the intramolecular forces and sizes of the components in a solution are near to each other the solution behaves more ideally. One example is a solution of n-pentane and n-hexane.
In addition, when a solution is diluted the solution becomes more ideal.
Back to your question, that wondering about vapor pressure of a solution to be a colligative property.
I think the vapor pressure in solutions that, the solute is non-volatile may only depend on solute concentration. If so, in this case the vapor pressure of the solution is a colligative property. But if the solution behaves ideally. Let me explain the reason for it.
At the surface of the solution, a number of solvent particles are exchanged by solute particles. In an ideal solution, the solute particles bond to solvent molecules, just as solvent molecules bond to each other (same intramolecular forces and molecular size). So regardless of the identity of desired solutes, the chance of solvent molecules to escape from the surface will decrease with respect to concentrations of solute. As the solute is non-volatile, the total vapor pressure is equal to solvent vapor pressure. And, that can be predicted by only the concentration of the solute(a colligative property). Conceptually, the ideality of the solution is an important factor in the answer of your question. Ideality is not an absolute property of a solution. Any solution can be considered to be non-ideal with sufficient precision in measurement. Thanks.
