The configuration you describe is the ground state configuration, the configuration of oxygen in which all electrons are in their lowest allowed energy states.
Substances can also be their excited states, where electrons are not in the lowest allowed energy states. Excited states happen when atoms or molecules absorb energy, usually electromagnetic radiation (but not always).
An excited state for oxygen may involve one of the $2s$ electrons being promoted to one of the remaining $2p$ spots:
$$\begin{aligned}
\ce{O} + E &\ce{-> O}^* \\
1s^2 2s^2 2p^4 &\ce{-> }1s^2 2s^1 2p^5
\end{aligned}$$
This configuration still describes an oxygen atom, but now the $2s$ subshell is partially empty.
The other reason you can't always assume that the lower subshells are always filled is that there are some exceptions to the aufbau principle. Some elements do not follow it, especially the lanthanides and actinides.
- Chromium: $\ce{Cr} - \ce{[Ar]}4s^1 3d^5$ ($s$ is not filled so that $d$ can be half-full)
- Copper: $\ce{Cu} - \ce{[Ar]}4s^1 3d^{10}$ ($s$ is not filled so that $d$ can be full)
- Palladium: $\ce{Pd} - \ce{[Kr]}4d^{10}$ ($s$ is empty so that $d$ can be full)
- Cerium: $\ce{Ce} - \ce{[Xe]} 6s^2 5d^1 4f^1$ ($f$-block elements are weird)